> ## Documentation Index
> Fetch the complete documentation index at: https://docs.automq.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy AutoMQ Software Via Helm Chart

> Deploy AutoMQ Software in a private Kubernetes data center using Helm Charts. Achieve 100% Kafka compatibility and cost efficiency with step-by-step setup instructions.

This document details the process for deploying AutoMQ Software in a private Kubernetes data center using a Helm Chart. To deploy AutoMQ in a public cloud environment, refer to [Overview▸](/automq-cloud/getting-started/overview).

## Prerequisites

Before installing AutoMQ with a Helm Chart, ensure the following prerequisites are satisfied:

1. **Prepare a Kubernetes Environment**: Establish an available Kubernetes cluster in advance, ensuring it meets the conditions below:

   1. **Allocate Resources for AutoMQ**: It is recommended to allocate 4 cores and 16GB of memory for each AutoMQ Pod. Deploying on a dedicated Node is advisable for stable network throughput performance.

   2. **Storage Plugin:** If your Kubernetes is provided by a cloud vendor, it is advisable to install the storage plugin offered by the vendor to manage EBS volume resources effectively.

2. **Prepare Object Storage Buckets:** Each AutoMQ cluster requires two separate object storage buckets: one Ops Bucket for system logs and metrics data, and one Data Bucket for message data. Please refer to the object storage product documentation for guidance on creating them.

3. **Install the Helm Chart Tool:** It is recommended to install version 3.6 or higher. You can [refer to the documentation](https://helm.sh/docs/intro/quickstart/) for detailed instructions.

## Obtain the Software Chart.

The AutoMQ Software Chart image is published and made available to the public through an Azure Container Registry (East US). You can test the pull with the following command.

```bash theme={null}
helm pull oci://automq.azurecr.io/helm/automq-enterprise-chart --version 5.3.4
```

## Install AutoMQ

AutoMQ Software offers two types of WAL storage options: EBSWAL and S3WAL. A comparison of the two storage engines is as follows; it is recommended to choose based on your needs. For detailed principles, please refer to the [Technical Architecture](https://docs.automq.com/automq/architecture/s3stream-shared-streaming-storage/wal-storage).

* **EBSWAL Mode:** WAL storage uses high-speed EBS volumes to deliver low-latency send performance, supported only in public cloud environments like AWS, GCP, and Azure. When using, you need to assign EBS volumes to AutoMQ's Pods via a StorageClass.

* **S3WAL Mode:** Deployment is relatively simple, as WAL storage writes directly to object storage, offering sub-100 ms send RT performance. It supports all public cloud environments as well as private data centers (as long as they provide S3-compatible object storage). Deployment is relatively straightforward, with no need to allocate EBS volumes.

<Tip>
  The following sections provide a simplified deployment scenario example for various cloud vendor environments. This example scenario deploys in S3WAL mode, uses static credentials to access cloud resources, and supports accessing AutoMQ within a Kubernetes cluster via a Headless service access point or from outside the Kubernetes cluster using a LoadBalancer access point.

  The AutoMQ team also provides a variety of advanced feature examples, including TLS, authentication and authorization, Auto-Scaler, and more. For details, please refer to the [Advanced Feature Examples ▸](https://github.com/AutoMQ/automq-labs/tree/main/software-examples/kubernetes/).
</Tip>

### Step 1: Create Credentials and Perform Authorization.

AutoMQ clusters require access to external services such as object storage and storage volumes. Therefore, before installation, you need to create credentials for AutoMQ and complete the authorization process.

<Tabs>
  <Tab title="AWS">
    If AutoMQ is deployed in the AWS public cloud environment using AWS S3 storage, you must access the IAM product to create an authorization policy. AutoMQ must be granted permission for the following operations to access AWS S3:

    ```yaml theme={null}
    - actions:
        - s3:GetLifecycleConfiguration
        - s3:PutLifecycleConfiguration
        - s3:ListBucket
        - s3:PutObject
        - s3:GetObject
        - s3:AbortMultipartUpload
        - s3:PutObjectTagging
        - s3:DeleteObject

    ```

    If you deploy using the EBSWAL mode, additional authorization for the following policy is required:

    ```yaml theme={null}
    - actions:
        - ec2:DescribeVolumes
        - ec2:DetachVolume
        - ec2:DescribeAvailabilityZones

    ```

    After creating an IAM authorization policy, credentials can be generated using two methods.

    * **Using IAM Subaccount Static AccessKey:** In this approach, attach the authorization policy to the IAM subaccount and utilize the subaccount's static AccessKeyId and AccessKeySecret as credentials to access AutoMQ.

    * **Using IAM Role Dynamic Credentials:** For this approach, create an IAM Role and attach the authorization policy to the Role. Dynamic credentials allow access to AutoMQ through a Pod assuming the EC2 Role in EKS.

    <Tip>
      To clarify illustrations, the following configuration file example employs a static AccessKey as credentials.
    </Tip>
  </Tab>

  <Tab title="Azure">
    Deploying AutoMQ in the Azure public cloud environment requires using Azure Blob Storage. Authorization credentials must be configured within the Storage Account's access control (IAM). Users can opt for either of the following permission entities to assign the system role to Blob: `Storage Blob Data Owner`.

    * **Using Service Principal Static Secret**: In this setup, the AccessKey corresponds to the App's clientId, and the SecretKey corresponds to the Secret value along with the tenantId. Together, these serve as static credentials for accessing AutoMQ.

    * **Using Managed Identity Dynamic Credentials**: With this approach, in addition to having the ServiceAccount assigned the necessary roles, it is vital to associate it with the corresponding VMSS (virtual machine scale set) for AutoMQ. This way, you can access AutoMQ on AKS using the clientId of this dynamic credential.
  </Tab>

  <Tab title="OCI">
    In an OCI environment, you need to generate a **Customer Secret Key** for a specific OCI user to serve as static credentials. This user must be granted appropriate permissions to access Object Storage.

    First, create the following IAM authorization policies for the user or user group managing AutoMQ:

    ```text theme={null}
    Allow group <your-group-name> to manage objects in compartment <your-compartment> where any {
      target.bucket.name = '<your-data-bucket>',
      target.bucket.name = '<your-ops-bucket>'
    }

    Allow group <your-group-name> to use buckets in compartment <your-compartment> where any {
      target.bucket.name = '<your-data-bucket>',
      target.bucket.name = '<your-ops-bucket>'
    }
    ```

    * The `manage objects` permission set includes operations like creating, reading, deleting, and listing objects.
    * The `use buckets` permission set includes permissions to manage bucket lifecycle policies.

    If you plan to use **EBSWAL** mode, OKE worker nodes require permissions to manage Block Volumes. Ensure you create a Dynamic Group for your node pool and grant it the following permissions:

    ```text theme={null}
    Allow dynamic-group <your-node-dynamic-group> to manage block-volumes in compartment <your-compartment>
    Allow dynamic-group <your-node-dynamic-group> to manage volume-attachments in compartment <your-compartment>
    Allow dynamic-group <your-node-dynamic-group> to use subnets in compartment <your-compartment>
    Allow dynamic-group <your-node-dynamic-group> to inspect instances in compartment <your-compartment>
    ```

    After creating the credentials, record the **Access Key** and **Secret Key**, as they will be used as static credentials in the subsequent Helm Chart configuration.
  </Tab>
</Tabs>

### Step 2: Create Storage Class

Before installing AutoMQ, you must declare a Storage Class in the Kubernetes cluster for allocating storage volumes. These storage volumes serve several purposes:

* **Storing AutoMQ Controller Metadata:** In the AutoMQ cluster, the Controller Pod responsible for metadata management must mount the storage volumes to store KRaft metadata.

* **EBSWAL Mode Storage for WAL Data (Optional):** If you plan to deploy using the EBSWAL mode, each Broker Pod will also require a mounted data volume for writing WAL data.

<Tip>
  Please specify the Storage Class based on the Kubernetes storage plugin from your cloud provider or private data center, then keep a record of the Storage Class name for later parameter configuration.
</Tip>

<Tabs>
  <Tab title="AWS">
    ```yaml theme={null}
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: automq-disk-eks-gp3
    provisioner: ebs.csi.aws.com
    volumeBindingMode: WaitForFirstConsumer
    parameters:
      type: gp3  # EBS Volume Type
    allowVolumeExpansion: true

    ```
  </Tab>

  <Tab title="Azure">
    ```yaml theme={null}
    # Example StorageClass Definition (Azure Premium SSD Example)
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: automq-disk-storage-class
    provisioner: disk.csi.azure.com
    parameters:
      skuName: PremiumV2_LRS
    reclaimPolicy: Delete
    volumeBindingMode: WaitForFirstConsumer
    allowVolumeExpansion: true

    ```
  </Tab>

  <Tab title="GCP">
    ```yaml theme={null}
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: automq-disk-gcp-pd-balanced
    provisioner: pd.csi.storage.gke.io
    volumeBindingMode: WaitForFirstConsumer
    allowVolumeExpansion: true
    parameters:
      type: pd-balanced  # High Performance Cloud Disk.

    ```
  </Tab>

  <Tab title="OCI">
    ```yaml theme={null}
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: automq-block-volume
    provisioner: blockvolume.csi.oraclecloud.com
    reclaimPolicy: Delete
    volumeBindingMode: WaitForFirstConsumer
    allowVolumeExpansion: true
    parameters:
      # vpusPerGB determines the IOPS and throughput performance of the volume
      # 10: Provides basic performance
      # 20: Provides higher performance, recommended for production environments
      vpusPerGB: "20"
      # attachmentType determines the type of volume attachment
      # iscsi: iSCSI attachment, general purpose
      # paravirtualized: Paravirtualized attachment, offers higher performance
      attachmentType: "paravirtualized"
    ```
  </Tab>
</Tabs>

### Step 3: Initialize the Configuration File

The configuration information for the AutoMQ Software Chart is composed of multiple parts, allowing for user customization via the values.yaml file.

<Tip>
  **Required: Set a Unique Instance ID**

  A critical parameter you must define is `global.automqInstanceId`. This serves as a **required** unique identifier for your AutoMQ cluster.

  Why is it important? This ID ensures that resources are properly isolated, preventing data corruption or conflicts. Each cluster must have its own distinct `automqInstanceId`.
</Tip>

First, create an empty file named `automq-values.yaml`. You can copy the example configuration below and edit it.

```yaml theme={null}
global:
  cloudProvider:
    name: "Replace With Your True Cloud Provider Name"
    credentials: "Replace With Your True Your Credentials"
  config: |
    s3.ops.buckets=Replace With Your True Ops Bucket URL
    s3.data.buckets=Replace With Your True Data Bucket URL
    s3.wal.path=Replace With Your True WAL PATH

controller:
  resources:
    requests:
      cpu: "3000m"
      memory: "12Gi"
    limits:
      cpu: "4000m"
      memory: "16Gi"
  persistence:
    metadata:
      storageClass: "Replace With Your True StroageClass"
    wal:
      enabled: false

  annotations:

  env:
    - name: "KAFKA_JVM_PERFORMANCE_OPTS"
      value: "-server -XX:+UseZGC -XX:ZCollectionInterval=5"
    - name: "KAFKA_OPTS"
      value: "-XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError"
    - name: "KAFKA_HEAP_OPTS"
      value: "-Xmx6g -Xms6g -XX:MaxDirectMemorySize=6g -XX:MetaspaceSize=96m"
    - name: "KAFKA_S3_ACCESS_KEY"
      value: "Replace With Your True ACCESS_KEY"
    - name: "KAFKA_S3_SECRET_KEY"
      value: "Replace With Your True SECRET_KEY"

broker:
  replicas: 0
  resources:
    requests:
      cpu: "3000m"
      memory: "12Gi"
    limits:
      cpu: "4000m"
      memory: "16Gi"

  persistence:
    wal:
      enabled: false

  annotations:

  env:
    - name: "KAFKA_JVM_PERFORMANCE_OPTS"
      value: "-server -XX:+UseZGC -XX:ZCollectionInterval=5"
    - name: "KAFKA_OPTS"
      value: "-XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError"
    - name: "KAFKA_HEAP_OPTS"
      value: "-Xmx6g -Xms6g -XX:MaxDirectMemorySize=6g -XX:MetaspaceSize=96m"
    - name: "KAFKA_S3_ACCESS_KEY"
      value: "Replace With Your True ACCESS_KEY"
    - name: "KAFKA_S3_SECRET_KEY"
      value: "Replace With Your True SECRET_KEY"
externalAccess:
  controller:
    enabled: true
    service:
      type: LoadBalancer
      annotations:
        <Add Cloud Provider Specific Annotations Here>

      extraPorts:
        - name: "tcp-9092"
          port: 9092
          protocol: "TCP"
          targetPort: 9092
```

In the configuration file generated in the previous step, certain parameters must be updated according to your actual conditions.

#### Modify Common Parameters.

**global.cloudProvider.name**

This parameter specifies the deployment cloud environment. Please insert the enumerated value according to the name of the cloud provider. If it is a private data center, you'll also need to fill it in with the enumerated value.

| **Deployment Environment**<br /> | **Parameter Enumerated Value**<br /> |
| -------------------------------- | ------------------------------------ |
| AWS<br />                        | aws<br />                            |
| Google Cloud<br />               | gcp<br />                            |
| Azure<br />                      | azure<br />                          |
| OCI<br />                        | noop<br />                           |
| Alibaba Cloud<br />              | aliyun<br />                         |

**global.cloudProvider.credentials**

This parameter details the public credentials used by the AutoMQ cluster to access cloud resources. The current example utilizes static credentials of the AccessKey type. To use the IAM Role method, please refer to the advanced parameter documentation for guidance on modifications.

```yaml theme={null}
global:
  cloudProvider:
    credentials: static://?accessKey=<your-accesskey>&secretKey=<your-secretkey>


```

**KAFKA\_S3\_ACCESS\_KEY and KAFKA\_S3\_SECRET\_KEY Environment Variables**

The example configuration file employs static credentials of the AccessKey type. Therefore, in addition to the global.cloudProvider.credentials parameter, you need to update the Controller and Broker environment variables with the correct credentials.

Refer to the credentials created in Step 1 to update the credentials in the example:

```yaml theme={null}
controller:
  env:
    - name: "KAFKA_S3_ACCESS_KEY"
      value: "Replace With Your True ACCESS_KEY"
    - name: "KAFKA_S3_SECRET_KEY"
      value: "Replace With Your True SECRET_KEY"

broker:
  env:
    - name: "KAFKA_S3_ACCESS_KEY"
      value: "Replace With Your True ACCESS_KEY"
    - name: "KAFKA_S3_SECRET_KEY"
      value: "Replace With Your True SECRET_KEY"

```

**global.config**

This parameter specifies the configuration for accessing object storage using S3URL, which includes three components: s3.ops.buckets, s3.data.buckets, and s3.wal.path.

<Tip>
  Below is an example configuration using the S3WAL mode with static credentials. If you need to use EBSWAL mode, please refer to the advanced configuration section for modification instructions.
</Tip>

<Tabs>
  <Tab title="AWS">
    Fill in the Ops Bucket and Data Bucket created in the prerequisites according to your actual scenario.

    ```yaml theme={null}
    config: |
      s3.data.buckets=0@s3://<your-data-bucket>?region=xxxx&endpoint=https://s3.xxxx.amazonaws.com&authType=static
      s3.ops.buckets=1@s3://<your-ops-bucket>?region=xxxx&endpoint=https://s3.xxxx.amazonaws.com&authType=static
      s3.wal.path=0@s3://<your-data-bucket>?region=xxxx&endpoint=https://s3.xxxx.amazonaws.com&authType=static

    ```
  </Tab>

  <Tab title="Azure">
    Fill in the Ops Bucket, Data Bucket, region, and Endpoint parameters according to your actual scenario.

    ```yaml theme={null}
    config: |
      s3.data.buckets=0@s3://<your-data-bucket>?region=xxxx&endpoint=https://xxx_storage-account.blob.core.windows.net&authType=static
      s3.ops.buckets=1@s3://<your-ops-bucket>?region=xxxx&endpoint=https://xxx_storage-account.blob.core.windows.net&authType=static
      s3.wal.path=1@s3://<your-data-bucket>?region=xxxx&endpoint=https://xxx_storage-account.blob.core.windows.net&authType=static

    ```
  </Tab>

  <Tab title="Google Cloud">
    AccessKey-type static credentials are not supported on GCP, requiring a different format. You need to fill in the Ops Bucket, Data Bucket, region, Endpoint, and GCP ServiceAccount parameters.

    ```yaml theme={null}
    config: |
      s3.data.buckets=0@s3://<your-data-bucket>?region=xxxx&endpoint=https://xxx_storage-account.blob.core.windows.net&authType=instance&role=<role-id>
      s3.ops.buckets=1@s3://<your-ops-bucket>?region=xxxx&endpoint=https://xxx_storage-account.blob.core.windows.net&authType=instance&role=<role-id>
      s3.wal.path=1@s3://<your-data-bucket>?region=xxxx&endpoint=https://xxx_storage-account.blob.core.windows.net&authType=instance&role=<role-id>

    ```
  </Tab>

  <Tab title="OCI">
    Fill in the Ops Bucket, Data Bucket, region, and Endpoint parameters according to your actual scenario.

    ```yaml theme={null}
    config: |
      s3.data.buckets=0@s3://<your-data-bucket>?region=xxxx&endpoint=https://xxx.compat.objectstorage.us-ashburn-1.oraclecloud.com&authType=static&pathStyle=true
      s3.ops.buckets=1@s3://<your-ops-bucket>?region=xxxx&endpoint=https://xxx.compat.objectstorage.us-ashburn-1.oraclecloud.com&authType=static&pathStyle=true
      s3.wal.path=0@s3://<your-data-bucket>?region=xxxx&endpoint=https://xxx.compat.objectstorage.us-ashburn-1.oraclecloud.com&authType=static&pathStyle=true
    ```
  </Tab>

  <Tab title="Alibaba Cloud">
    Input the Ops Bucket, Data Bucket, region, and Endpoint parameters according to the actual requirements.

    ```yaml theme={null}

    config: |
      s3.data.buckets=0@s3://<your-data-bucket>?region=xxxx&endpoint=https://oss-cn-xxxx.aliyuncs.com&authType=static
      s3.ops.buckets=1@s3://<your-ops-bucket>?region=xxxx&endpoint=https://oss-cn-xxxx.aliyuncs.com&authType=static
      s3.wal.path=1@s3://<your-data-bucket>?region=xxxx&endpoint=https://oss-cn-xxxx.aliyuncs.com&authType=static

    ```
  </Tab>
</Tabs>

#### Set LoadBalancer Annotations to Enable External Kubernetes Cluster Access

If you need to access AutoMQ from outside the Kubernetes cluster, you need to enable `externalAccess`. To configure an **internal LoadBalancer**, you should modify the `values.yaml` file, specifically the `externalAccess.controller.service.loadBalancerAnnotations` section, and add the following annotations based on your cloud provider:

<Tabs>
  <Tab title="AWS">
    To create an internal Network Load Balancer (NLB), add the following annotations:

    ```yaml theme={null}
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
    # If you need to specify subnets, add:
    # service.beta.kubernetes.io/aws-load-balancer-subnets: "subnet-xxxx,subnet-yyyy"
    ```
  </Tab>

  <Tab title="Azure">
    To create an internal standard Load Balancer, add the following annotations:

    ```yaml theme={null}
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    # If you need to specify a subnet, add:
    # service.beta.kubernetes.io/azure-load-balancer-internal-subnet: "your-subnet-name"
    ```
  </Tab>

  <Tab title="GCP">
    To create an internal TCP/UDP Load Balancer, add the following annotations:

    ```yaml theme={null}
    networking.gke.io/load-balancer-type: "Internal"
    # If you need to specify a subnet, add:
    # cloud.google.com/load-balancer-subnet: "your-subnet-name"
    ```
  </Tab>

  <Tab title="OCI">
    To create an internal Load Balancer, add the following annotations:

    ```yaml theme={null}
    oci.oraclecloud.com/load-balancer-type: "nlb"
    oci-network-load-balancer.oraclecloud.com/internal: "true"
    oci-network-load-balancer.oraclecloud.com/subnet: "<Replace With Your True Subnet OCID>"
    ```
  </Tab>
</Tabs>

**controller.persistence.metadata.storageClass**

Substitute this parameter with the name of the Storage Class created in step 2, which is designated for storing metadata in the AutoMQ Controller Pod.

#### Revise the Cluster Topology and Resource Request Parameters.

Adjust the cluster topology and resource request parameters based on the resources allocated to AutoMQ Node. The parameters that need modification are as follows:

**broker.replicas**

The AutoMQ Software Chart will start with three Controller Pods by default. These Controller Pods also provide data read and write capabilities. If users wish to horizontally scale more Brokers, they can set the broker.replicas parameter.

* Default value: 0, which represents a three-node cluster without the need for additional Brokers.

* Setting range: >= 0, configured as needed.

**Resource Request Parameters**

AutoMQ Software Controller and Broker Pods need to have the Request and Limit parameters adjusted, along with the corresponding JVM HEAP settings. The configuration files mentioned earlier default to a 4Core16GB specification. Please update these parameters based on the actual allocated computing resources.

* controller.resources.requests.cpu

* controller.resources.requests.memory

* controller.resources.limits.cpu

* controller.resources.limits.memory

* controller.env.\[KAFKA\_HEAP\_OPTS]

* broker.resources.requests.cpu

* broker.resources.requests.memory

* broker.resources.limits.cpu

* broker.resources.limits.memory

* broker.env.\[KAFKA\_HEAP\_OPTS]

### Step 4: Install Chart and Access the Cluster

After customizing the values.yaml configuration file to suit your deployment requirements, proceed with the installation of AutoMQ.

```bash theme={null}

helm upgrade --install <release-name> oci://automq.azurecr.io/helm/automq-enterprise --version 5.3.4 -f <your-custom-values.yaml> --namespace <namespace> --create-namespace

```

Once the installation is complete, users can access AutoMQ within the Kubernetes cluster, either through a Headless service or a LoadBalancer.

<Tip>
  Note: We recommend deploying an Internal LoadBalancer to prevent changes in Pod IP addresses.
</Tip>

### Step 5: Connect and Test the Cluster

#### Headless Service

1. **Locate the Headless service**

```bash theme={null}

kubectl get svc --namespace <namespace> -l "app.kubernetes.io/component=controller" -w

```

2. **Connecting and Testing Using Kafka Clients**

Use the Headless Service for your Kafka client’s `--bootstrap-server` option to send and receive messages. Here’s the command you can use:

```bash theme={null}

./kafka-console-producer.sh \
  --bootstrap-server <release-name>-automq-enterprise-controller-0.<release-name>-automq-enterprise-controller-headless.<namespace>.svc.cluster.local:9092 \
  --topic test-topic

```

#### LoadBalancer

1. **Find External Address**

Wait for the EXTERNAL-IP to be assigned. Use the following command:

```bash theme={null}

kubectl get svc automq-release-automq-enterprise-controller-loadbalancer -n automq -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

```

You can obtain the external IP of the LoadBalancer.

2. **Connect and Test Using Kafka Clients**

Port `9092` is used for client access.

```bash theme={null}

# Replace <EXTERNAL-IP> with the Address from the Previous Step
./kafka-console-producer.sh \
  --bootstrap-server <EXTERNAL-IP>:9092 \
  --topic test-topic

```

## Other Advanced Configurations

The deployment document above provides a basic example of deploying AutoMQ in S3WAL mode. In real-world production environments, users can choose more advanced configurations like EBSWAL and integrate Auto-Scaler support. For the full configuration file, refer to [Helm Chart Values Readme▸](/automq-cloud/appendix/helm-chart-values-readme).

### Configuring the WAL Type

In the previously mentioned installation steps, S3WAL was used as an example. AutoMQ supports deployment options for both EBSWAL and S3WAL modes.

<Tabs>
  <Tab title="S3WAL Mode">
    In S3WAL mode, there's no need to mount a WAL data volume, making the configuration relatively straightforward. First, configure the `global.config.s3.wal.path` parameter.

    ```yaml theme={null}

    config: |
      s3.wal.path=0@s3://<xxx-data-bucket>?region=<region>&endpoint=<endpoint>&authType=<authType>

    ```

    Then, disable `controller.persistence.wal.enabled` and `broker.persistence.wal.enabled`.

    ```yaml theme={null}
    # Applying StorageClass in Controller/broker
    controller:
      persistence:
        metadata:
          storageClass: "your-storage-class"
        wal:
          enabled: false
    broker:
      persistence:
        wal:
          enabled: false


    ```
  </Tab>

  <Tab title="EBSWAL Mode">
    In EBSWAL mode, each controller and broker requires a mounted WAL data volume. Therefore, you need to set `controller.persistence.wal.storageClass` and `broker.persistence.wal.storageClass`, ensuring these values are set to the previously created Storage Class.

    ```yaml theme={null}
    # Applying StorageClass in Controller/broker
    controller:
      persistence:
        metadata:
          storageClass: "your-storage-class"
        wal:
          storageClass: "your-storage-class"
    broker:
      persistence:
        wal:
          storageClass: "your-storage-class"


    ```

    Next, configure the `global.config.s3.wal.path` parameter.

    ```yaml theme={null}
    config: |
      s3.wal.path=0@block:///dev/waliobandwidth=131072000&iodepth=8&iops=3000&capacity=2147483648

    ```
  </Tab>
</Tabs>

### Setting Credentials

AutoMQ supports accessing external resources using either static AccessKeys or dynamic IAM Roles. To prevent the leakage of static AccessKey configurations in production environments, it is recommended to use dynamically generated credentials provided by the cloud provider's IAM Roles.

<Tabs>
  <Tab title="IAM Role Credentials">
    When using IAM Role Credentials, it is necessary to attach the authorization policy to the Role in Step 1. Then, refer to the example below to modify the Credentials configuration.

    ```yaml theme={null}
    global:
      cloudProvider:
        credentials: instance://?role=<your-instance-profile>

      config: |
        s3.data.buckets=0@s3://<your-bucket>?authType=instance&role=<role-id>
        s3.ops.buckets=1@s3://<your-bucket>?authType=instance&role=<role-id>
    ```

    The format for filling out credentials parameters is outlined in the following table:\*\*

    | **Deployment Environment** | **Parameter Values**                                                                                                 |
    | -------------------------- | -------------------------------------------------------------------------------------------------------------------- |
    | AWS<br />                  | instance://?role=\<your-instance-profile><br />For Role, enter the IAM instance profile, not the Role ARN.<br />     |
    | Google Cloud<br />         | instance://?role=\<your-service-account-name><br />For Role, enter the name of the GCP ServiceAccount.<br />         |
    | Azure<br />                | instance://?role=\<your-managed-identity-client-id><br />For Role, enter the Azure Managed Identity Client ID.<br /> |
    | Alibaba Cloud<br />        | instance://?role=\<your-role-id><br />For Role, enter the RAM Role name of Alibaba Cloud.<br />                      |
  </Tab>

  <Tab title="AccessKey Credentials">
    When using AccessKey Credentials, you need to attach the authorization policy to the sub-account or service account in Step 1. Then modify the Credentials configuration as shown in the example below, setting authType to static.

    ```yaml theme={null}
    global:
      cloudProvider:
        credentials: static://?accessKey=<your-accesskey>&secretKey=<your-secretkey>

      config: |
        s3.data.buckets=0@s3://<your-bucket>?authType=static
        s3.ops.buckets=1@s3://<your-bucket>?authType=static
    ```

    After configuring static AccessKey credentials, you must also set these credentials in the environment variables of the Controller and Broker as required by your cloud provider.

    ```yaml theme={null}
    controller:
      env:
        - name: "KAFKA_S3_ACCESS_KEY"
          value: "Replace With Your True ACCESS_KEY"
        - name: "KAFKA_S3_SECRET_KEY"
          value: "Replace With Your True SECRET_KEY"

    broker:
      env:
        - name: "KAFKA_S3_ACCESS_KEY"
          value: "Replace With Your True ACCESS_KEY"
        - name: "KAFKA_S3_SECRET_KEY"
          value: "Replace With Your True SECRET_KEY"

    ```

    <Tip>
      If deploying in Azure, you need to set an additional environment variable: AZURE\_TENANT\_ID, which corresponds to the current tenant ID (tenantId).
    </Tip>
  </Tab>
</Tabs>

### Set Fine-grained Scheduling Policies

In Kubernetes, AutoMQ's fine-grained scheduling policy is implemented using node affinities and tolerations. Users are advised to customize label matching rules based on their node types:

#### Tolerations

It's recommended to add a taint to the Kubernetes node group with the key "dedicated," operator "Equal," value "automq," and effect "NoSchedule." Then, configure the corresponding toleration rules in global.tolerations to schedule Pods:

```yaml theme={null}
global:
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "automq"
    effect: "NoSchedule"

```

#### Node Affinities

Override default values in the controller/agent configuration to align with node labels (e.g., node-type: automq-worker):

```yaml theme={null}
controller:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: "node-type"
          operator: In
          values: ["automq-worker"]

```

### Set up Auto-scaling

#### Number of Controllers

By default, the cluster deploys 3 Controller Pods, but users can customize the number of Controller replicas.

<Danger>
  Note: Once the cluster is deployed, adjusting the number of Controller replicas is not supported to avoid unforeseen risks.
</Danger>

### Number of Brokers

The number of brokers is managed by the `broker.replicas` parameter, which allows for horizontal scaling. By default, there are 0 brokers.

### Auto-scaling Configuration

By default, HPA (Horizontal Pod Autoscaler) is disabled. To activate it, two conditions must be fulfilled:

* broker.replicas > 0

* Enable and configure parameters in `global.autoscaling.hpa`:

```yaml theme={null}
global:
  autoscaling:
    hpa:
      enabled: true        # Enable HPA
      minReplicas: "1"     # Minimum Replicas
      maxReplicas: "3"     # Maximum Replicas
      targetCPU: "60"      # Target CPU Utilization (%)
      targetMemory: ""     # Target Memory Utilization (%) (optional)

```

### Identity Recognition Configuration

AutoMQ allows overriding of protocol listeners and enabling secure authentication. By default, it uses the following ports:

* Client to server access: 9092 (PLAINTEXT).

* Internal communication between Controllers: 9093 (PLAINTEXT).

* Internal communication between Brokers: 9094 (PLAINTEXT).

AutoMQ also enables secure authentication by configuring listener overrides (e.g., enabling SASL authentication) for custom ports and protocols. Allowed values include 'PLAINTEXT', 'SASL\_PLAINTEXT', 'SASL\_SSL', and 'SSL'.

```yaml theme={null}
listeners:
  client:
    - containerPort: 9092
      protocol: SASL_PLAINTEXT
      name: BROKER_SASL
  controller:
    - containerPort: 9093
      protocol: SASL_PLAINTEXT
      name: CONTROLLER_SASL
  interbroker:
    - containerPort: 9094
      protocol: SASL_PLAINTEXT
      name: BROKER_SASL

```

<Tip>
  Additionally, you can set a password for it, which is randomly generated by default.
</Tip>

```yaml theme={null}
sasl:
  controller:
    user: "user1"
    password: "PWDxxx"
  interbroker:
    user: "user2"
    password: "PWDxxx"
  client:
    user: "user3"
    password: "PWDxxx"

```

### Configure Prometheus RemoteWrite Metrics Integration

AutoMQ Server supports pushing cluster metrics directly to a user-defined Prometheus instance via the Prometheus RemoteWrite protocol. This approach eliminates the need to deploy additional Prometheus scraping components in the Kubernetes cluster, simplifying the monitoring architecture.

Configure the `s3.telemetry.metrics.exporter.uri` parameter in `global.config` within your `values.yaml` to enable this feature. Choose the appropriate configuration format based on your Prometheus endpoint's authentication method:

<Tabs>
  <Tab title="No Auth">
    For Prometheus endpoints that do not require authentication:

    ```yaml theme={null}
    global:
      config: |
        s3.telemetry.metrics.exporter.uri=rw://?endpoint=<your-prometheus-endpoint>
    ```
  </Tab>

  <Tab title="Basic Auth">
    Using username and password for basic authentication:

    ```yaml theme={null}
    global:
      config: |
        s3.telemetry.metrics.exporter.uri=rw://?endpoint=<your-prometheus-endpoint>&auth=basic&username=<your-prometheus-username>&password=<your-prometheus-password>
    ```
  </Tab>

  <Tab title="Bearer Token">
    Using Bearer Token for authentication:

    ```yaml theme={null}
    global:
      config: |
        s3.telemetry.metrics.exporter.uri=rw://?endpoint=<your-prometheus-endpoint>&auth=bearer&token=<your-prometheus-token>
    ```
  </Tab>

  <Tab title="AWS SigV4">
    Using AWS SigV4 signature authentication (suitable for Amazon Managed Prometheus and similar services):

    ```yaml theme={null}
    global:
      config: |
        s3.telemetry.metrics.exporter.uri=rw://?endpoint=<your-prometheus-endpoint>&auth=sigv4&region=<your-prometheus-region>
    ```
  </Tab>
</Tabs>

<Tip>
  Once RemoteWrite is enabled, AutoMQ will automatically push core metrics from Brokers and Controllers to the specified Prometheus endpoint. For detailed information on metrics monitoring and alerting configuration, refer to [Prometheus Monitoring & Alerting](/automq-cloud/monitoring-alert/monitoring-alert-via-prometheus).
</Tip>

## Security and Access Control

AutoMQ supports multiple security configurations to protect your data in transit and control client access. This section covers the two primary security models for client authentication when deploying with the Helm chart: `SASL_SSL` and `SSL` (mutual TLS).

These two paths are mutually exclusive. Choose the one that aligns with your organization's security policies.

<Tip>
  Starting with the latest Helm chart, you only need to provide **one** TLS secret containing `ca.crt`, `tls.crt`, and `tls.key`. AutoMQ reuses this PEM bundle for **all** listeners and internal clients (AutoBalancer/admin) so there is no need to prepare separate certificates, and hostname verification for these built-in clients is automatically disabled when mTLS is enabled.
</Tip>

<Note>
  The Helm values `tls.keystorePassword` and `tls.truststorePassword` are used only when loading JKS/PKCS12 keystores. For PEM-based deployments (the default when `tls.type=PEM` or certificates are auto-generated) the chart mounts the PEM files directly, so these passwords are typically left blank unless your private key itself is protected.
</Note>

***

### Path 1: Configuring SASL\_SSL Authentication

This is a common security model where clients authenticate using a username and password, and the communication channel is encrypted with TLS.

#### Step 1: Configure `values.yaml` for SASL\_SSL

You need to define a `SASL_SSL` listener, enable ACLs, and configure SASL users and their passwords. The server will present a TLS certificate to clients, but clients do not need their own certificate to authenticate.

**Example `values.yaml` configuration:**

````yaml theme={null}
# 1. Define a listener for SASL_SSL clients
listeners:
  client:
    - containerPort: 9112
      protocol: SASL_SSL
      name: CLIENT_SASL_SSL
      advertisedHostnames:
        enabled: true
        baseDomain: automq.private
        externalDns:
          privateZoneId: <your-route53-zone-id>

# 2. Reference the secret containing the server's TLS certificate
tls:
  type: PEM
  existingSecret: automq-server-tls # Secret must contain server.crt, server.key, ca.crt

# 3. Expose the SASL_SSL port externally
externalAccess:
  controller:
    enabled: true
    service:
      type: LoadBalancer
      extraPorts:
        - name: "sasl-client"
          port: 9112
          protocol: "TCP"
          targetPort: 9112
    externalDns:
      enabled: true
      hostname: automq-bootstrap.automq.private
      privateZoneId: <your-route53-zone-id>
      recordType: A
      ttl: 60
      annotations:
        external-dns.alpha.kubernetes.io/evaluate-target-health: "false"

# 4. Enable ACLs and define the SASL superuser
acl:
  enabled: true
  superUsers:
    - "_automq"

# 5. Define credentials for the superuser and regular client users
sasl:
  controller:
    password: "\<strong-password-for-automq\>"
    user: _automq
  interbroker:
    password: "\<strong-password-for-automq\>"
    user: _automq
  client:
    users:
      - "my-user" # A regular application user
    passwords:
      - "<password-for-my-user>"

#### (Optional) Automate Route 53 binding with external-dns

If you deploy on AWS and want Helm to publish the bootstrap DNS record automatically:

1. Install [external-dns](https://github.com/kubernetes-sigs/external-dns) in your cluster with `--source=service --provider=aws --policy=upsert-only --registry=txt`. Grant its IAM role `route53:ListHostedZones`, `route53:ListResourceRecordSets`, and `route53:ChangeResourceRecordSets`, and enable IRSA/credentials accordingly.
2. Ensure the base domain you configured under `listeners.client[].advertisedHostnames.baseDomain` exactly matches the Route 53 hosted zone (for example `automq.private`) and the `privateZoneId` fields reference the same zone ID.
3. Add/verify the following block: the controller Service keeps owning the bootstrap hostname, while the listener block tells Kafka which FQDNs to advertise.

```yaml
listeners:
  client:
    - name: CLIENT_SASL_SSL
      containerPort: 9112
      protocol: SASL_SSL
      advertisedHostnames:
        enabled: true
        baseDomain: automq.private
        externalDns:
          privateZoneId: <your-route53-zone-id>

externalAccess:
  controller:
    enabled: true
    service:
      type: LoadBalancer
    externalDns:
      enabled: true
      hostname: automq-bootstrap.automq.private
      privateZoneId: <your-route53-zone-id>
      recordType: A
      ttl: 60
```yaml

Once the controller Service obtains an NLB and renders those annotations, external-dns will upsert `automq-bootstrap.automq.private` automatically. Set `externalAccess.controller.externalDns.enabled=false` if you prefer to copy the hostname and create the record manually.
````

#### Step 2: Post-Deployment ACL Management

After deploying the cluster, you must use the superuser (`_automq`) to grant permissions to regular users like `my-user`.

1. **Configure an Admin Client (`superuser.properties`):**
   This file allows you to run admin tools by authenticating as `_automq`.
   ```properties theme={null}
   security.protocol=SASL_SSL
   sasl.mechanism=PLAIN
   sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
     username="_automq" \
     password="\<strong-password-for-automq\>";
   # The client needs to trust the server's certificate
   ssl.truststore.certificates=/path/to/your/ca.crt
   ```

2. **Grant Permissions:**
   Use `kafka-acls.sh` with the admin configuration to grant `WRITE` and `READ` permissions to `my-user`.
   ```bash theme={null}
   kafka-acls.sh --bootstrap-server <your-load-balancer-dns>:9112 \
     --command-config superuser.properties \
     --add --allow-principal User:my-user --operation WRITE --topic my-topic
   ```

#### Step 3: Client Configuration

A regular application client (`my-user`) would use the following configuration. The client only needs to trust the server, which can be done by providing the CA certificate in a PEM file or a JKS truststore.

**Example using PEM Truststore:**

```properties theme={null}
# client.properties for 'my-user'
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="my-user" \
  password="<password-for-my-user>";

# Path to the PEM file containing the CA certificate
ssl.truststore.certificates=/path/to/your/ca.crt
```

**Example using JKS Truststore:**

```properties theme={null}
# client.properties for 'my-user'
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="my-user" \
  password="<password-for-my-user>";

ssl.truststore.location=/path/to/your/truststore.jks
ssl.truststore.password=your_truststore_password
```

***

### Path 2: Configuring SSL (mTLS) Authentication

In this model, clients authenticate by presenting a TLS certificate that is trusted by the cluster. This is known as mutual TLS (mTLS).

#### Step 1: Prepare Certificates

You will need a hierarchy of certificates:

* **Server Certificate:** For the AutoMQ brokers.
* **Admin Client Certificate:** A certificate with a specific Common Name (e.g., `CN=automq-admin`) for an administrator who will be designated as a superuser.
* **Application Client Certificate:** A unique certificate for each client application (e.g., `CN=my-app`).

#### Step 2: Configure `values.yaml` for mTLS

You need to define an `SSL` listener, require client authentication, and set the admin certificate's principal as the superuser.

**Example `values.yaml` configuration:**

```yaml theme={null}
# 1. Define a listener for mTLS clients and require client certs
listeners:
  client:
    - containerPort: 9122
      protocol: SSL
      name: CLIENT_MTLS
      # This enforces mTLS
      sslClientAuth: required
      advertisedHostnames:
        enabled: true
        baseDomain: automq.private
        externalDns:
          privateZoneId: <your-route53-zone-id>

# 2. Reference the secret containing the server's TLS certificate
tls:
  type: PEM
  existingSecret: automq-server-tls

# 3. Expose the SSL port externally
externalAccess:
  controller:
    enabled: true
    service:
      type: LoadBalancer
      extraPorts:
        - name: "tls-client"
          port: 9122
          protocol: "TCP"
          targetPort: 9122
    externalDns:
      enabled: true
      hostname: automq-bootstrap.automq.private
      privateZoneId: <your-route53-zone-id>
      recordType: A
      ttl: 60

# 4. Enable ACLs and define the certificate-based superuser
acl:
  enabled: true
  # The principal is 'User:' + the full Subject of the admin certificate
  superUsers:
    - "User:CN=automq-admin"

# 5. SASL is still used for internal communication
sasl:
  # ... configuration for _automq ...
```

#### Step 3: Post-Deployment ACL Management

After deployment, use the admin certificate to grant permissions to regular application principals.

1. **Configure an Admin Client (`admin.properties`):**
   This file uses the admin certificate (`CN=automq-admin`) to authenticate.

   **Example using PEM files (Recommended):**

   ```properties theme={null}
   security.protocol=SSL
   ssl.truststore.certificates=/path/to/your/ca.crt
   ssl.keystore.key=/path/to/your/admin.key
   ssl.keystore.certificate.chain=/path/to/your/admin.crt
   ```

   **Example using JKS files:**

   ```properties theme={null}
   security.protocol=SSL
   ssl.truststore.location=/path/to/your/truststore.jks
   ssl.truststore.password=your_truststore_password
   ssl.keystore.location=/path/to/your/admin-keystore.jks
   ssl.keystore.password=your_keystore_password
   ```

2. **Grant Permissions:**
   Use `kafka-acls.sh` to grant permissions to the application principal `User:CN=my-app`.
   ```bash theme={null}
   kafka-acls.sh --bootstrap-server <your-load-balancer-dns>:9122 \
     --command-config admin.properties \
     --add --allow-principal "User:CN=my-app" --operation WRITE --topic my-topic
   ```

#### Step 4: Client Configuration

A regular application client would use its own unique certificate (`CN=my-app`) to connect.

**Example using PEM files (Recommended):**

```properties theme={null}
# client.properties for 'my-app'
security.protocol=SSL
ssl.truststore.certificates=/path/to/your/ca.crt
ssl.keystore.key=/path/to/your/myapp.key
ssl.keystore.certificate.chain=/path/to/your/myapp.crt
```

**Example using JKS files:**

```properties theme={null}
# client.properties for 'my-app'
security.protocol=SSL
ssl.truststore.location=/path/to/your/truststore.jks
ssl.truststore.password=your_truststore_password
ssl.keystore.location=/path/to/your/myapp-keystore.jks
ssl.keystore.password=your_keystore_password
```

```bash theme={null}

# Create a topic
./kafka-topics.sh --create --bootstrap-server <bootstrap-server> \
  --replication-factor 1 --partitions 3 --topic <your-topic> \
  --command-config client-mtls.properties

```

## Performance Tuning

For more details on performance tuning, refer to the [AutoMQ Performance Tuning Guide](https://docs.automq.com/automq/deployment/performance-tuning-for-broker).