All components of the AutoMQ Cloud BYOC environment are deployed within your Alibaba Cloud account to ensure data privacy and security. Before installing AutoMQ, you need to prepare various Alibaba Cloud resources in advance. This article describes how to prepare the necessary cloud resources for AutoMQ on the Alibaba Cloud public cloud.
In this document, “AutoMQ,” “AutoMQ service provider,” and “the AutoMQ service” all refer to AutoMQ HK Limited and its affiliates.
Prepare VPC
You can choose to use an existing VPC when installing AutoMQ, or create a new one for it. For production environments, we recommend checking and adjusting your existing VPC configuration to meet AutoMQ’s requirements. For POC testing, creating a new VPC is a quicker option.
VPC Requirements
To deploy the AutoMQ BYOC environment on Alibaba Cloud, your Virtual Private Cloud (VPC) must meet the following conditions:
| Resource | Requirement Description |
|---|
| Control Plane VSwitch | 1. Used to deploy the AutoMQ BYOC environment console, which needs to connect to the public network via a NAT Gateway or other methods. |
| Data Plane VSwitch | 1 or 3. Used to deploy the AutoMQ data plane cluster. The number of VSwitches depends on whether you choose a single-AZ or three-AZ deployment model. |
| DNS Configuration | Enabled. The VPC must have private domain name resolution enabled, as AutoMQ relies on Alibaba Cloud DNS PrivateZone for DNS resolution within the VPC. |
| NAT Gateway | 1. If you plan to deploy the AutoMQ data plane cluster on ACK, you must configure a NAT gateway and the correct public outbound routes for the VSwitches where the ACK worker nodes are located.
Note: If the NAT gateway is not configured correctly, the AutoMQ cluster installation on ACK will fail. |
Scenario 1: Create a New VPC
If you need to create a new VPC for AutoMQ, we recommend using the standard creation process in the Alibaba Cloud VPC console.
- Log in to the Alibaba Cloud VPC console.
- Click Create VPC.
- Configure the Region and Availability Zone.
- Set the Name and IPv4 CIDR Block for the VPC.
- Simultaneously, create the required VSwitches for the VPC. Create the appropriate number of VSwitches in different availability zones based on your deployment model (single-AZ or three-AZ).
- Depending on your deployment needs, create a NAT Gateway during the creation process or in a subsequent step.
- NAT Gateway: If you plan to deploy AutoMQ on ACK, ensure you configure a NAT Gateway for the VSwitches where the worker nodes are located.
- Click Confirm to create.
If you are deploying AutoMQ in an existing VPC, please verify that the following configurations meet the requirements.
1. Check DNS Configuration
- In the Alibaba Cloud VPC console, select your target VPC.
- In the VPC details, confirm that the private domain name resolution feature is enabled. This feature is enabled by default for Alibaba Cloud VPCs.
Prepare PrivateZone (Optional)
The AutoMQ cluster exposes service addresses to applications through Private DNS, which relies on private domain name resolution within the VPC. Therefore, a PrivateZone must be configured to provide reliable domain name resolution services. You can choose to have the AutoMQ console create the PrivateZone automatically or prepare it yourself by following the tutorial below.
PrivateZone Requirements
When configuring PrivateZone, the following two key conditions must be met:
Condition
| Description
| Importance
|
|---|
Associated VPC
| The PrivateZone must be associated with the VPC where AutoMQ is deployed. | Required |
Domain Name and TLS Consistency
| If you need to use the TLS feature later, the domain name of the PrivateZone must be consistent with the domain name in the TLS certificate. | As needed |
PrivateZone Configuration Steps
Step 1: Check Existing PrivateZone
- Log in to the Alibaba Cloud console and visit the Alibaba Cloud DNS PrivateZone console.
- In the left navigation bar, select Private DNS.
- Check if a Zone associated with the target VPC already exists.
If a PrivateZone already exists:
- Select the target Zone and click Set Association Scope.
- In the Associated Resolution Scopes section, ensure that the VPC where AutoMQ is deployed is included.
- Record the domain name information for subsequent TLS feature configuration.
If you need to create a new PrivateZone:
- Click Add Zone.
- Configure the basic information:
- Zone Name: Enter a private domain name (e.g.,
internal.example.com).
- Click OK.
- After successful creation, go to the Associated VPCs page for the Zone and associate the VPC where AutoMQ is deployed.
Step 3: Verify the Configuration
- On the Zone’s details page, confirm the VPC association status.
- Check if the domain name configuration meets the planning requirements for the TLS feature.
- Record the Zone’s ID and domain name, which will be provided to the AutoMQ components during subsequent installation.
Prepare Cluster Security Group (Optional)
During the cluster deployment process, you can opt to use a pre-configured security group for precise network access control. If you choose this method, you must create a security group in advance and configure its inbound rules according to the table below. If you prefer to simplify network configuration, you can choose to have AutoMQ automatically create a new security group with all the necessary rules during cluster creation, in which case you can skip this section.
| Port | Purpose | Required Access Source | Enablement Condition |
|---|
| 9093 | Exposes the PLAINTEXT protocol for internal communication among Controller nodes. | - Internal cluster components
| Enabled by default |
| 9103 | Exposes the SASL_PLAINTEXT protocol for internal communication among Controller nodes. | - Internal cluster components
| Enabled by default |
| 9092 | Exposes the Kafka service to clients via the PLAINTEXT protocol. | - Console
- Internal cluster components
- Applications
| Enabled by default |
| 9102 | Exposes the Kafka service to clients via the SASL_PLAINTEXT protocol. | - Console
- Internal cluster components
- Applications
| Enabled by default |
| 9112 | Exposes the Kafka service to clients via the SASL_SSL protocol. | - Console
- Internal cluster components
- Applications
| Optional |
| 9122 | Exposes the Kafka service to clients via the SSL protocol. | - Console
- Internal cluster components
- Applications
| Optional |
| 9090 | Provides an endpoint for Prometheus Metrics Exporter to read metrics. | - Console
- Applications (Optional)
| Enabled by default |
| 22 | Default SSH port for cluster nodes, exposed to the console node for emergency troubleshooting. | | Enabled by default |
| 8081 | Service port for the managed Schema Registry. | - Internal cluster components
- Applications
| Optional, required only when TableTopic is enabled. |
Prepare NAS File System Security Group (Optional)
If you plan to use AutoMQ in an Alibaba Cloud environment and leverage Apsara File Storage NAS as the storage backend for the WAL (Write-Ahead Log), you need to configure a dedicated permission group for the NAS file system. This section details the required rules for that permission group. If you prefer AutoMQ to automatically create and configure this permission group during deployment, you can skip this section.
| Port | Access Source | Purpose | Enablement Condition |
|---|
| 443 | - Console
- Internal cluster components
| Allows the AutoMQ cluster and console to call the NAS management API. | Enabled by default |
| 22 | - Internal cluster components
| Allows nodes within the AutoMQ cluster to access the NAS service. | Enabled by default |
| 111 | - Internal cluster components
| Allows nodes within the AutoMQ cluster to access the NAS service. | Enabled by default |
| 635 | - Internal cluster components
| Allows nodes within the AutoMQ cluster to access the NAS service. | Enabled by default |
| 2049 | - Internal cluster components
| Allows nodes within the AutoMQ cluster to access the NAS service. | Enabled by default |
| 4045 | - Internal cluster components
| Allows nodes within the AutoMQ cluster to access the NAS service. | Enabled by default |
| 4046 | - Internal cluster components
| Allows nodes within the AutoMQ cluster to access the NAS service. | Enabled by default |
