Deploy a scalable AutoMQ test cluster on Docker to evaluate cloud-native message streaming with full Kafka compatibility and feature exploration.
This document explains how to deploy a three-node AutoMQ cluster in a local development environment (by deploying MinIO to provide object storage services). In this setup, users can test cluster-related features such as partition reassignment and data auto-balancing.Besides the Docker deployment option, users can reference the following documentation for alternative deployment methods to verify cluster-related features:
Deploying AutoMQ and tuning parameters for production environment loads can be complex. For necessary assistance and best practices, you can contact the AutoMQ team through this form.
Additionally, if you’d like to completely bypass the installation and deployment process, you can try the fully managed cloud service provided by the AutoMQ team through the link below. A free 30-day trial is available on AWS Marketplace.
Use the following command to download the Docker Compose configuration file for the three-node deployment plan and start the three-node AutoMQ cluster using Docker Compose.
curl -O https://raw.githubusercontent.com/AutoMQ/automq/refs/tags/1.7.3/docker/docker-compose-cluster.yaml && docker compose -f docker-compose-cluster.yaml up -d
After a successful startup, the cluster address will be exposed under the Docker network named automq_net, and the cluster’s bootstrap address will be “server1:9092,server2:9092,server3:9092.” You can connect to the cluster using this address subsequently.
Note:In the above test environment, AutoMQ utilizes a Docker network named automq_net. To connect to the cluster, you need to configure the —network automq_net option to use the same Docker network for the connection.
After installing and starting the AutoMQ cluster, you can test message sending, consumption, and other functionalities by running the following commands.
Execute the following command to create a Topic using the Kafka CLI tool kafka-topics.sh within a Docker environment.
In addition to core message production and consumption features, AutoMQ offers advanced capabilities like second-level partition reassignment and automatic data rebalancing, which address Apache Kafka’s challenges with slow reassignment and scaling. You can refer to the following documentation for testing and validation:
