Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.automq.com/llms.txt

Use this file to discover all available pages before exploring further.

This document outlines the process for using Kafka CLI tools to verify AutoMQ’s sub-second partition reassignment capability. The Kafka CLI tool is executed via a Docker image provided by AutoMQ. First, create a new Kafka topic with a single partition and initiate a producer to write a certain amount of data into it. Next, attempt to reassign the partition among different AutoMQ brokers using the reassign command. Observe the time it takes to complete the partition reassignment.

​
Prerequisites

Before carrying out a partition reassignment test, the following conditions must be met: Complete the installation and deployment of the AutoMQ cluster. You can refer to the following methods to install and deploy AutoMQ: Additionally, the host running the test program must meet the following conditions:
  • Linux/Mac/Windows Subsystem for Linux
  • Docker

​
Experience Partition Reassignment

If the previous AutoMQ cluster was deployed following the Deploy Multi-Nodes Test Cluster on Docker▸ guide, the obtained cluster Bootstrap address will resemble “server1:9092,server2:9092,server3:9092,” and the AutoMQ cluster is on the “automq_net” Docker network.
Please replace the bootstrap-server address below with the actual cluster’s address based on the deployment configuration.

​
Create Topic

docker run --network automq_net automqinc/automq:1.5.5 /bin/bash -c "/opt/kafka/kafka/bin/kafka-topics.sh --create --topic reassign-topic --bootstrap-server server1:9092,server2:9092,server3:9092"

​
Send Messages

Execute the following command to send a specified amount of data. 
docker run --network automq_net  automqinc/automq:1.5.5 /bin/bash -c  "/opt/kafka/kafka/bin/kafka-producer-perf-test.sh --topic reassign-topic --num-records=1024000 --throughput 5120 --record-size 1024 --producer-props bootstrap.servers=server1:9092,server2:9092,server3:9092 linger.ms=100 batch.size=524288 buffer.memory=134217728 max.request.size=67108864"

​
View Partition Distribution

docker run --network automq_net  automqinc/automq:1.5.5 /bin/bash -c "/opt/kafka/kafka/bin/kafka-topics.sh --topic reassign-topic --describe --bootstrap-server server1:9092,server2:9092,server3:9092"

​
Reassign Partitions Between Brokers

In this step, it’s assumed that partition 0 of the test topic “reassign-topic” is currently on node 1, and the goal is to reassign this partition to node 2. Depending on the actual partition distribution during testing, you might opt to reassign to different nodes.
  1. Create the partition reassignment plan.
echo '{
    "partitions": [
        {
            "topic": "reassign-topic",
            "partition": 0,
            "replicas": [
                2
            ]
        }
    ],
    "version": 1
}' > move.json
  1. Execute the partition reassignment plan.
docker run --network automq_net -v $(pwd)/move.json:/move.json automqinc/automq:1.5.5 /bin/bash -c "/opt/kafka/kafka/bin/kafka-reassign-partitions.sh --bootstrap-server  server1:9092,server2:9092,server3:9092 --reassignment-json-file /move.json --execute"

​
3. Check if the Reassignment Is Complete.

docker run --network automq_net -v $(pwd)/move.json:/move.json automqinc/automq:1.5.5 /bin/bash -c "/opt/kafka/kafka/bin/kafka-reassign-partitions.sh --bootstrap-server server1:9092,server2:9092,server3:9092 --reassignment-json-file /move.json --verify"