Skip to main content
This guide provides production best practices for migrating Apache Kafka workloads to AutoMQ with Kafka Linking. Complete the prerequisite review, then use this guide to create a runbook for each migration batch.
Kafka Linking synchronizes Kafka topic data and committed Kafka Consumer Group offsets. It does not rewrite offsets stored in Flink checkpoints, savepoints, application databases, or other external systems. Identify the offset source that each consumer actually uses before cutover.

​
Plan each migration batch

Start with a non-critical but representative workload. Expand the scope only after the pilot passes. Keep dependent producers, topics, and consumers in the same batch so that a business data flow is not split across migration stages. Record the following information before migration: Avoid unrelated source-cluster upgrades, broker replacements, network changes, authentication changes, and topic recreation during the migration window. Limiting concurrent changes makes failures easier to attribute.

​
Configure reliable source endpoints

A Kafka client uses bootstrap.servers to obtain cluster metadata. It then connects to the broker addresses returned through advertised.listeners. A successful connection to only the bootstrap address does not prove that Kafka Linking can reach the cluster.
Enter a stable bootstrap DNS name or the formal endpoint provided by the source Kafka service. Do not use Kubernetes Pod IPs, node IP and NodePort combinations, or other temporary addresses that change during Pod recreation, node replacement, or scaling.
Validate the source connection as follows:
  • Use stable DNS names or the source Kafka service’s formal endpoint. When you configure multiple endpoints, distribute them across failure domains.
  • Confirm that the AutoMQ data plane can resolve and connect to every broker address and port returned in metadata.
  • For a Kubernetes source cluster, use the Kafka Operator or platform-provided stable bootstrap service and stable per-broker external listeners.
  • Allow both the bootstrap endpoints and all advertised broker endpoints through firewalls, security groups, routes, and network access controls.
  • For TLS or mTLS, verify the trust chain, the hostname against the certificate SAN, and the certificate lifetime.
  • Use a dedicated migration identity with the permissions required to read the selected topics, inspect Consumer Groups, and discover metadata. Do not rotate or delete this identity during migration.
Run a metadata check from an environment with a network path equivalent to the AutoMQ data plane. For example:
The command must return the expected broker list. Then validate DNS resolution, port connectivity, and the TLS handshake for every returned endpoint. Fix advertised.listeners or network access before creating the Kafka Link if any broker is unreachable.

​
Prepare the source and target clusters

Complete these checks before creating a Kafka Link:
  • Reserve enough target capacity for migration traffic as described in Prerequisites.
  • Make sure that a topic with the same name does not already exist on the target. Kafka Linking must create a Mirror Topic for the source topic.
  • Keep source topic identity and partition structure stable within a migration batch. Do not delete and recreate a topic with the same name.
  • Keep target data long enough to cover the migration and observation windows. The target must not expire records that a consumer still needs before cutover.
  • Validate Kafka ACLs, certificates, Schema Registry, connectors, and other external dependencies on the target. Do not assume that topic data synchronization migrates every surrounding configuration or state.
  • Save a baseline of topic configurations, partition ranges, Consumer Group offsets, and business traffic on both clusters.
From the time a Mirror Topic is created until it is promoted to PROMOTED, do not increase the partition count of either the source topic or the target Mirror Topic. The source and target partition counts must remain unchanged while Kafka Linking is in progress.
After the Kafka Link enters the linking state, monitor replication lag and errors. Start client cutover only when lag converges and no sustained authentication, network, or request errors remain.
Keep the source cluster, its network path, DNS endpoints, and migration identity available until every Mirror Topic and Consumer Group is promoted and business acceptance passes. Do not stop, scale to zero, delete, or release the source cluster early, and do not revoke Kafka Linking’s access permissions.

​
Cut over standard producers and consumers

This workflow applies to producers that do not use Kafka transactions. Producers with a transactional.id must use the transactional producer workflow.
  1. Move standard producers to the target AutoMQ instance in batches, then recreate or restart the clients. While a Mirror Topic is LINKING, Kafka Linking routes these writes to the source and replicates them back to the target.
  2. Check every producer deployment and send callback. Confirm that no untracked producer instance continues to write directly to the source.
  3. Before moving a Consumer Group, confirm that the actual startup offset for every partition is within the target topic’s readable range. See Validate the consumer’s actual offset source for the offset sources used by different clients.
  4. Change the Consumer connection configuration and perform a rolling deployment so that instances connect to the target AutoMQ instance in batches. Briefly running the same group.id on the source and target is expected during the rollout. Kafka Linking completes Consumer Group promotion after all source instances have exited.
  5. Confirm that every Consumer instance connects to the target and Consumer Group promotion is complete. Validate consumption rate, lag, business results, and errors, and complete the observation window before moving the next workload.
The rolling cutover must converge with every instance connected to the target. Do not run the same group.id on the source and target for an extended period; this prevents Consumer Group promotion and makes consumer progress and rollback positions ambiguous.

​
Cut over transactional producers

Kafka transactions cannot use the rolling proxy workflow for standard producers. Use the following order for any application that sets transactional.id or depends on exactly-once semantics:
  1. Keep the transactional producer on the source while target replication lag converges.
  2. Stop transactional producers on the source. Confirm that every open transaction has committed or aborted.
  3. Compare source and target end offsets for every partition. Confirm that the target has caught up with the source.
  4. Promote every Mirror Topic involved in the transactions and wait for the PROMOTED state.
  5. Confirm that old producer instances cannot restart. Then start the transactional producers on the target.
  6. Use read_committed consumers to verify committed records from before and after promotion, and validate the end-to-end transactional result.
Do not point a transactional producer at the target while its Mirror Topic is still LINKING. For transactions across multiple topics, place all involved topics in one migration batch and stop, catch up, and promote them together.

​
Validate the consumer’s actual offset source

The consumer startup position depends on how the client manages state: For every topic-partition, verify:
Pause the consumer cutover if any partition fails this check. Options include continuing on the source until the checkpoint advances, changing the synchronization or retention range, or resetting client state after assessing the business impact. For example, if a Consumer’s actual startup offset for Partition X is 100, it can cut over when the target readable range is [80, 150]. A range of [120, 150] means that the historical record at offset 100 is no longer readable. A range of [80, 90] means that the target has not replicated through offset 100, so wait for the target to catch up.
auto.offset.reset applies only when the client has no valid starting offset. It does not override a Flink restore or an explicit assign and seek from an external offset store.

​
Promote Mirror Topics

Promotion stops write routing and replication from the source for a topic. Pass every gate before promotion: Kafka Linking remains connected to the source and routes writes during LINKING, so source-cluster connections alone do not indicate that workload Producers have not switched. For a low-frequency topic, do not use a short zero-traffic window as the only proof that a Producer has stopped. Check the workload Producer inventory and deployment configuration, and compare source and target write requests over a window longer than the normal message interval. Promote topics by business batch instead of promoting the entire migration at once. Validate each batch before continuing.
During Mirror Topic promotion, a small number of writes that still reach the old routing path can briefly receive OUT_OF_ORDER_SEQUENCE_NUMBER. For non-transactional producers with idempotence enabled and normal retry settings, Kafka clients typically reset the sequence state and retry automatically, so applications usually do not need to intervene. Monitor final send failures. Investigate client configuration, Topic state, and residual routed traffic only if the error is ultimately returned to the application or continues after promotion; compensate only records confirmed as failed according to the application’s idempotency policy.

​
Define rollback boundaries

Rollback semantics change after promotion:
  • Before promotion: Standard producers can return to the source. Before moving consumers back, reconcile the source Consumer Group offset or expect records already processed on the target to be replayed.
  • After promotion: New target records no longer replicate to the source. Changing only bootstrap.servers would return clients to a source cluster that is missing post-promotion records. Stop the cutover and use a data reconciliation or reverse-migration plan.

​
Complete the migration

End a Kafka Linking migration only after all of these conditions are met:
  • Every Mirror Topic in the Kafka Link is PROMOTED.
  • Every Consumer Group in the Kafka Link has completed promotion.
  • Production, consumption, lag, error rate, and critical business results on the target AutoMQ instance pass acceptance, including the agreed observation window.
After all conditions pass, delete the Kafka Link itself in AutoMQ Console. Deleting the Kafka Link marks the Kafka Linking migration as fully complete.
Do not delete the promoted Mirror Topics or Consumer Groups one by one. Deleting these resources deletes the corresponding topic or Group from the target AutoMQ instance and can cause message data or consumer offsets to be lost, or interrupt the workload. To complete the migration, delete only the Kafka Link itself.
After deleting the Kafka Link, retain the source cluster in an idle state for an observation period. Stop new workload writes, but preserve the original data and required access. Reclaim the source cluster’s network, storage, compute resources, and migration identity only after the target remains stable and rollback or data reconciliation is no longer needed. See Executing migration for the operational steps.

​
Use the migration checklist

Keep at least the following evidence for each batch:
  • Stable source bootstrap endpoints and connectivity results for every advertised broker endpoint.
  • Migration identity permissions, certificate lifetime, and firewall approvals.
  • Topic, producer, Consumer Group, external offset source, owner, and dependency inventories.
  • Partition earliest offsets, end offsets, committed Group offsets, and replication lag before and after cutover.
  • The selected standard or transactional producer cutover order.
  • Evidence that source Groups became Empty, target consumers stabilized, and business acceptance passed.
  • Mirror Topic promotion approval, execution time, observation result, and rollback boundary.
  • Promotion results for every Mirror Topic and Consumer Group, the Kafka Link deletion record, and approval to reclaim the source cluster.
After completing these checks, follow Executing migration to create the Kafka Link in AutoMQ Console and run the migration.