Monitoring & Alert Via CloudWatch

Metrics are critical analytical data for system observability. AutoMQ supports exposing multiple native Apache Kafka® metrics via CloudWatch.

Principles of Metrics Collection and Application

Internal components of AutoMQ support collecting various Kafka server metrics. However, the current Business Edition does not provide built-in metrics dashboards and alerting capabilities. Users can implement custom metrics monitoring and analysis based on the integration features provided by AutoMQ. The overall architecture is shown below:

Referencing the diagram above, use the metrics integration feature Manage Integrations▸ to forward metrics data to the CloudWatch service.

CloudWatch Metrics Definition

Detailed definitions of the related metrics exposed in the integration mentioned above are as follows:

Metric Name	Metric Description
kafka_controller_active_count	Meaning: Number of active Controller nodes, unit: count. Dimension: Cluster
kafka_network_io_bytes_in_sec	Meaning: Write throughput, unit: Byte/s. Dimension: Cluster, Topic
kafka_network_io_bytes_out_sec	Meaning: Read throughput, unit: Byte/s. Dimension: Cluster, Topic
kafka_server_connection_count	Meaning: Number of client connections, unit: count. Dimension: Cluster
kafka_partition_total_count	Meaning: Total number of partitions in the cluster, excluding replicas, unit: count. Dimension: Cluster
kafka_topic_count	Meaning: Total number of Topics in the cluster, unit: count. Dimension: Cluster
kafka_consume_offset_lag	Meaning: Number of messages in consumer backlog, unit: count. Dimension: Topic, Consumer Group
kafka_message_count_in_sec	Meaning: Number of messages written per second, unit: messages/second. Dimension: Cluster, Topic
kafka_partition_offline_count	Meaning: The number of offline partitions without leaders in the cluster, measured in units. Dimension: Cluster
kafka_log_size	Meaning: The size of message storage, measured in Bytes. Dimension: Cluster, Topic