Skip to Main Content

CloudWatch Monitoring & Alerts Via CloudWatch

Metrics are essential analytical data for system observability. AutoMQ supports exposing several native Apache Kafka® Metrics via CloudWatch.

Metrics Collection and Application Principles​

Internal components of AutoMQ support the collection of various Kafka Server Metrics data. However, the current business edition does not provide built-in Metrics dashboards and monitoring alert capabilities. Users can implement custom Metrics monitoring and analysis based on the integration features provided by AutoMQ. The overall architecture is referenced in the diagram below:

Referencing the diagram above, use the Metrics integration feature Manage Integrations▸ to forward Metrics data to the CloudWatch service.

CloudWatch Metrics Definition​

Detailed definitions of the relevant Metrics exposed through the above integration are as follows:

Metric Name
Metric Description
kafka_controller_active_count
  • Meaning: The number of active Controller nodes, unit: count.
  • Dimension: Cluster
kafka_network_io_bytes_in_sec
  • Meaning: Write traffic throughput, unit: Byte/s.
  • Dimension: Cluster, Topic
kafka_network_io_bytes_out_sec
  • Meaning: Read traffic throughput, unit: Byte/s.
  • Dimension: Cluster, Topic
kafka_server_connection_count
  • Meaning: The number of client connections, unit: count.
  • Dimension: Cluster
kafka_partition_total_count
  • Meaning: The total number of partitions in the cluster, not considering replicas, unit: count.
  • Dimension: Cluster
kafka_topic_count
  • Meaning: The total number of Topics in the cluster, unit: count.
  • Dimension: Cluster
kafka_consume_offset_lag
  • Meaning: The number of messages accumulated for consumption, unit: count.
  • Dimension: Topic, Consumer Group
kafka_message_count_in_sec
  • Meaning: The number of messages written per second, unit: messages/sec.
  • Dimension: Cluster, Topic
kafka_partition_offline_count
  • Meaning: Number of offline, leaderless partitions in the cluster, measured in units.
  • Dimension: Cluster
kafka_log_size
  • Meaning: Size of message storage space, measured in Bytes.
  • Dimension: Cluster, Topic