Skip to Main Content

Overview

The environment is a crucial operational resource for AutoMQ Cloud. This article introduces the definition, resource model, internal attributes, classification, and usage recommendations of the AutoMQ Cloud environment.

In this article, references to AutoMQ product service providers, AutoMQ service providers, and AutoMQ specifically refer to AutoMQ HK Limited.

Terms such as cloud providers and Public Cloud providers mentioned in this article refer to the cloud service providers that AutoMQ Cloud relies on, as specified by the user, including AWS, Google Cloud, Azure, Alibaba Cloud, Tencent Cloud, and Huawei Cloud.

Definition​

The environment is a logical abstract resource provided by AutoMQ Cloud, used to determine the foundational configuration information that needs to be shared at the runtime level of Kafka and to manage the actual usage of Kafka instances (clusters) in a unified manner. The environment defines information such as infrastructure type, network environment, and permission policies, and all subsequent Kafka instances are created and managed under the environment.

The main roles of the environment are elaborated as follows:

  • It stipulates the underlying infrastructure of the Kafka service, clarifying the basic information such as the type of cloud provider, region, and permission policies of the infrastructure.

  • It stipulates the basic information such as the network topology, availability zone topology, and external service exposure methods of the entire control system.

  • Common foundational configurations across instances, such as metrics integration configurations.

The environments are divided into SaaS environments and BYOC environments based on the ownership and operational responsibilities of the underlying resources. In a SaaS environment, all physical resources belong to the AutoMQ service provider, who also provides maintenance and management. In a BYOC environment, all physical resources belong to the user, and the user authorizes the AutoMQ service provider to offer maintenance and management.

Model Relationships​

In the domain model of AutoMQ Cloud, the environment is the top-level resource for managing product instances, as roughly illustrated below:

Internal Attributes​

Attribute: Infrastructure Configuration​

Infrastructure configuration is a necessary setup for deploying AutoMQ Cloud environments, specifying the type of infrastructure, network topology, permission policies, and other configuration information. The environment console will use this configuration to deploy and manage actual Kafka clusters.

Attribute: Integration Configuration​

Integration configuration abstracts configuration at the environment level for external data integration across clusters. For example, the Metrics integration configuration can support pushing Metrics data from multiple clusters within the environment to a user-defined Prometheus cluster for custom monitoring and analysis.

Attribute: Environment Version​

Technically, the environment itself is a software foundation and, as such, will define multiple environment versions with the continuous iteration of the software. Upgrading the environment version generally does not affect the upper-layer product services. Users are advised to update to the latest environment version regularly to ensure defect fixes and feature optimizations.

Attribute: Environment Status​

The environment status shows the various service states in the environment lifecycle. The complete lifecycle of an environment will include the following processes and states:

  • Creating: The initial environment creation operation is an asynchronous process. When the creation operation is completed, the backend will asynchronously deploy the full software foundation. During this process, the user cannot make any changes until the creation is complete.

  • Running: The current environment is in an active running state, allowing normal operational use such as creating, deleting, updating, and querying internal product instances.

  • Changing: The current environment is performing operational changes, version upgrades, or other operations. Since these operations are asynchronous, any changes to the environment are prohibited until the process is complete.

  • Service Exception: The current environment is unable to provide normal services due to the execution of creation, maintenance changes, or underlying infrastructure anomalies.

BYOC Environment​

Environment Architecture​

As mentioned above, the BYOC environment is maintained by the AutoMQ Cloud agent. The underlying resources belong to the user's custom cloud account under the VPC, allowing for complete data isolation and autonomous control. In this environment type, the data plane system and control plane system are as shown in the following figure:

  • Resource Location: The control plane system (environment console) and the data plane system (Kafka service cluster) are all deployed in the user-defined network environment.

  • Network Exposure: The environment console can be accessed via the public network by default based on user choice, or it can be restricted to private network access only. The data plane system currently only supports private network access.

  • Maintenance Authorization: The BYOC environment is a fully privately isolated private environment that requires user authorization for AutoMQ to perform environment maintenance (issue troubleshooting and version upgrades).

  • System Observability: The BYOC environment collects system metrics and log data through the maintenance bucket provided by the cloud providers. Users authorize AutoMQ service providers with read-only access to achieve system operation monitoring and alerting.

SaaS Environment​

Environment Architecture​

As mentioned above, the SaaS environment is a fully managed environment operated by AutoMQ Cloud. The underlying resources and operations are managed by AutoMQ. In this type of environment, the data plane and control plane systems are as shown in the following figure: