Runs on MinIO
Prerequisites
- For installing MinIO, refer to: https://min.io/docs/minio/linux/operations/installation.html
Configuring WAL
Since MinIO does not support block storage, AutoMQ cannot depend entirely on MinIO to deliver highly reliable Kafka services. In use cases involving logging, it's advisable to use local SSDs rather than distributed block storage. It's crucial to recognize that a failure in the local SSD could lead to a temporary interruption in data being uploaded to MinIO, potentially resulting in the loss of recent data.
To establish AutoMQ's Write-Ahead-Log (WAL) on local SSD storage, ensure that the designated file path is on an SSD with at least 10GB of free space.
--override s3.wal.path=/home/admin/automq-wal
Configuring S3URL
The Access key and Secret Key match the environment variables MINIO_ROOT_USER and MINIO_ROOT_PASSWORD needed for MinIO installation
You can utilize the following command to check the endpoint, formatted as follows:
sudo systemctl status minio.service
API: http://10.1.0.240:9000 http://172.16.1.104:9000 http://172.16.1.103:9000 http://172.16.1.102:9000
Create a Bucket for MinIO
- Configure the necessary Access Key and Secret Key for the AWS CLI by setting environment variables.
export AWS_ACCESS_KEY_ID=minioadmin
export AWS_SECRET_ACCESS_KEY=minio-secret-key-CHANGE-ME
- Use the AWS CLI to create an S3 bucket.
aws s3api create-bucket --bucket automq-data --endpoint=http://10.1.0.240:9000
Deploy AutoMQ
Below are the necessary parameters required to generate an S3 URL:
| Parameter Name | Default Value in This Example | Description |
|---|---|---|
| --s3-access-key | minioadmin | Environment variable MINIO_ROOT_USER |
| --s1-secret-key | minio-secret-key-CHANGE-ME | Environment variable MINIO_ROOT_PASSWORD |
| --s3-region | us-west-2 | This parameter is ineffective in MinIO and can be set to any value, such as us-west-2 |
| --s3-endpoint | http://10.1.0.240:9000 | Obtain the endpoint by running the command sudo systemctl status minio.service |
| --s3-data-bucket | automq-data | - |
| --s3-ops-bucket | automq-ops | - |
After setting up WAL and S3URL, you are ready to deploy AutoMQ. Please refer to the instructions in Cluster Deployment on Linux▸.