Skip to main content
Kafka Connect plugins are a collection of components used for streaming data between Kafka and other systems. They typically encapsulate connectors, transforms, and converters for specific data processing scenarios. The AutoMQ managed Kafka Connect service comes with several system plugins that have been officially tested and verified. At the same time, to meet diverse data integration needs, AutoMQ also supports users uploading and managing custom plugins.

​
View Plugin List

Users can view all available system plugins and custom plugins in the current environment through the plugin list. Procedure:
  1. Log in to the AutoMQ console.
  2. In the left navigation bar, click the Kafka Connect menu.
  3. You will be directed to the Plugins list page by default.
On this page, you can see all integrated system plugins, which can be directly used to create Connect tasks. If the system plugins cannot meet your specific needs, you can download plugins from the third-party community or develop and upload your own.

​
Create Custom Plugin

When you need to use a plugin that is not officially provided, you can create a custom plugin through the console. Procedure:
  1. First, the user uploads the custom plugin to a publicly downloadable location (e.g., S3 Bucket or internal file server) and obtains the plugin URL (e.g., S3 URL or CDN download address).
  2. On the plugin list page, click the “Create Custom Plugin” button.
  3. In the dialog box, fill in the basic metadata for the plugin, including: (a) Name, version, and description; (b) Main Class of the plugin; (c) Other necessary parameters.
The system will pre-validate the plugin, and finally, click confirm to complete the plugin creation.

​
Delete Custom Plugin

When you no longer need a specific version of a custom plugin, you can choose to delete it to free up resources and simplify management. Procedure:
  1. On the Plugins list page, find the custom plugin you want to delete.
  2. Click the Delete button on the right side of the corresponding plugin.
  3. Complete the operation in the confirmation dialog box.
Please note the following constraints:
  • System Plugins: Cannot be deleted.
  • In-Use Plugins: If a plugin (or any of its versions) is being used by one or more Connect tasks, it cannot be deleted. You must first stop and delete the relevant tasks.

​
Update Plugin

When the community releases a new version of a plugin, or your own custom plugin has a new version, you can update it by uploading the new version. Procedure: The procedure for updating a plugin is the same as for uploading a custom plugin. The AutoMQ managed Kafka Connect service supports the coexistence of multiple versions of the same plugin. Different versions are independent of each other and do not interfere with each other. You can choose to use a specific version of the plugin when creating a Connect task as needed.