Skip to main content

​
Overview

The SAP ODP Source Connector reads data from SAP Operational Data Provisioning (ODP) sources and writes ODP records to a Kafka topic. It uses SAP Java Connector (JCo) to connect to SAP, opens an ODP subscription, reads field metadata, fetches data in packages, and closes the subscription when the task stops. Each record is represented as a Connect Struct; decimal fields are mapped to FLOAT64 and other fields are mapped to strings. The connector is intended to bring SAP ODP datasets into Kafka for downstream streaming, integration, or analytics. I_EXTRACTION_MODE selects the extraction mode passed to the ODP open request, and Topic selects the Kafka topic that receives all records.

​
Prerequisites

  • The SAP system has enabled the target ODP source and provides the ODP context, subscriber type, subscriber name, and subscriber process used by the connector.
  • The Connect Worker plugin path contains the connector files from the ZIP, and the Worker also provides a SAP Java Connector runtime compatible with its platform; SAP JCo must be obtained separately through SAP.
  • The SAP account can call RODPS_REPL_ODP_OPEN, RODPS_REPL_ODP_GET_DETAIL, RODPS_REPL_ODP_FETCH, and RODPS_REPL_ODP_CLOSE.

​
License

Uses Apache License 2.0.

​
Quick Start

Prepare a Connect Cluster, Kafka, and an accessible SAP ODP source. Confirm the SAP connection details, ODP subscription parameters, and Kafka topic are ready. For general connector management, see AutoMQ’s Manage Connectors. The following is a minimum configuration. Replace the values in angle brackets with values from your environment and submit these entries as the connector configuration.
Use jco.client.passwd only for the SAP connection. Do not put it in logs, tickets, or source control. I_EXTRACTION_MODE=D uses the connector’s default extraction mode; confirm the corresponding subscription and source semantics before using another mode.

​
Configuration

​
SAP JCo Connection

​
jco.client.ashost

The SAP application server host name or address.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: An address reachable through SAP JCo. Required.

​
jco.client.sysnr

The SAP system number.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: A system number accepted by SAP. Required.

​
jco.client.client

The SAP client number.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: The target SAP client. Required.

​
jco.client.user

The user name used to connect to SAP.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: The account must be authorized to call the target ODP RFC functions. Required.

​
jco.client.passwd

The password for the SAP user.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: Required. Treat as sensitive data and protect it with Connect security or external secret management; do not log it.

​
jco.client.lang

The SAP logon language code.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: A language code accepted by the SAP system. Required.

​
jco.destination.pool_capacity

The SAP JCo destination pool capacity.
  • Type: int
  • Default: None
  • Importance: High
  • Valid values / notes: An integer accepted by SAP JCo. Required; size it according to SAP connection capacity.

​
jco.destination.peak_limit

The peak connection limit for the SAP JCo destination pool.
  • Type: int
  • Default: None
  • Importance: High
  • Valid values / notes: An integer accepted by SAP JCo. Required; keep it within the capacity allowed by SAP.

​
ODP Subscription

​
I_ODPNAME

The ODP name to read.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: An ODP name available in the SAP system. Required.

​
I_CONTEXT

The ODP context.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: Must match the context used by the target ODP in SAP. Required.

​
I_SUBSCRIBER_TYPE

The ODP subscriber type.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: A subscriber type accepted by the SAP ODP RFC interface. Required.

​
I_SUBSCRIBER_NAME

The ODP subscriber name.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: The name used by the SAP ODP subscription. Required.

​
I_SUBSCRIBER_PROCESS

The ODP subscriber process.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: A process accepted by the target SAP ODP RFC functions. Required.

​
I_EXTRACTION_MODE

The extraction mode used when opening the ODP subscription.
  • Type: string
  • Default: D
  • Importance: High
  • Valid values / notes: The ConfigDef recommender exposes D, F, and R. Project materials describe D as delta extraction and F as full extraction; confirm the target SAP ODP semantics before using R.

​
Kafka Output

​
Topic

The Kafka topic that receives ODP records.
  • Type: string
  • Default: None
  • Importance: High
  • Valid values / notes: An existing topic to which the Connect Worker can write. Required.

​
Fetch Packages

​
I_MAXPACKAGESIZE

The package size sent with each SAP ODP fetch request.
  • Type: int
  • Default: None
  • Importance: High
  • Valid values / notes: A positive integer accepted by the SAP ODP RFC interface. Larger values increase the amount of data in a single request and its memory and processing pressure. Required.

​
Best Practices

​
Establish an SAP ODP baseline on first connection

Applicable scenario: You are connecting an SAP ODP source to Kafka for the first time and need to read the source’s full data before continuing with subsequent data. Use the full quick-start configuration, set I_EXTRACTION_MODE to F, and keep the subscriber parameters that match the target ODP.
Key points: Full extraction is selected by the SAP ODP open request, and its availability depends on the source ODP. Before the first run, confirm the target topic retention and the downstream consumer start position so the full baseline matches the downstream initialization plan.

​
Receive only subsequent changes when a baseline already exists

Applicable scenario: Downstream systems already have a historical baseline created by another process and only need subsequent changes from SAP ODP. Use the full quick-start configuration, keep the existing ODP and subscriber identity, and set I_EXTRACTION_MODE to D.
Key points: D is the ConfigDef default, but the SAP ODP source must still support delta semantics for the target dataset. The connector does not provide source partition or source offset values in SourceRecord; downstream systems should define their own duplicate handling and restart reconciliation strategy.

​
Balance throughput and request pressure with package size

Applicable scenario: The connector reads SAP ODP data successfully, but you need to adjust the size of each fetch according to data volume and SAP system load. Keep the other settings unchanged and adjust I_MAXPACKAGESIZE.
Key points: Larger packages can reduce request frequency but increase per-request SAP RFC processing and memory pressure. Smaller packages make request size easier to control but can increase call frequency. Adjust gradually while observing SAP load, Worker memory, and topic consumer lag.

​
Monitoring

​
What to Monitor

Monitor Kafka Connect Worker, Connector, and Task status; record throughput; processing latency; Source Offset commits; errors; retries; task restarts; and Worker JVM heap and thread usage. Also monitor SAP RFC errors and connection-pool pressure. Because this implementation supplies empty source partition and source offset values, do not use Offset metrics alone to determine whether the SAP ODP subscription has advanced reliably.

​
Import the Grafana Dashboard

Download the Dashboard, use a data source containing Kafka Connect Worker, Connector, and Task metrics, and import the JSON in Grafana with the matching data source and labels.

​
Limitations

  • The connector creates only one Task configuration; increasing tasks.max does not split one SAP ODP source across multiple Tasks.
  • The connector creates SourceRecord values without source partition or source offset, so it cannot be described as a Kafka Connect Source Offset recovery mechanism.
  • Running the connector requires the SAP Java Connector runtime and SAP ODP RFC functions; SAP JCo is not included in the public connector ZIP.

​
FAQ

​
Why does the connector fail to connect to SAP after startup?

Check jco.client.ashost, jco.client.sysnr, jco.client.client, jco.client.user, jco.client.passwd, and jco.client.lang against the SAP logon parameters. Confirm that the Connect Worker can load the SAP JCo native library and Java dependencies. Then verify the account’s permissions for the target ODP RFC functions and that the SAP application server accepts connections from the Worker.

​
Why does configuration validation report missing parameters?

Except for I_EXTRACTION_MODE, the connector-specific settings have no ConfigDef default. Confirm that the SAP JCo, ODP subscription, topic, and I_MAXPACKAGESIZE settings in the quick-start example are all populated, and check the spelling, case, and underscores in each property name.

​
Why are there no records in the Kafka topic?

Confirm that Topic is the intended Kafka topic and that the connector account can write to it. Then check that I_ODPNAME, I_CONTEXT, I_SUBSCRIBER_TYPE, I_SUBSCRIBER_NAME, and I_SUBSCRIBER_PROCESS identify an existing, subscribable ODP in SAP. With D, confirm that the source has data available for delta extraction; with F, confirm that the ODP supports full extraction and inspect the SAP RFC result.

​
Why do latency or errors increase after using a larger package size?

I_MAXPACKAGESIZE is passed to RODPS_REPL_ODP_FETCH; a larger value increases the size and processing pressure of one SAP RFC response. Reduce it gradually while observing SAP connection-pool usage, Worker JVM memory, Task latency, and errors until the request size is balanced with system load.