Skip to main content

​
Overview

The Splunk Sink Connector consumes records from Kafka topics and writes them to Splunk through the Splunk HTTP Event Collector (HEC). It sits between Kafka and Splunk and can continuously deliver application logs, audit events, metric data, and business events for search, analysis, and alerting. By default, the connector uses the HEC /event endpoint, wraps record values as events, and can assign different indexes, sources, and sourcetypes to different topics. It can also use the /raw endpoint for raw content or override Splunk metadata dynamically through Kafka headers. Tasks send records in batches. With HEC acknowledgement disabled by default, a successful HTTP response marks a batch as complete; when acknowledgement is enabled, the connector waits for the corresponding Splunk indexer acknowledgement before advancing Kafka offsets. Recovery can still replay records whose offsets have not been committed, so downstream processing should tolerate duplicate events.

​
Prerequisites

  • HEC is enabled in Splunk, a usable HEC URI and Token are available, and the Token has permission to write to the default or explicitly configured index.
  • When HTTPS certificate validation is enabled, every Worker that may run a Task can read a trust store file containing the required CA chain.
  • When HEC acknowledgement is enabled, indexer acknowledgement is enabled for the corresponding Token in Splunk.

​
License

Uses Apache License 2.0.

​
Quick Start

Prepare a Connect Cluster, Kafka input Topic, and Splunk HEC in advance. Confirm network connectivity, Token permissions, and Worker access to the trust store. For creation and management operations, see Manage Connectors. The following configuration uses the HTTPS /event endpoint and relies on the HEC Token’s defaults for the index, source, and sourcetype.
Replace the input Topic, HEC address, Token, and Worker-local trust store information. The default trust store type is JKS; when using another JVM-supported type, add splunk.hec.ssl.trust.store.type. Supply the Token and trust store password through a controlled configuration management mechanism. This example inherits the Worker’s Converters. Explicitly configure key.converter and value.converter when the actual record encoding does not match them.

​
Configuration

​
HEC Connection and Authentication

​
splunk.hec.token

The Token used to access Splunk HEC.
  • Type: password
  • Default: None, required
  • Importance: High
  • Valid Values / Notes: Use an enabled Token that is authorized to write to the target index. Manage it as a sensitive configuration value and do not include it in logs or shared examples.

​
splunk.hec.uri

The base URI for Splunk HEC.
  • Type: string
  • Default: None, required
  • Importance: High
  • Valid Values / Notes: Multiple HTTP or HTTPS URIs may be separated by commas. HTTPS is enforced by default. Avoid whitespace before or after each URI entry.

​
splunk.validation.disable

Whether to skip live requests to Splunk during Connector configuration validation.
  • Type: boolean
  • Default: false
  • Importance: Medium
  • Valid Values / Notes: false sends a validation event to the configured index through the first HEC URI. true skips only this remote validation; it does not skip required-field, URI, Kerberos, or Task configuration checks.

​
kerberos.user.principal

The Kerberos Principal used for HEC HTTP requests.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: When Kerberos is enabled, set this together with kerberos.keytab.path. Kerberos does not replace the HEC Token.

​
kerberos.keytab.path

The Worker-local path to the Kerberos Keytab.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: When Kerberos is enabled, set this together with kerberos.user.principal, and ensure that every Worker hosting a Task can read the file.

​
TLS Security

​
splunk.hec.ssl.enforced

Whether HEC URIs must use HTTPS.
  • Type: boolean
  • Default: true
  • Importance: High
  • Valid Values / Notes: true rejects HTTP. When set to false, only HTTP or HTTPS is still accepted. HTTP transmits events and the Token in plaintext.

​
splunk.hec.ssl.validate.certs

Whether HTTPS connections validate certificates and hostnames.
  • Type: boolean
  • Default: true
  • Importance: Medium
  • Valid Values / Notes: When kept at true, a nonempty splunk.hec.ssl.trust.store.path must be configured. Disable validation only for controlled diagnostics or when the risk is explicitly accepted.

​
splunk.hec.ssl.trust.store.path

The path to the trust store file used to validate Splunk certificates.
  • Type: string
  • Default: Empty string
  • Importance: High
  • Valid Values / Notes: Required when HTTPS and certificate validation are enabled. The path must be visible and readable to every Task process.

​
splunk.hec.ssl.trust.store.type

The trust store type.
  • Type: string
  • Default: JKS
  • Importance: Low
  • Valid Values / Notes: Common values include JKS and PKCS12. Actual support depends on the JVM security providers.

​
splunk.hec.ssl.trust.store.password

The password required to load the trust store.
  • Type: password
  • Default: Empty string
  • Importance: High
  • Valid Values / Notes: Enter the password configured for the trust store and manage it as a sensitive configuration value. Empty string is the ConfigDef default.

​
Endpoint and Event Format

​
splunk.hec.raw

Selects the HEC /raw or /event endpoint.
  • Type: boolean
  • Default: false
  • Importance: Medium
  • Valid Values / Notes: false uses the JSON /event endpoint. true uses /raw, where metadata applies to the entire request batch and event-field enrichment and custom Header fields do not apply.

​
splunk.hec.raw.line.breaker

Appends a delimiter to each record in /raw mode.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: Used only when splunk.hec.raw=true. It should match the line-breaking and event-segmentation rules of the Splunk sourcetype.

​
splunk.hec.json.event.formatted

Attempts to parse the record value as a complete HEC JSON Event object.
  • Type: boolean
  • Default: false
  • Importance: Low
  • Valid Values / Notes: Used only with /event. If parsing fails, the record value falls back to ordinary event content rather than retaining preformatted-object semantics.

​
Splunk Metadata Routing

​
splunk.indexes

Specifies the Splunk index for explicitly subscribed Topics.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: An empty value uses the HEC Token’s default index. Supply one value shared by all Topics, or the same number of comma-separated values in topics order. topics.regex does not support this positional mapping.

​
splunk.sources

Specifies the Splunk source for explicitly subscribed Topics.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: Supply one shared value, or the same number of comma-separated values in topics order. An empty value uses the HEC default, and topics.regex does not support positional mapping.

​
splunk.sourcetypes

Specifies the Splunk sourcetype for explicitly subscribed Topics.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: Supply one shared value, or the same number of comma-separated values in topics order. An empty value uses the HEC default, and topics.regex does not support positional mapping.

​
Kafka Header Mapping

​
splunk.header.support

Enables mapping from Kafka Headers to Splunk metadata and fields.
  • Type: boolean
  • Default: false
  • Importance: Medium
  • Valid Values / Notes: When enabled, configured Headers can override the index, host, source, and sourcetype. /raw groups records by these four metadata categories.

​
splunk.header.custom

Specifies custom Kafka Header names to write as /event fields.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: Separate Header names with commas and also enable splunk.header.support. These Headers are not used as indexed fields with /raw.

​
splunk.header.index

Specifies the Kafka Header name that overrides the Splunk index.
  • Type: string
  • Default: splunk.header.index
  • Importance: Medium
  • Valid Values / Notes: Used only when splunk.header.support=true. If multiple Headers have the same name, the last value is used.

​
splunk.header.source

Specifies the Kafka Header name that overrides the Splunk source.
  • Type: string
  • Default: splunk.header.source
  • Importance: Medium
  • Valid Values / Notes: Used only when splunk.header.support=true. The Header value is converted to a string.

​
splunk.header.sourcetype

Specifies the Kafka Header name that overrides the Splunk sourcetype.
  • Type: string
  • Default: splunk.header.sourcetype
  • Importance: Medium
  • Valid Values / Notes: Used only when splunk.header.support=true. The Header value is converted to a string.

​
splunk.header.host

Specifies the Kafka Header name that overrides the Splunk host.
  • Type: string
  • Default: splunk.header.host
  • Importance: Medium
  • Valid Values / Notes: Used only when splunk.header.support=true. The Header value is converted to a string.

​
Event Fields and Time

​
splunk.hec.json.event.enrichment

Adds fixed fields to /event records.
  • Type: string
  • Default: Empty string
  • Importance: Low
  • Valid Values / Notes: Use comma-separated key=value entries. Each entry must produce exactly one nonempty key and value; for duplicate keys, the last value wins. Does not apply to /raw.

​
splunk.hec.track.data

Adds Kafka position and record-tracking fields to /event.
  • Type: boolean
  • Default: false
  • Importance: Low
  • Valid Values / Notes: May include the Topic, partition, Offset, timestamp, record key, and Connect host information. Assess exposure of sensitive data before enabling it.

​
splunk.hec.use.record.timestamp

Uses the Kafka record timestamp as the event time for ordinary /event events.
  • Type: boolean
  • Default: true
  • Importance: Medium
  • Valid Values / Notes: Successful regular-expression time extraction can override this time. It is not used for /raw or successfully parsed preformatted events.

​
splunk.hec.auto.extract.timestamp

Whether to append the auto_extract_timestamp parameter to HEC requests.
  • Type: boolean
  • Default: null
  • Importance: Low
  • Valid Values / Notes: null means the parameter is not sent. Explicitly setting true or false passes that value to HEC. This is independent of the Connector’s regular-expression time extraction.

​
enable.timestamp.extraction

Enables regular-expression event-time extraction in the Connector.
  • Type: boolean
  • Default: false
  • Importance: Medium
  • Valid Values / Notes: Used only for ordinary /event events. When enabled, timestamp.regex must contain a named group called time, and an applicable time format must be configured.

​
timestamp.regex

The Java regular expression used to extract a time value from event text.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: When extraction is enabled, the expression must contain a (?<time>...) named group. The expression runs against the string representation of the event.

​
timestamp.regex.timeout.ms

The timeout for running the time regular expression on one record.
  • Type: int
  • Default: 500
  • Importance: Medium
  • Valid Values / Notes: In milliseconds, with a minimum of 1. On timeout, the event retains any time value it already had.

​
timestamp.format

The time format used to parse the value extracted by the regular expression.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: Use epoch when extracting Unix Epoch time. Other values are parsed as Java SimpleDateFormat patterns. Set this explicitly when extraction is enabled.

​
timestamp.timezone

The time zone used to parse non-Epoch time.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: Use a valid Java time zone ID. It applies only with regular-expression extraction and a non-Epoch format.

​
HEC Acknowledgement and Retries

​
splunk.hec.ack.enabled

Enables polling for Splunk HEC indexer acknowledgement.
  • Type: boolean
  • Default: false
  • Importance: Medium
  • Valid Values / Notes: Enable indexer acknowledgement for the HEC Token before enabling this setting. When disabled, a successful HTTP response marks a batch as complete but does not prove that its events are durable and searchable. The default is the ConfigDef value false.

​
splunk.hec.event.timeout

The timeout for waiting for HEC acknowledgement.
  • Type: int
  • Default: 300
  • Importance: Medium
  • Valid Values / Notes: In seconds and used only when splunk.hec.ack.enabled=true. Set a positive value appropriate for HEC processing latency.

​
splunk.hec.ack.poll.interval

The interval for polling HEC acknowledgement status.
  • Type: int
  • Default: 10
  • Importance: Medium
  • Valid Values / Notes: In seconds and used only when HEC acknowledgement is enabled. Thread scheduling requires a usable positive value.

​
splunk.hec.ack.poll.threads

The number of threads used to poll HEC acknowledgement status.
  • Type: int
  • Default: 2
  • Importance: Medium
  • Valid Values / Notes: Used only when HEC acknowledgement is enabled and should be a positive integer. The default is the ConfigDef value 2.

​
splunk.hec.max.outstanding.events

Limits the number of in-flight events that have not completed.
  • Type: int
  • Default: 1000000
  • Importance: Medium
  • Valid Values / Notes: Reaching the threshold triggers backpressure and a retriable exception. Set a positive value based on individual record size, batching, acknowledgement latency, and Worker memory.

​
splunk.hec.max.retries

Controls the Connector’s internal retry count for failed batches.
  • Type: int
  • Default: 5
  • Importance: Medium
  • Valid Values / Notes: A positive value limits the retry count for a failed batch. In this implementation, 0 or a negative value does not enter the count limit and behaves as unlimited retries. This differs from the Kafka Connect framework’s errors.retry.timeout.

​
splunk.hec.backoff.threshhold.seconds

Sets the backpressure duration after an HEC channel failure.
  • Type: int
  • Default: 60
  • Importance: Medium
  • Valid Values / Notes: In seconds. The spelling threshhold in the configuration name is the existing public key and must be used verbatim.

​
Batching, Concurrency, and Transport

​
splunk.hec.max.batch.size

Sets the maximum number of records in one HEC batch.
  • Type: int
  • Default: 500
  • Importance: Medium
  • Valid Values / Notes: Measured in records rather than bytes. The ConfigDef default is 500. Use a positive value and adjust it based on individual event size and HEC request limits.

​
splunk.flush.window

Sets the time window for batch sending.
  • Type: int
  • Default: 30
  • Importance: Low
  • Valid Values / Notes: In seconds. Sending is triggered when the batch record count is reached or when a subsequent record arrives after the window has elapsed. Only a positive value replaces the internal window.

​
splunk.hec.threads

Sets the number of threads in each Task that send HEC batches.
  • Type: int
  • Default: 1
  • Importance: Low
  • Valid Values / Notes: A value greater than 1 enables concurrent sending and a bounded queue. This differs from tasks.max; increasing it may change batch completion order.

​
splunk.hec.concurrent.queue.capacity

Sets the concurrent HEC sending queue capacity in batches.
  • Type: int
  • Default: 100
  • Importance: Low
  • Valid Values / Notes: Must be greater than 0. It primarily holds pending batches when splunk.hec.threads>1.

​
splunk.hec.total.channels

Sets the total number of HEC channels used by each Task.
  • Type: int
  • Default: 2
  • Importance: High
  • Valid Values / Notes: Use a positive value. Together with multiple URIs and HEC threads, it determines channel allocation; it is not the number of Tasks.

​
splunk.hec.max.http.connection.per.channel

Sets the upper limit for the HTTP connection pool per target.
  • Type: int
  • Default: 2
  • Importance: Medium
  • Valid Values / Notes: Use a positive value. The total connection pool is also affected by the number of HEC URIs.

​
splunk.hec.http.keepalive

Whether HEC HTTP connections use Keep-Alive.
  • Type: boolean
  • Default: true
  • Importance: Medium
  • Valid Values / Notes: true or false. Normally keep it enabled to reuse connections.

​
splunk.hec.socket.timeout

Sets the Socket timeout declared by the HEC client.
  • Type: int
  • Default: 60
  • Importance: Low
  • Valid Values / Notes: In seconds. Use a positive value based on network conditions and HEC response time.

​
splunk.hec.lb.poll.interval

Sets the health-check interval for HEC URIs.
  • Type: int
  • Default: 120
  • Importance: Low
  • Valid Values / Notes: In seconds. A value less than or equal to 0 disables out-of-band health checks.

​
splunk.hec.enable.compression

Whether HEC request entities use Gzip compression.
  • Type: boolean
  • Default: false
  • Importance: Medium
  • Valid Values / Notes: Applies to both /event and /raw. It can reduce network transfer volume but increases compression processing overhead on the Worker and Splunk.

​
Kafka Connect Identity, Subscription, and Conversion

​
connector.class

Specifies the Connector implementation class.
  • Type: string
  • Default: None, required
  • Importance: High
  • Valid Values / Notes: Use com.splunk.kafka.connect.SplunkSinkConnector.

​
tasks.max

Sets the maximum number of Tasks the Connector may create.
  • Type: int
  • Default: 1
  • Importance: High
  • Valid Values / Notes: The minimum is 1. Effective parallelism is also limited by the number of input Topic partitions. Each Task has an independent HEC client and internal threads.

​
tasks.max.enforce

Whether Kafka Connect enforces the tasks.max limit for the Connector.
  • Type: boolean
  • Default: true
  • Importance: Low
  • Valid Values / Notes: This setting is deprecated and planned for removal in a future major version. Keep it at true and manage Task count through tasks.max; there is no same-named replacement property.

​
topics

Specifies the Kafka Topics consumed by the Connector.
  • Type: list
  • Default: Empty list
  • Importance: High
  • Valid Values / Notes: Use comma-separated values and choose exactly one of this setting and topics.regex. This setting is required when using positional splunk.indexes, splunk.sources, or splunk.sourcetypes values.

​
topics.regex

Subscribes to Kafka Topics with a regular expression.
  • Type: string
  • Default: Empty string
  • Importance: High
  • Valid Values / Notes: Must be a valid Java regular expression, and exactly one of this setting and topics must be used. Positional Splunk metadata lists cannot be used; use HEC defaults or Kafka Headers instead.

​
key.converter

Specifies the Kafka Connect Converter for record keys.
  • Type: class
  • Default: null
  • Importance: Low
  • Valid Values / Notes: null inherits the Worker configuration. An explicit class must be an instantiable Converter. By default, the record key is not sent as event content.

​
value.converter

Specifies the Kafka Connect Converter for record values.
  • Type: class
  • Default: null
  • Importance: Low
  • Valid Values / Notes: null inherits the Worker configuration. The Connector serializes the converted result as text, JSON, or HEC Event content.

​
header.converter

Specifies the Converter for Kafka Headers.
  • Type: class
  • Default: null
  • Importance: Low
  • Valid Values / Notes: null inherits the Worker configuration. When splunk.header.support is enabled, ensure that Header conversion produces the expected values.

​
transforms

Specifies the aliases of single-message transforms (SMTs) applied in order before SinkTask processing.
  • Type: list
  • Default: Empty list
  • Importance: Low
  • Valid Values / Notes: Aliases must be unique. Each transform also requires transforms.<alias>.type and its transform-specific properties.

​
config.action.reload

Controls how changes or expiration of Config Provider values are handled.
  • Type: string
  • Default: restart
  • Importance: Low
  • Valid Values / Notes: Allowed values are restart and none. Config Provider registration is part of Worker configuration.

​
Kafka Connect Error Handling

​
errors.retry.timeout

Sets the total retry duration for retriable errors in Kafka Connect framework stages.
  • Type: long
  • Default: 0
  • Importance: Medium
  • Valid Values / Notes: In milliseconds. 0 disables retries, and -1 means unlimited retries. This does not replace the Connector’s own HEC batch retries.

​
errors.retry.delay.max.ms

Sets the maximum wait between Kafka Connect framework retries.
  • Type: long
  • Default: 60000
  • Importance: Medium
  • Valid Values / Notes: In milliseconds and used with errors.retry.timeout. It does not affect HEC channel backpressure duration.

​
errors.tolerance

Controls whether the Kafka Connect framework tolerates reportable record-processing errors.
  • Type: string
  • Default: none
  • Importance: Medium
  • Valid Values / Notes: Allowed values are none and all. This primarily applies to framework stages such as conversion and SMTs and does not guarantee coverage of asynchronous HEC failures inside the Connector.

​
errors.log.enable

Whether Kafka Connect framework processing errors are logged.
  • Type: boolean
  • Default: false
  • Importance: Medium
  • Valid Values / Notes: This logging is independent of the Connector’s own logs. When enabled, use it with appropriate log access controls and retention policies.

​
errors.log.include.messages

Whether framework error logs include record context.
  • Type: boolean
  • Default: false
  • Importance: Medium
  • Valid Values / Notes: When enabled, logs may contain the Topic, partition, Offset, timestamp, and other information. Assess the risk of data exposure in logs and use it with errors.log.enable.

​
errors.deadletterqueue.topic.name

Specifies the Kafka Connect framework dead-letter Topic.
  • Type: string
  • Default: Empty string
  • Importance: Medium
  • Valid Values / Notes: An empty value disables the DLQ. It is commonly used with errors.tolerance=all. The DLQ Topic must not be subscribed by topics or topics.regex, and this mechanism does not cover every internal HEC failure in the Connector.

​
errors.deadletterqueue.topic.replication.factor

Sets the replication factor when Kafka Connect automatically creates the DLQ Topic.
  • Type: short
  • Default: 3
  • Importance: Medium
  • Valid Values / Notes: Used only when Connect creates the DLQ Topic. The value must not exceed the number of available Brokers in the Kafka cluster.

​
errors.deadletterqueue.context.headers.enable

Whether Kafka Connect error-context Headers are added to DLQ records.
  • Type: boolean
  • Default: false
  • Importance: Medium
  • Valid Values / Notes: When enabled, adds __connect.errors.* Headers. This is meaningful only when a DLQ is configured.

​
Best Practices

​
Enable HEC Acknowledgement for Production Use

Applicable Scenario: The Quick Start has established basic delivery, and the production environment needs to advance the corresponding Kafka offset only after Splunk returns indexer acknowledgement. The environment can tolerate duplicate events that may occur during failure recovery. Configuration Example: Keep the Quick Start configuration, enable indexer acknowledgement in Splunk for the same HEC Token, and add the following settings.
Key Notes: When indexer acknowledgement is enabled for the HEC Token and acknowledgement polling operates normally, this mode provides at-least-once delivery, not end-to-end transactions or exactly-once delivery. A batch may be replayed if HEC receives it but the acknowledgement response is lost, or after a timeout, restart, or rebalance. Adjust the timeout and polling threads to actual indexing latency and concurrency. Give events stable identifiers or implement deduplication in Splunk when strict deduplication is required.

​
Expand to Fixed Metadata Routing for Multiple Topics

Applicable Scenario: The initial single-Topic integration is stable, and application logs and audit logs now need to be written to different indexes with an explicit source and sourcetype for each data category. Configuration Example: Replace topics in the Quick Start configuration and add metadata lists whose positions correspond one-to-one with the Topic order.
Key Notes: Each metadata list may contain one value shared by all Topics or the same number of values as topics. With multiple values, positions must correspond exactly. This approach does not apply to topics.regex. For dynamic routing, enable Kafka Header mapping or use the HEC Token’s default metadata, and validate producer Header value ranges and access permissions.

​
Increase Sending Parallelism Gradually as Throughput Grows

Applicable Scenario: One Task and one HEC sending thread have been stable, but the Kafka backlog continues to grow while Splunk HEC and the network still have spare capacity. More Tasks, connections, and compression are needed to increase throughput. Configuration Example: Assuming at least four input partitions and sufficient HEC concurrency capacity, add the following starting values to the Quick Start configuration, then adjust them individually through load testing.
Key Notes: Effective Task count is limited by the number of input partitions, while HEC threads and channels apply to each Task. More parallelism increases Worker memory usage, connections, and Splunk processing pressure, and it does not guarantee global ordering across partitions, Tasks, channels, or batches. Adjust only one group of capacity settings at a time, observe Kafka lag, HEC responses, retries, Worker heap memory, and CPU, and then decide whether to scale further.

​
Monitoring

​
What to Monitor

Monitor Kafka Connect cluster health, Connector and Task status, consumption and HEC write throughput, Kafka lag and end-to-end latency, Offset commit progress, errors and retries, and Worker JVM heap, GC, and thread signals. Also monitor HEC health, acknowledgement timeouts, backpressure, and duplicate events. Monitor DLQ activity only when the corresponding framework error handling is enabled in the deployment; do not treat the DLQ as a fallback for all HEC delivery failures.

​
Import the Grafana Dashboard

Download the AutoMQ Connect Cluster Grafana Dashboard, prepare a Prometheus-compatible data source that can query Kafka Connect metrics, and ensure that collected labels match the dashboard filters. On Grafana’s import page, upload the JSON, select the corresponding data source, and save the dashboard.

​
Limitations

  • With HEC acknowledgement enabled and its conditions satisfied, delivery is at least once; retries, crashes, rebalances, or missing acknowledgement responses can still produce duplicate events. Without acknowledgement, the connector completes a batch after a successful HTTP response and provides no indexer-acknowledgement protection. Neither mode provides exactly-once delivery or destination-side deduplication.
  • When Splunk returns Invalid data format, the Connector treats the response as completed and may advance the Kafka Offset even if the corresponding event is ignored.
  • topics.regex cannot use positional mapping from splunk.indexes, splunk.sources, and splunk.sourcetypes; use Kafka Headers or HEC Token defaults instead.
  • When HTTPS certificate validation is enabled, a custom trust store path readable by the Worker must be configured.
  • In /raw mode, Splunk metadata applies to the entire request batch, and custom Kafka Headers are not added to raw events as indexed fields.

​
FAQ

​
The Connector Is Running Normally, but No Events Appear in Splunk

Check Task logs and status, HEC health, whether the Token is enabled and allowed to write to the target index, and whether the URI, endpoint mode, and /raw line-breaking rules are correct. If HEC acknowledgement is enabled, verify the Splunk Token’s indexer acknowledgement setting. If Splunk returns Invalid data format, the Connector may already have advanced the Offset. After correcting HEC, permissions, or event formatting, validate again with one small, valid event.

​
Duplicate Events Appear After Recovery or Restart

HEC may have received a batch before the Kafka Offset was committed, or the acknowledgement response may have been lost during a timeout, connection switch, or rebalance. Check HEC acknowledgement and retry logs, Task restart records, and Offset progress. Enable indexer acknowledgement correctly and set the timeout according to actual latency. For data that cannot tolerate duplicates, retain a stable identifier in each event and deduplicate during queries or downstream processing.

​
Kafka Lag Keeps Growing or the Same Batch Is Replayed Repeatedly

Check the HEC response, acknowledgement polling, failure retries, and in-flight event count for the earliest incomplete batch, and confirm that at least one HEC channel is healthy. Restore the Token, index, network, or HEC capacity first, then evaluate parallelism. Do not reset Offsets before confirming the data boundary, because doing so may cause data loss or more duplicates.

​
Trust Store Errors Occur During HTTPS Validation or Task Startup

Confirm that splunk.hec.ssl.trust.store.path exists and is readable on every Worker, that its type and password are correct, that the trust store contains the issuing chain for the Splunk certificate, and that the URI hostname matches the certificate. Restart the Task after correcting the trust store. Do not use disabled certificate validation as a long-term solution.

​
Events Are Written to the Wrong Index, Source, or Sourcetype

Check the order of topics, the number and order of the three metadata lists, and whether Kafka Header mapping overrides the static values. With topics.regex, positional mapping does not take effect; use Headers or the HEC Token defaults instead. /raw also sends a batch with one set of request metadata, so ensure that records in the same batch can share that metadata.

​
Event Times in Splunk Do Not Match Expectations

Check the Kafka record timestamp, splunk.hec.use.record.timestamp, the regular-expression named group, timeout logs, timestamp.format, timestamp.timezone, and HEC automatic time extraction settings in that order. Choose one primary time source and validate it with representative events. Avoid enabling extraction paths that override one another.