Kafka Plugin
The Kafka plugin gives Drift the ability to publish to and consume from Apache Kafka topics. It is used alongside the AsyncAPI plugin to verify message-based APIs whose AsyncAPI document declares the kafka protocol.
Drift connects to your broker and exchanges real messages. You must have a running Kafka broker, and the topics in your document must already exist.
Property | Value |
|---|---|
Plugin name |
|
Purpose | Kafka transport for AsyncAPI operations |
Protocol |
|
Used with |
|
Supports | Asynchronous and request/reply interactions |
Enabling the Plugin
plugins: - name: asyncapi - name: kafka - name: json
The plugin activates for AsyncAPI documents whose server declares protocol: kafka.
Plugin | Role |
|---|---|
| Reads the document and plans the interaction |
| Connects to the broker and moves messages |
| Builds and validates message payloads |
The plugin activates for AsyncAPI documents whose server declares protocol: kafka.
Supported Execution Modes
afka is the most complete transport. All four execution modes are available.
Mode | Supported | What Drift does |
|---|---|---|
| Yes | Subscribes to the topic and captures the message your service publishes |
| Yes | Publishes to the topic, then runs your probe |
| Yes | Subscribes to a probe topic, publishes, then captures the harness result |
| Yes | Subscribes to the reply topic, publishes the request, captures the correlated reply |
Connection Details
Drift reads the broker address from your AsyncAPI servers block:
servers:
local:
host: localhost:9092
protocol: kafka
description: Local Kafka brokerOverride it per environment through plugin configuration, or per test through the test case. Test case values win, then plugin configuration, then the document.
Channel Addresses
The AsyncAPI channel address is the Kafka topic name:
channels:
orderCreated:
address: drift.examples.order-createdCorrelation
Drift matches captured messages against a correlation identifier carried in a Kafka record header. Declare its location in your message:
components:
messages:
orderCreated:
correlationId:
location: "$message.header#/correlation-id"Declaring the location is required. The value is optional: Drift takes it from the AsyncAPI document, then from the correlation-id parameter in your test case, and generates a UUID when neither supplies one. Drift filters captured records to the one whose header matches, and ignores everything else on the topic.
Set the value yourself when a trigger hook or an expectation needs the same identifier. Where you do set it, give each operation a distinct value - a shared one lets a message from one test satisfy another.
Capture Behavior
When Drift subscribes, it records the current end of the topic log and reads forward from there. Messages published before the subscription are not captured, which is what makes the subscribe-then-trigger ordering reliable.
Drift polls for new records until a matching message arrives or the capture window closes.
Setting | Where | Default | Purpose |
|---|---|---|---|
| Test case parameter | 30000 | How long Drift waits for a matching message |
| Plugin configuration | 500 | How often Drift polls the broker between fetches |
parameters: timeout-ms: 5000
Lower poll-interval-ms reduces capture latency at the cost of more requests to the broker. The default suits local development and CI.
Creating Topics
Drift does not create topics. Create every topic in your AsyncAPI document, and any probe topic, before the run.
Kafka's automatic topic creation fires only when a producer or consumer first touches a topic. That is too late for async-inject-capture, where Drift publishes to the input topic before any consumer has subscribed.
docker exec <kafka-container> kafka-topics \ --bootstrap-server localhost:9092 \ --create --if-not-exists \ --topic drift.examples.order-created \ --partitions 1 --replication-factor 1
Running a Local Broker
A local broker is the recommended setup for development and CI. It is fast, repeatable, and nothing you publish affects anyone else.
services:
kafka:
image: confluentinc/cp-kafka:latest
ports:
- "9092:9092"
environment:
KAFKA_ENABLE_KRAFT: "yes"
KAFKA_NODE_ID: "1"
KAFKA_PROCESS_ROLES: "broker,controller"
KAFKA_CONTROLLER_LISTENER_NAMES: "CONTROLLER"
KAFKA_LISTENERS: "PLAINTEXT://:9092,CONTROLLER://:9093"
KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://localhost:9092"
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT"
KAFKA_CONTROLLER_QUORUM_VOTERS: "1@kafka:9093"
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: "1"
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: "1"
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: "1"
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: "0"Running against a shared cluster is supported, but that cluster is yours to manage. Messages Drift publishes are real messages, and downstream consumers will process them.
Transport Security
The Kafka plugin speaks the Kafka wire protocol over raw TCP sockets. It does not support authentication or encryption in transit:
No SASL authentication of any mechanism
No TLS, and no mutual TLS
Every connection is plaintext
Point Drift at a broker that accepts plaintext connections, and treat contract verification as something you run against a local, disposable, or otherwise non-sensitive broker rather than a secured production cluster.
Limitations
Drift captures the first record matching the correlation ID. It does not verify ordering across a stream.
Drift does not create topics.
Drift does not verify consumer group behavior, partition assignment, rebalancing, or offset management.
Drift does not verify delivery guarantees, retry behavior, or dead-letter handling.
Authentication and TLS are not supported. Connections are plaintext.
Troubleshooting
Symptom | Likely cause |
|---|---|
Connection refused | The broker is not running, or |
Connection fails against a secured broker | The plugin connects over plaintext TCP only. It cannot reach a broker that requires SASL or TLS. |
Capture timeout, trigger succeeded | The published record carries a different correlation ID, or went to a different topic |
Capture timeout on | The probe topic did not exist before the run |
Topic not found | Create the topic in advance. Automatic creation is too late for Drift. |
Works locally, times out in CI | The broker service container was not ready. Add a readiness check before the Drift step. |
See Debugging Test Cases.