There was an error while loading. Please reload this page.
This document describes how to a setup kafka sink locally using Docker.
The simplest kafka setup:
git clone https://github.com/confluentinc/kafka-images.git cd kafka-images/examples/kafka-single-node docker-compose up -d
For more details see here.
Creates a source that produces events consistently and a kafka sink that consumes them.
[sources.demo_logs] format = "json" interval = 1.0 type = "demo_logs" [sinks.demo_kafka] type = "kafka" inputs = [ "demo_logs" ] bootstrap_servers = "0.0.0.0:9092" topic = "test" compression = "gzip" encoding.codec = "json" librdkafka_options."message.max.bytes" = "1000000"
Open one terminal and run:
path/to/vector/target/debug/vector -v --config path/to/config_provided_above
Open another terminal and run:
docker exec -it kafka-single-node-kafka-1 /bin/bash /bin/kafka-console-consumer --topic=test --bootstrap-server=localhost:9092
The above terminal should output events.