A sample repo that sets up a Kafka Broker with Zookeeper, with SSL authentication and Access Control Lists configured.

Before running the sample, generate the certificates.

./certificates/setup.sh

This requires Java to be installed (more specifically, it uses the keytool command).

To avoid installing Java on the host machine, do the following:

# Host
docker run --rm -it -v $(pwd):$(pwd) -w $(pwd) amazoncorretto:21

# Docker container
yum install -y openssl
./certificates/setup.sh
exit

# Host
sudo chown -R $(id -u):$(id -g) ./certificates

docker compose up -d setup

The docker-compose.yml file has dependencies configured so the consumer service will only start after the other services are up / completed.

docker compose down
docker compose up --exit-code-from consumer

Because the services should have completed, use the logs to determine what has happened.

• Check the logs for all containers at once:

  docker compose logs

• Check the logs for individual containers:

  docker compose logs zookeeper
  docker compose logs broker
  docker compose logs setup
  docker compose logs producer
  docker compose logs consumer

If you would like to access the Kafka cluster from other machines on the network, you may need to open the ports on the firewall.

sudo ufw allow 2181
sudo ufw allow 2888
sudo ufw allow 3888
sudo ufw allow 8080
sudo ufw allow 9092
sudo ufw allow 9094

docker compose down