Keycloak Kafka Module

Simple module for Keycloak to produce keycloak events to Kafka.

Fork notice. This is a maintained fork of SnuK87/keycloak-kafka (Apache License 2.0). It continues development that stalled upstream — notably support for current Apache Kafka and Keycloak releases (see the compatibility table below), a KRaft-based docker-compose, expanded tests, and a renamed com.opusdns namespace. See NOTICE for attribution.

• Keycloak Kafka Module
  • Build
  • Installation
  • Module Configuration
    • Kafka client configuration
    • Kafka client using secure connection
  • Module Deployment
  • Keycloak Configuration
    • Enable Events in keycloak
  • Docker Container
  • Sample Client

Compatibility

Why this range holds:

• The artifact is compiled to Java 17 bytecode, so it loads on any Keycloak server running Java 17+.
• The Keycloak SPI is provided — supplied by the server at runtime, not bundled in the jar — and the APIs used (EventListenerProviderFactory, Event, AdminEvent, Config.Scope) are stable across the range.
• The bundled Kafka client 4.3.1 talks to any broker 2.1 or newer (KIP-896).

CI builds and runs the integration test against the newest versions (the Built & tested against column). The lower bound is verified to compile against Keycloak 19.0.3 / Kafka 3.3.2; intermediate versions are expected to work but are not exercised. To build against a specific version, override the Maven properties, e.g. mvn clean package -Dkeycloak.version=19.0.3 -Dkafka.version=3.3.2.

You can simply use Maven to build the jar file. Thanks to the assembly plugin the build process will create a fat jar that includes all dependencies and makes the deployment quite easy. Just use the following command to build the jar file.

mvn clean package

Unit tests can be run locally with Maven or via Docker (no local Maven installation required):

docker compose --profile test run --rm test

Maven dependencies are cached in a named Docker volume (maven-cache) so subsequent runs are fast.

There is also an end-to-end integration test (KafkaEventListenerProviderIT) that produces an event to a real Kafka broker started via Testcontainers and consumes it back. It runs during mvn verify and skips itself automatically when no Docker daemon is available, so it needs an accessible Docker socket (CI runners and most local Docker setups provide one):

mvn verify

First you need to build or download the keycloak-kafka module.

To install the module to your keycloak server you have to configure the module and deploy it. If you deploy the module without configuration, your keycloak server will fail to start throwing a NullPointerException.

If you want to install the module manually as described in the initial version you can follow this guide.

The following properties can be set via environment variables (e.g. ${KAFKA_TOPIC}) or as parameters when starting keycloak (e.g. --spi-events-listener-kafka-topic-events).

• topicEvents (env KAFKA_TOPIC): The name of the kafka topic to where the events will be produced to.

• clientId (env KAFKA_CLIENT_ID): The client.id used to identify the client in kafka.

• bootstrapServers (env KAFKA_BOOTSTRAP_SERVERS): A comma separated list of available brokers.

• events (env KAFKA_EVENTS): A comma separated list of the event types that will be sent to kafka. Defaults to REGISTER when not set. Unknown event types are skipped and logged with a warning, so check the logs if an expected event never arrives.

• topicAdminEvents (env KAFKA_ADMIN_TOPIC): (Optional) The name of the kafka topic to where the admin events will be produced to. No events will be produced when this property isn't set.

A list of available events can be found here

It's also possible to configure the kafka client with environment variables or by adding parameters to the keycloak start command. This makes it possible to connect this module to a kafka broker that requires SSL/TLS connections. For example to change the timeout of how long the producer will block the thread to 10 seconds you just have to pass the following parameter to the start command.

./kc.sh start --spi-events-listener-kafka-max-block-ms 10000

Or set the following environnment variable.

KAFKA_MAX_BLOCK_MS=10000

A full list of available configurations can be found in the official kafka docs.

As mentioned above the kafka client can be configured by passing parameters to the start command. To make kafka open a SSL/TLS secured connection you can add the following parameters:

./kc.sh start \
  --spi-events-listener-kafka-security-protocol SSL \
  --spi-events-listener-kafka-ssl-truststore-location kafka.client.truststore.jks \
  --spi-events-listener-kafka-ssl-truststore-password test1234

Events are produced asynchronously (fire-and-forget): onEvent does not wait for the broker to acknowledge a send. If a send fails the error is logged but the event is not retried and may be lost. Tune the relevant Kafka producer settings to match your durability needs, e.g.:

• acks — set to all for the strongest delivery guarantee.
• retries / delivery.timeout.ms — how hard the client retries before giving up.
• max.block.ms — how long a send may block when the broker is unreachable or buffers are full. The Kafka default is 60 seconds, and this blocking happens on the Keycloak request thread — set it to a low value (e.g. KAFKA_MAX_BLOCK_MS=5000 or lower) so an unreachable broker cannot stall logins for a minute per event.

All of these can be set via the KAFKA_* env vars or --spi-events-listener-kafka-* parameters described above.

Copy the keycloak-kafka-<version>-jar-with-dependencies.jar into the $KEYCLOAK_HOME/providers folder. Keycloak will automatically install the module with all it's dependencies on start up.

1. Open administration console
2. Choose realm
3. Go to Events
4. Open Config tab and add kafka to Event Listeners.

The simplest way to enable the kafka module in a docker container is to create a custom docker image from the keycloak base image. A simple example can be found in the Dockerfile. When you build this image on your local machine by using docker build . -t keycloak-kafka, you can test everything by running the docker-compose file on your local machine. This just provides a simple example to show how it's working. Please consider to read this documentation and create your own Dockerfile.

The following snippet shows a minimal Spring Boot Kafka client to consume keycloak events. Additional properties can be added to the KeycloakEvent class.

@SpringBootApplication
@Log4j2
public class KafkaConsumerApplication {

	public static void main(String[] args) {
		SpringApplication.run(KafkaConsumerApplication.class, args);
	}

	@KafkaListener(topics = "keycloak-events", groupId = "event-consumer")
	public void handleKeycloakEvent(KeycloakEvent event) {
		log.info("Consumed event: " + event);
	}

	@KafkaListener(topics = "keycloak-admin-events", groupId = "event-consumer")
	public void handleKeycloakAdminEvent(KeycloakAdminEvent event) {
		log.info("Consumed admin event: " + event);
	}

	@Bean
	public StringJsonMessageConverter jsonConverter() {
		return new StringJsonMessageConverter();
	}
}

@Data
class KeycloakEvent {
	private String userId;
	private String type;
}

@Data
class KeycloakAdminEvent {
	private String realmId;
	private String operationType;
}

Any kind of contributions are welcome.