feat(inkless): resolve client AZ from listener map in metadata transformer#684
Merged
Conversation
gqmelo
changed the base branch from
main
to
gqmelo/producer-rack-add-listener-map-config
July 6, 2026 08:51
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request adds listener-based client AZ resolution for diskless-topic metadata routing, allowing clients without a diskless_az=<az> marker in client.id to still receive AZ-aware leader selection based on the request listener.
Changes:
- Extend
InklessTopicMetadataTransformerto resolve client AZ viaclient.idmarker first, then via a new listener→AZ mapping, else fall back to non-AZ-aware routing. - Add and validate new broker config
inkless.client.az.listener.map(comma-separatedLISTENER=azpairs) and expose it viaInklessConfig.clientAzListenerMap(). - Add unit + integration tests for parsing and listener-based inference, and document the new config.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| storage/inkless/src/main/java/io/aiven/inkless/metadata/InklessTopicMetadataTransformer.java | Adds listener-based AZ inference with explicit client.id marker taking precedence. |
| storage/inkless/src/main/java/io/aiven/inkless/config/InklessConfig.java | Introduces client.az.listener.map config, parsing/validation, and an accessor for the normalized map. |
| core/src/main/scala/kafka/server/KafkaApis.scala | Wires the new config map into the metadata transformer construction. |
| storage/inkless/src/test/java/io/aiven/inkless/metadata/InklessTopicMetadataTransformerTest.java | Updates constructor usage and adds coverage for listener-based AZ resolution behavior. |
| storage/inkless/src/test/java/io/aiven/inkless/config/InklessConfigTest.java | Adds tests for parsing, normalization, defaults, and validation failures of the new config. |
| core/src/test/java/kafka/server/InklessManagedReplicasClusterTest.java | Adds integration coverage for listener-inferred AZ without client.id marker. |
| docs/inkless/configs.rst | Documents the new client.az.listener.map configuration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
gqmelo
force-pushed
the
gqmelo/producer-rack-metadata
branch
from
July 6, 2026 09:52
35dacfc to
74cae99
Compare
gqmelo
force-pushed
the
gqmelo/producer-rack-metadata
branch
from
July 6, 2026 11:27
74cae99 to
191d880
Compare
Base automatically changed from
gqmelo/producer-rack-add-listener-map-config
to
main
July 7, 2026 13:29
…ormer This adds support for producers to be rack-aware without having to change their client IDs. We are still keeping compatibility with `client.id=my-app,diskless_az=<AZ>` and it takes precedence over the listener option. For the listener option, all brokers will keep a config mapping LISTENER_NAME=AZ, then clients connect to the exact listener based on the AZ they want. When handling the metadata request, the listener used for the request is mapped back to the AZ associated to that listener and a leader in that AZ is returned. [KC-258]
gqmelo
force-pushed
the
gqmelo/producer-rack-metadata
branch
from
July 7, 2026 13:45
191d880 to
61fccd7
Compare
gqmelo
marked this pull request as ready for review
July 7, 2026 13:45
giuseppelillo
approved these changes
Jul 7, 2026
jeqo
pushed a commit
that referenced
this pull request
Jul 8, 2026
…ormer (#684) This adds support for producers to be rack-aware without having to change their client IDs. We are still keeping compatibility with `client.id=my-app,diskless_az=<AZ>` and it takes precedence over the listener option. For the listener option, all brokers will keep a config mapping LISTENER_NAME=AZ, then clients connect to the exact listener based on the AZ they want. When handling the metadata request, the listener used for the request is mapped back to the AZ associated to that listener and a leader in that AZ is returned. [KC-258]
jeqo
pushed a commit
that referenced
this pull request
Jul 8, 2026
…ormer (#684) This adds support for producers to be rack-aware without having to change their client IDs. We are still keeping compatibility with `client.id=my-app,diskless_az=<AZ>` and it takes precedence over the listener option. For the listener option, all brokers will keep a config mapping LISTENER_NAME=AZ, then clients connect to the exact listener based on the AZ they want. When handling the metadata request, the listener used for the request is mapped back to the AZ associated to that listener and a leader in that AZ is returned. [KC-258]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This was created on top #683 so the base branch isn't
mainyet. It will be updated once the other PR is merged.This adds support for producers to be rack-aware without having to change their client IDs.
We are still keeping compatibility with
client.id=my-app,diskless_az=<AZ>and it takes precedence over thelistener option.
For the listener option, all brokers will keep a config mapping LISTENER_NAME=AZ, then clients connect to the exact listener based on the AZ they want.
When handling the metadata request, the listener used for the request is mapped back to the AZ associated to that listener and a leader in that AZ is returned.
KC-258