feat(inkless): POD-2457 Allow transactional offset commits for Diskless sources#596
Conversation
…ss sources Permit TxnOffsetCommitRequest to reference Diskless topic-partitions as offset keys, enabling transactional offset commits for consumers reading from Diskless topics.
There was a problem hiding this comment.
Pull request overview
This PR enables transactional offset commits for consumers reading from Diskless topics by allowing TxnOffsetCommitRequest to use Diskless topic-partitions as offset keys, while keeping transactional production to Diskless topics unsupported.
Changes:
- Removes the Kafka API rejection of Diskless topics in transactional offset commit requests.
- Updates unit and integration tests to cover successful transactional offset commits for Diskless sources.
- Updates Inkless feature documentation to distinguish transactional offset commits from unsupported transactional production paths.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
core/src/main/scala/kafka/server/KafkaApis.scala |
Allows validated Diskless topic partitions to flow to the group coordinator for transactional offset commits. |
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala |
Updates the Diskless transactional offset commit unit test to expect success and verify coordinator delegation. |
core/src/test/java/kafka/server/InklessLegacyClusterTest.java |
Adds an end-to-end legacy cluster test for sendOffsetsToTransaction with a Diskless source topic. |
docs/inkless/FEATURES.md |
Documents transactional offset commits as supported and clarifies remaining unsupported transactional operations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Do you have some broader context here? Why can we allow transactional offset commits? |
|
@viktorsomogyi it turns out it prevents diskless topics from participating in transactional offset commits, even if there were not writes to them. Particularly, Iceberg connector cannot work on a diskless topic as source (i.e. without writes). |
…ss sources (#596) Permit TxnOffsetCommitRequest to reference Diskless topic-partitions as offset keys, enabling transactional offset commits for consumers reading from Diskless topics.
Permit TxnOffsetCommitRequest to reference Diskless topic-partitions as offset keys, enabling transactional offset commits for consumers reading from Diskless topics.