feat(inkless:consolidation): make fetch quota config dynamic#637
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR makes the diskless consolidation fetch quota (diskless.consolidation.fetch.rate.limit.bytes.per.second) dynamically updatable at runtime, aligning it with the existing broker replication throttle update flow.
Changes:
- Adds the consolidation fetch rate limit to the broker’s dynamic-config allowlist and documents it as dynamically updatable.
- Moves consolidation quota manager ownership into
QuotaFactory.QuotaManagersand updatesReplicaManagerto use it. - Applies dynamic updates via
BrokerConfigHandler, usingcurrentKafkaConfigto preserve statically configured values across unrelated dynamic updates, and adds unit tests for both dynamic-update and static-preservation scenarios.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| server-common/src/main/java/org/apache/kafka/server/config/ServerConfigs.java | Documents dynamic updatability and relaxes validation to allow 0 as a valid rate limit. |
| core/src/main/scala/kafka/server/DynamicBrokerConfig.scala | Allows the consolidation fetch rate limit to be updated dynamically. |
| core/src/main/scala/kafka/server/ConfigHandler.scala | Updates the consolidation quota from the merged (static + dynamic) currentKafkaConfig. |
| core/src/main/java/kafka/server/QuotaFactory.java | Adds a dedicated consolidation ReplicationQuotaManager to QuotaManagers and seeds it from static config. |
| core/src/main/scala/kafka/server/ReplicaManager.scala | Switches consolidation quota usage to the QuotaManagers-owned manager. |
| core/src/test/scala/unit/kafka/server/DynamicConfigChangeTest.scala | Adds tests for dynamic update and “don’t clobber static value on unrelated change”. |
| core/src/test/scala/unit/kafka/server/KafkaApisTest.scala | Updates QuotaManagers construction to match the new parameter list. |
| core/src/test/scala/unit/kafka/server/ControllerApisTest.scala | Updates QuotaManagers construction to match the new parameter list. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jeqo
force-pushed
the
jeqo/ts-consolidation-quota-dynamic
branch
from
June 10, 2026 11:22
de9cd34 to
10b53e5
Compare
Make diskless.consolidation.fetch.rate.limit.bytes.per.second a dynamic broker config, aligned with the leader/follower replication throttle pattern: - Add to AllDynamicConfigs so KAdmin accepts it as a dynamic update - Move ReplicationQuotaManager ownership to QuotaManagers (seeded from static config on startup via QuotaFactory.instantiate) - Apply dynamic updates in BrokerConfigHandler.processConfigChanges, reading from currentKafkaConfig to pick up the merged static+dynamic value (avoids clobbering a static limit on unrelated config changes) - Relax atLeast(1) to atLeast(0) and document as dynamically updatable - Add DynamicConfigChangeUnitTest cases covering dynamic update and static-value-preserved-on-unrelated-change scenarios Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jeqo
force-pushed
the
jeqo/ts-consolidation-quota-dynamic
branch
from
June 10, 2026 11:38
10b53e5 to
7757deb
Compare
jeqo
marked this pull request as ready for review
June 10, 2026 12:10
viktorsomogyi
approved these changes
Jun 10, 2026
jeqo
added a commit
that referenced
this pull request
Jun 10, 2026
Make diskless.consolidation.fetch.rate.limit.bytes.per.second a dynamic broker config, aligned with the leader/follower replication throttle pattern: - Add to AllDynamicConfigs so KAdmin accepts it as a dynamic update - Move ReplicationQuotaManager ownership to QuotaManagers (seeded from static config on startup via QuotaFactory.instantiate) - Apply dynamic updates in BrokerConfigHandler.processConfigChanges, reading from currentKafkaConfig to pick up the merged static+dynamic value (avoids clobbering a static limit on unrelated config changes) - Relax atLeast(1) to atLeast(0) and document as dynamically updatable - Add DynamicConfigChangeUnitTest cases covering dynamic update and static-value-preserved-on-unrelated-change scenarios Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
jeqo
added a commit
that referenced
this pull request
Jun 10, 2026
Make diskless.consolidation.fetch.rate.limit.bytes.per.second a dynamic broker config, aligned with the leader/follower replication throttle pattern: - Add to AllDynamicConfigs so KAdmin accepts it as a dynamic update - Move ReplicationQuotaManager ownership to QuotaManagers (seeded from static config on startup via QuotaFactory.instantiate) - Apply dynamic updates in BrokerConfigHandler.processConfigChanges, reading from currentKafkaConfig to pick up the merged static+dynamic value (avoids clobbering a static limit on unrelated config changes) - Relax atLeast(1) to atLeast(0) and document as dynamically updatable - Add DynamicConfigChangeUnitTest cases covering dynamic update and static-value-preserved-on-unrelated-change scenarios Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make diskless.consolidation.fetch.rate.limit.bytes.per.second a dynamic broker config, aligned with the leader/follower replication throttle pattern: