feat(inkless:consume): meter control-plane fetch errors separately from data-plane - #726
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves observability for Inkless consumer fetches by distinguishing control-plane findBatches errors (e.g. OFFSET_OUT_OF_RANGE) from data-plane extraction/storage failures in FetchCompleter, making it easier to attribute steady fetch-error rates to the correct error class.
Changes:
- Add a new Inkless fetch metric (
PartitionControlPlaneErrorRate) to count per-partition control-planefindBatcheserrors. - Record the new metric and emit a WARN log including the specific
Errorscode whenfindBatchesreturns a non-NONEerror inFetchCompleter. - Add a unit test asserting the new metric is marked and that data-plane error metrics are not marked for control-plane failures; regenerate Inkless metrics docs.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| storage/inkless/src/test/java/io/aiven/inkless/consume/FetchCompleterTest.java | Adds coverage to ensure control-plane findBatches errors increment the new metric and don’t affect data-plane error metrics. |
| storage/inkless/src/main/java/io/aiven/inkless/consume/InklessFetchMetrics.java | Introduces and wires the PartitionControlPlaneErrorRate meter, plus a public recording method and template registration for docs. |
| storage/inkless/src/main/java/io/aiven/inkless/consume/FetchCompleter.java | Records the new metric and logs the specific control-plane error when findBatches returns non-NONE. |
| docs/inkless/metrics.rst | Regenerates Inkless metrics documentation to include the new metric entry. |
…om data-plane FetchCompleter surfaced a non-NONE error returned by the control-plane findBatches response (e.g. OFFSET_OUT_OF_RANGE, UNKNOWN_TOPIC_OR_PARTITION) with no metric and no log; it reached the broker only as the generic FailedFetchRequestsPerSec, so a steady failed-fetch trickle could not be attributed to a control-plane error class. Add PartitionControlPlaneErrorRate on InklessFetchMetrics, marked at the findBatches-error branch in FetchCompleter, plus a WARN log carrying the specific error code. This is distinct from the existing PartitionStorageErrorRate / PartitionCorruptRecordRate, which classify data-plane failures during record extraction. Regenerates docs/inkless/metrics.rst (also picking up the reader throughput meters added in the previous commit, whose doc had not been regenerated). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jeqo
force-pushed
the
jeqo/cp-fetch-error
branch
from
July 28, 2026 15:49
ab67a9e to
526aa2a
Compare
jeqo
marked this pull request as ready for review
July 28, 2026 16:21
giuseppelillo
approved these changes
Jul 29, 2026
giuseppelillo
pushed a commit
that referenced
this pull request
Jul 29, 2026
…om data-plane (#726) FetchCompleter surfaced a non-NONE error returned by the control-plane findBatches response (e.g. OFFSET_OUT_OF_RANGE, UNKNOWN_TOPIC_OR_PARTITION) with no metric and no log; it reached the broker only as the generic FailedFetchRequestsPerSec, so a steady failed-fetch trickle could not be attributed to a control-plane error class. Add PartitionControlPlaneErrorRate on InklessFetchMetrics, marked at the findBatches-error branch in FetchCompleter, plus a WARN log carrying the specific error code. This is distinct from the existing PartitionStorageErrorRate / PartitionCorruptRecordRate, which classify data-plane failures during record extraction. Regenerates docs/inkless/metrics.rst (also picking up the reader throughput meters added in the previous commit, whose doc had not been regenerated). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
giuseppelillo
pushed a commit
that referenced
this pull request
Jul 30, 2026
…om data-plane (#726) FetchCompleter surfaced a non-NONE error returned by the control-plane findBatches response (e.g. OFFSET_OUT_OF_RANGE, UNKNOWN_TOPIC_OR_PARTITION) with no metric and no log; it reached the broker only as the generic FailedFetchRequestsPerSec, so a steady failed-fetch trickle could not be attributed to a control-plane error class. Add PartitionControlPlaneErrorRate on InklessFetchMetrics, marked at the findBatches-error branch in FetchCompleter, plus a WARN log carrying the specific error code. This is distinct from the existing PartitionStorageErrorRate / PartitionCorruptRecordRate, which classify data-plane failures during record extraction. Regenerates docs/inkless/metrics.rst (also picking up the reader throughput meters added in the previous commit, whose doc had not been regenerated). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
FetchCompleter surfaced a non-NONE error returned by the control-plane findBatches response (e.g. OFFSET_OUT_OF_RANGE, UNKNOWN_TOPIC_OR_PARTITION) with no metric and no log; it reached the broker only as the generic FailedFetchRequestsPerSec, so a steady failed-fetch trickle could not be attributed to a control-plane error class.
Add PartitionControlPlaneErrorRate on InklessFetchMetrics, marked at the findBatches-error branch in FetchCompleter, plus a WARN log carrying the specific error code. This is distinct from the existing PartitionStorageErrorRate / PartitionCorruptRecordRate, which classify data-plane failures during record extraction.
Regenerates docs/inkless/metrics.rst (also picking up the reader throughput meters added in the previous commit, whose doc had not been regenerated).