test(inkless:consolidation): DeleteRecords across tiers e2e test [KC-332] - #712
Open
viktorsomogyi wants to merge 1 commit into
Open
test(inkless:consolidation): DeleteRecords across tiers e2e test [KC-332]#712viktorsomogyi wants to merge 1 commit into
viktorsomogyi wants to merge 1 commit into
Conversation
viktorsomogyi
force-pushed
the
svv/ts-unification-delete-records-e2e
branch
from
July 21, 2026 15:02
474ab37 to
98c8387
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an end-to-end ducktape regression suite validating DeleteRecords semantics for Inkless consolidating topics across classic+remote tiers, including broker-to-broker earliest-offset consistency and post-failover safety.
Changes:
- Add a new e2e test covering cross-tier
DeleteRecordson switched (hybrid) consolidating topics, including leader failover and surviving-prefix readback verification. - Add verifier helpers to support the test: per-broker bootstrap pinning, per-broker earliest queries + convergence waiting,
DeleteRecordswrapper, born-consolidated topic creation, and a “tiered object count stabilized” waiter.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tests/kafkatest/tests/inkless/consolidation_delete_records_across_tiers_test.py | New e2e test suite for cross-tier DeleteRecords correctness on switched and born-consolidated topics, including failover coverage. |
| tests/kafkatest/services/inkless/consolidation_verifier.py | Adds helper APIs used by the new test (delete-records runner, per-broker earliest checks, born-consolidated topic creation, tiered-count stabilization waiter). |
Comment on lines
+266
to
+272
| state = {"last": -1, "stable": 0, "value": 0} | ||
|
|
||
| def check(): | ||
| cur = self.tiered_object_count() | ||
| state["stable"] = state["stable"] + 1 if cur == state["last"] else 0 | ||
| state["last"] = cur | ||
| state["value"] = cur |
| values = set(per_broker.values()) | ||
| agreed = len(values) == 1 and all(v > 0 for v in values) | ||
| cur = next(iter(values)) if agreed else None | ||
| state["stable"] = state["stable"] + 1 if (agreed and cur == state["last"]) else 0 |
viktorsomogyi
force-pushed
the
svv/ts-unification-reclaim-floor
branch
2 times, most recently
from
July 22, 2026 13:48
654af13 to
2336424
Compare
viktorsomogyi
force-pushed
the
svv/ts-unification-delete-records-e2e
branch
from
July 22, 2026 13:52
98c8387 to
cdfdd1c
Compare
viktorsomogyi
force-pushed
the
svv/ts-unification-reclaim-floor
branch
2 times, most recently
from
July 23, 2026 07:59
ea588ff to
66ddbb7
Compare
viktorsomogyi
changed the base branch from
svv/ts-unification-reclaim-floor
to
main
July 23, 2026 08:32
viktorsomogyi
force-pushed
the
svv/ts-unification-delete-records-e2e
branch
2 times, most recently
from
July 23, 2026 10:01
3d4a2a3 to
d52365c
Compare
viktorsomogyi
changed the base branch from
main
to
svv/ts-unification-reclaim-floor
July 23, 2026 10:06
viktorsomogyi
force-pushed
the
svv/ts-unification-reclaim-floor
branch
4 times, most recently
from
July 24, 2026 12:17
f6c0f80 to
e3f351d
Compare
viktorsomogyi
force-pushed
the
svv/ts-unification-delete-records-e2e
branch
from
July 24, 2026 12:17
d52365c to
796731c
Compare
viktorsomogyi
force-pushed
the
svv/ts-unification-reclaim-floor
branch
from
July 24, 2026 13:07
e3f351d to
5079dcd
Compare
viktorsomogyi
force-pushed
the
svv/ts-unification-delete-records-e2e
branch
4 times, most recently
from
July 27, 2026 10:38
9bc2388 to
a25c580
Compare
viktorsomogyi
changed the base branch from
svv/ts-unification-reclaim-floor
to
svv/ts-unification-follower-reads
July 27, 2026 10:39
viktorsomogyi
force-pushed
the
svv/ts-unification-delete-records-e2e
branch
from
July 27, 2026 12:28
a25c580 to
c368943
Compare
viktorsomogyi
force-pushed
the
svv/ts-unification-follower-reads
branch
from
July 28, 2026 08:52
dda940f to
96ef5b2
Compare
viktorsomogyi
force-pushed
the
svv/ts-unification-delete-records-e2e
branch
from
July 28, 2026 08:57
c368943 to
0878b80
Compare
viktorsomogyi
force-pushed
the
svv/ts-unification-follower-reads
branch
3 times, most recently
from
July 28, 2026 12:51
762c659 to
692f42e
Compare
viktorsomogyi
force-pushed
the
svv/ts-unification-follower-reads
branch
from
July 28, 2026 13:46
692f42e to
ed0554d
Compare
…332] Adds consolidation_delete_records_across_tiers_test.py, the end-to-end regression for Problem A and Problem B on a switched consolidating topic: after DeleteRecords the earliest offset advances off 0, is identical on every broker, settles exactly at the delete boundary, and the surviving classic prefix [delete_before, seal) reads back contiguous. Also covers leader-failover metadata-propagation timing and born-consolidated topics. Adds the verifier helpers these need: delete_records, per-broker earliest_on_each_broker / wait_for_consistent_earliest_across_brokers, create_consolidated_topic, _broker_bootstrap, offset_at bootstrap pinning, and wait_for_tiered_count_stable (a settled pre-reclaim tiered-count peak). Co-authored-by: Cursor <cursoragent@cursor.com>
viktorsomogyi
force-pushed
the
svv/ts-unification-delete-records-e2e
branch
from
July 29, 2026 07:56
0878b80 to
3f7ea8d
Compare
viktorsomogyi
marked this pull request as ready for review
July 29, 2026 07:57
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds consolidation_delete_records_across_tiers_test.py, the end-to-end regression for Problem A and Problem B on a switched consolidating topic: after DeleteRecords the earliest offset advances off 0, is identical on every broker, settles exactly at the delete boundary, and the surviving classic prefix [delete_before, seal) reads back contiguous. Also covers leader-failover metadata-propagation timing and born-consolidated topics.
Adds the verifier helpers these need: delete_records, per-broker earliest_on_each_broker / wait_for_consistent_earliest_across_brokers, create_consolidated_topic, _broker_bootstrap, offset_at bootstrap pinning, and wait_for_tiered_count_stable (a settled pre-reclaim tiered-count peak).