test(inkless:consolidation): add read-from-remote system tests, harness support by viktorsomogyi · Pull Request #654 · aiven/inkless

viktorsomogyi · 2026-06-16T17:59:53Z

Adds system tests proving consolidated diskless topics stay readable after every local copy is destroyed — reads must be served from the remote tier (MinIO) + control plane (Postgres) alone. Covers all three consolidation entry points, plus the KafkaService/ConsolidationVerifier harness support they depend on.

Tests

ReadFromRemoteAfterPruneTest: born-consolidated. Produce, let consolidation tier and prune the WAL, wipe all local logs, restart empty, and assert remote-only reads are lossless (earliest == 0, fetch from 0 returns 0, contiguous content, full read-back).
SwitchedReadFromRemoteAfterPruneTest: classic→diskless (switched). Same wipe-and-read pattern; offset 0 lives in the classic interval under the classic epoch, so serving it from remote requires the rebuild to resolve that epoch. Also asserts a window straddling the seal is contiguous with correct content across the boundary.
ConsolidationEnabledOnUntieredDisklessTest: upgrade transition. Reach an untiered-diskless topic with consolidation OFF, flip the cluster-wide feature on via in-flight restart, then remote.storage.enable=true to start consolidating without a leader change. Asserts the remote tier grows, the WAL prunes above the seal, and reads stay lossless across the seal (nothing is wiped).

Harness

KafkaService.alter_topic_configs for the atomic in-place config transitions the switched and upgrade tests rely on (combined diskless.enable + remote.storage.enable alter that seals and starts consolidation in one step; the remote-only flip that starts fetchers via the config hook).

ConsolidationVerifier extensions: MinIO tiered-object queries, Postgres WAL-batch / log-start queries, JMX consolidation-gauge scraping, remote-read and cross-boundary contiguity checks, and a local-log wipe helper.

Copilot

Pull request overview

Adds Inkless consolidation system tests that validate read-from-remote behavior after WAL pruning and complete local log loss across multiple consolidation entry paths, plus KafkaService harness support for atomic multi-config topic alters required by some transitions.

Changes:

Add born-consolidated durability system test verifying earliest/first-fetch-from-0 after wipe+restart.
Add switched classic→diskless consolidated durability system test covering classic-epoch boundary reads after prune + wipe.
Add upgrade test covering untiered-diskless → consolidation-enabled transition and end-to-end readability.
Extend KafkaService with alter_topic_configs() to apply multiple topic configs in one kafka-configs --add-config invocation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
tests/kafkatest/tests/inkless/consolidation_switched_read_from_remote_after_prune_test.py	New switched-topic durability test; includes offset/read helpers using console-consumer.
tests/kafkatest/tests/inkless/consolidation_read_from_remote_after_prune_test.py	New born-consolidated durability test verifying earliest and fetch-from-0 after wipe+restart.
tests/kafkatest/tests/inkless/consolidation_enable_on_untiered_diskless_test.py	New upgrade test for enabling consolidation on an existing untiered diskless topic.
tests/kafkatest/services/kafka/kafka.py	Adds atomic multi-config topic alter helper (`alter_topic_configs`).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

…ss support Adds system tests proving consolidated diskless topics stay readable after every local copy is destroyed, reads must be served from the remote tier (MinIO) + control plane (Postgres) alone. Covers all three consolidation entry points, plus the KafkaService/ConsolidationVerifier harness support they depend on. - ReadFromRemoteAfterPruneTest: born-consolidated. Produce, let consolidation tier and prune the WAL, wipe all local logs, restart empty, and assert remote-only reads are lossless (earliest == 0, fetch from 0 returns 0, contiguous content, full read-back). - SwitchedReadFromRemoteAfterPruneTest: classic→diskless (switched). Same wipe-and-read pattern; offset 0 lives in the classic interval under the classic epoch, so serving it from remote requires the rebuild to resolve that epoch. Also asserts a window straddling the seal is contiguous with correct content across the boundary. - ConsolidationEnabledOnUntieredDisklessTest: upgrade transition. Reach an untiered-diskless topic with consolidation OFF, flip the cluster-wide feature on via in-flight restart, then remote.storage.enable=true to start consolidating without a leader change. Asserts the remote tier grows, the WAL prunes above the seal, and reads stay lossless across the seal (nothing is wiped). Co-authored-by: Cursor <cursoragent@cursor.com>

…ss support (#654) Adds system tests proving consolidated diskless topics stay readable after every local copy is destroyed, reads must be served from the remote tier (MinIO) + control plane (Postgres) alone. Covers all three consolidation entry points, plus the KafkaService/ConsolidationVerifier harness support they depend on. - ReadFromRemoteAfterPruneTest: born-consolidated. Produce, let consolidation tier and prune the WAL, wipe all local logs, restart empty, and assert remote-only reads are lossless (earliest == 0, fetch from 0 returns 0, contiguous content, full read-back). - SwitchedReadFromRemoteAfterPruneTest: classic→diskless (switched). Same wipe-and-read pattern; offset 0 lives in the classic interval under the classic epoch, so serving it from remote requires the rebuild to resolve that epoch. Also asserts a window straddling the seal is contiguous with correct content across the boundary. - ConsolidationEnabledOnUntieredDisklessTest: upgrade transition. Reach an untiered-diskless topic with consolidation OFF, flip the cluster-wide feature on via in-flight restart, then remote.storage.enable=true to start consolidating without a leader change. Asserts the remote tier grows, the WAL prunes above the seal, and reads stay lossless across the seal (nothing is wiped). Co-authored-by: Cursor <cursoragent@cursor.com>

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch 2 times, most recently from 7bd6055 to 055dd90 Compare June 16, 2026 19:37

viktorsomogyi force-pushed the svv/ts-unification-recover-leader branch from 3437d4c to 986aaa3 Compare June 23, 2026 08:44

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 055dd90 to 6451701 Compare June 23, 2026 08:45

viktorsomogyi force-pushed the svv/ts-unification-recover-leader branch 2 times, most recently from d29c23c to 17d63d7 Compare June 23, 2026 12:17

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 6451701 to 6e205fa Compare June 23, 2026 12:59

viktorsomogyi requested a review from Copilot June 23, 2026 13:00

Copilot started reviewing on behalf of viktorsomogyi June 23, 2026 13:00 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

viktorsomogyi changed the base branch from svv/ts-unification-recover-leader to svv/ts-unification-systest-test1 June 23, 2026 14:27

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 6e205fa to 253d3c7 Compare June 23, 2026 14:28

viktorsomogyi force-pushed the svv/ts-unification-systest-test1 branch from 1395419 to b31f319 Compare June 23, 2026 14:30

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch 3 times, most recently from d78b81f to 643ff3d Compare June 23, 2026 14:44

viktorsomogyi requested a review from Copilot June 23, 2026 14:44

Copilot started reviewing on behalf of viktorsomogyi June 23, 2026 14:45 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

Comment thread tests/kafkatest/tests/inkless/consolidation_read_from_remote_after_prune_test.py Outdated

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 643ff3d to 24eea0d Compare June 23, 2026 14:51

viktorsomogyi requested a review from Copilot June 23, 2026 14:52

Copilot started reviewing on behalf of viktorsomogyi June 23, 2026 14:52 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

Comment thread tests/kafkatest/services/kafka/kafka.py Outdated

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 24eea0d to 380bfa8 Compare June 23, 2026 15:01

viktorsomogyi requested a review from Copilot June 23, 2026 15:02

Copilot started reviewing on behalf of viktorsomogyi June 23, 2026 15:02 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

Comment thread tests/kafkatest/tests/inkless/consolidation_enable_on_untiered_diskless_test.py Outdated

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 380bfa8 to 8a416c4 Compare June 23, 2026 15:27

viktorsomogyi requested a review from Copilot June 23, 2026 15:29

Copilot started reviewing on behalf of viktorsomogyi June 23, 2026 15:30 View session

viktorsomogyi force-pushed the svv/ts-unification-recover-leader branch from c551884 to 89f76e2 Compare June 29, 2026 15:53

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 1353235 to 7ef5195 Compare June 29, 2026 15:58

viktorsomogyi changed the base branch from svv/ts-unification-recover-leader to svv/ts-unification-systest-test1 June 29, 2026 16:00

viktorsomogyi marked this pull request as draft June 29, 2026 16:01

viktorsomogyi force-pushed the svv/ts-unification-systest-test1 branch from 21101ef to 162024b Compare June 30, 2026 09:44

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 7ef5195 to a5d0879 Compare June 30, 2026 09:54

viktorsomogyi requested a review from Copilot June 30, 2026 09:55

Copilot started reviewing on behalf of viktorsomogyi June 30, 2026 09:55 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread tests/kafkatest/services/inkless/consolidation_verifier.py

Comment thread tests/kafkatest/services/inkless/consolidation_verifier.py

Comment thread tests/kafkatest/services/inkless/consolidation_verifier.py

Comment thread tests/kafkatest/services/inkless/consolidation_verifier.py

viktorsomogyi force-pushed the svv/ts-unification-systest-test1 branch from 162024b to 3b59a48 Compare June 30, 2026 11:38

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch 2 times, most recently from 9b7ae16 to 0011cb9 Compare June 30, 2026 14:58

Base automatically changed from svv/ts-unification-systest-test1 to main June 30, 2026 14:59

viktorsomogyi changed the base branch from main to svv/ts-unification-leader-below-seal June 30, 2026 15:38

viktorsomogyi force-pushed the svv/ts-unification-leader-below-seal branch 2 times, most recently from 53c392e to e526d64 Compare July 1, 2026 09:08

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 0011cb9 to 73f50b4 Compare July 1, 2026 13:50

Base automatically changed from svv/ts-unification-leader-below-seal to main July 1, 2026 14:09

viktorsomogyi marked this pull request as ready for review July 1, 2026 14:41

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 73f50b4 to b068188 Compare July 1, 2026 14:43

viktorsomogyi marked this pull request as draft July 1, 2026 15:20

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from b068188 to 3fc45f7 Compare July 2, 2026 09:23

viktorsomogyi marked this pull request as ready for review July 2, 2026 09:24

viktorsomogyi force-pushed the svv/ts-unification-recovery-systest branch from 3fc45f7 to 095b60b Compare July 2, 2026 12:19

jeqo approved these changes Jul 2, 2026

View reviewed changes

jeqo merged commit f617b89 into main Jul 2, 2026
4 checks passed

jeqo deleted the svv/ts-unification-recovery-systest branch July 2, 2026 13:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test(inkless:consolidation): add read-from-remote system tests, harness support#654

test(inkless:consolidation): add read-from-remote system tests, harness support#654
jeqo merged 1 commit into
mainfrom
svv/ts-unification-recovery-systest

viktorsomogyi commented Jun 16, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

viktorsomogyi commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tests

Harness

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

viktorsomogyi commented Jun 16, 2026 •

edited

Loading