fix(transaction): merge manifests during overwrite commits - #234
Merged
chenzl25 merged 4 commits intoSep 11, 2026
Merged
Conversation
chenzl25
requested review from
Li0k,
agaddis02,
mwylde,
nagraham,
netgusto,
silver-ymz and
vovacf201
as code owners
September 8, 2026 13:30
8 tasks
silver-ymz
approved these changes
Sep 9, 2026
Install Hive directly Signed-off-by: Jannik Steinmann <jannik.steinmann@datadoghq.com> (cherry picked from commit 4687d26)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Forward-port the COW overwrite manifest fix in #233 to the dependency line used by RisingWave main. Based on RisingWave's current pin
cd6e1e6acbfb2e41c9af5b6faedaca5eb83f361c.What changes are included in this PR?
Overwrite commits merge data manifests inline, using the existing 8 MiB target and 100-manifest first-bin threshold. Snapshot properties override table properties;
commit.manifest-merge.enabled=falsedisables merging. Group by partition spec, preserve original data/file sequence numbers and this commit's added/deleted entries, and keep delete-content manifests unchanged. Drop old deletion-only manifests before loading them during file replacement; retained snapshots remain readable.This dependency line has no inline merge implementation, so the port uses one private helper on the snapshot producer. It does not add another snapshot or rewrite data files. There are no new unit tests or V3-specific branches.
Are these changes tested?
nightly-2026-03-05toolchain.-p iceberg --lib --tests --locked -- -D warnings), workspace formatting and diff checks pass.AI Disclosure
Implemented and validated with Codex assistance at the maintainer's request.