[Compression] Prevent overriding COMPRESSION_EMPTY with COMPRESSION_CONSTANT
#19913
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…filter logic doesn't work otherwise
Contributor
Author
|
We should assert that |
Collaborator
I don't think that's necessary - the |
Collaborator
|
Thanks! |
1 similar comment
Collaborator
|
Thanks! |
github-actions bot
pushed a commit
to duckdb/duckdb-r
that referenced
this pull request
Nov 27, 2025
More testing for appender and attach-detach (duckdb/duckdb#19708) Make `make tidy-check-diff` compare against base branch, instead of always comparing against `origin/main` (duckdb/duckdb#19917) [Compression] Prevent overriding `COMPRESSION_EMPTY` with `COMPRESSION_CONSTANT` (duckdb/duckdb#19913) free disk space in Upload Extensions job (duckdb/duckdb#19912)
github-actions bot
added a commit
to duckdb/duckdb-r
that referenced
this pull request
Nov 27, 2025
More testing for appender and attach-detach (duckdb/duckdb#19708) Make `make tidy-check-diff` compare against base branch, instead of always comparing against `origin/main` (duckdb/duckdb#19917) [Compression] Prevent overriding `COMPRESSION_EMPTY` with `COMPRESSION_CONSTANT` (duckdb/duckdb#19913) free disk space in Upload Extensions job (duckdb/duckdb#19912) Co-authored-by: krlmlr <krlmlr@users.noreply.github.com>
Tishj
added a commit
to Tishj/duckdb
that referenced
this pull request
Dec 9, 2025
Mytherin
added a commit
that referenced
this pull request
Dec 9, 2025
This PR fixes duckdblabs/duckdb-internal#6797 #19913 turned out to be a bandaid If the stats say it's constant, then it's actually constant, otherwise it wouldn't get there.
NiclasHaderer
pushed a commit
to NiclasHaderer/duckdb
that referenced
this pull request
Dec 22, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes https://github.com/duckdblabs/duckdb-internal/issues/6656
COMPRESSION_EMPTYwas added by #15591COMPRESSION_EMPTYis used when we detect that the compression algorithm used for the data can also encode the validity of the column.So the validity does not need to be separately compressed.
COMPRESSION_CONSTANTis used when we detect that all values in the segment are the same. This is also used for Validity in the case where everything is null / everything is non-null.So the data does not need to be compressed.
They are semantically similar, but some of the callbacks on the CompressionFunction have different behaviors, for example
filter.This callback is used to implement filter pushdown at the storage level
For a constant-compressed validity segment this is either 0 or all for a null/non-null filter.
For an empty-compressed validity segment this should be a no-op and implemented by the column's compression function.