Releases: duckdb/duckdb
v1.4.3 Bugfix Release
This is a bug fix release for various issues discovered after we released 1.4.2.
What's Changed
- implement proper equals for table functions by @Maxxen in #19722
- Fix behavior for HAVING clause without a GROUP BY by @Tishj in #19739
- Remove httpfs patches and bump by @carlopi in #19763
- Correctly use a lock when accessing the EncryptionKeyManager by @Mytherin in #19772
- Bump iceberg to now default available extension by @carlopi in #19764
- Extract all column bindings of json each function by @Tmonster in #19766
- constraint violation bug fix by @artjomPlaunov in #19776
- Avoid binding macro arguments for untyped macros by @lnkuiper in #19779
- It should probably be
${EXTENSION_CONFIG_BUILD}instead ofEXTENSION_CONFIG_BUILDby @hannes in #19694 - Fix #19575: Fix illegal utf8 character by @flashmouse in #19699
- chore: bump vortex to 0.56.0 by @0ax1 in #19815
- PositionalScan: handle also HAVE_MORE_OUTPUT + empty chunk via iteration by @carlopi in #19824
- AttachDatabase: first set whether file is remote, then do the rest by @carlopi in #19826
- Add CacheBehavior::AUTOMATIC to DBInstanceCache that automatically does the right thing™ by @Mytherin in #19841
- Reuse metadata even in the presence of deletes by @ywelsch in #19823
- DuckIndexScanState::TableScanFunc, split into 2 explicit phases by @carlopi in #19838
- Bump httpfs and iceberg by @carlopi in #19859
- Keep cte_root alive while binding materialized CTEs in MERGE INTO children by @Mytherin in #19863
- CI Cleanup (#19840) by @yan-alex in #19857
- Fix #19517: preserve relation name for table-qualified star LIKE expression by @henry8128 in #19887
- avoid underflow/"inf" loop while reporting unittest summary (backport… by @benfleis in #19900
- Fixes incorrect handing of APPROX_QUANTILE TIME by @Damon07 in #19891
- Add v1.4.3 to Storage Version by @maiadegraaf in #19907
- clean up tmp files while building extensions by @c-herrewijn in #19908
- Bump: ducklake, mysql_scanner by @maiadegraaf in #19910
- remove large limit optimization whenever there is a filter by @guillesd in #19911
- free disk space in Upload Extensions job by @c-herrewijn in #19912
- [Compression] Prevent overriding
COMPRESSION_EMPTYwithCOMPRESSION_CONSTANTby @Tishj in #19913 - Make
make tidy-check-diffcompare against base branch, instead of always comparing againstorigin/mainby @Mytherin in #19917 - More testing for appender and attach-detach by @taniabogatsch in #19708
- Script to release a extension manually by @samansmink in #19881
- Assert that result types match with column data collection when fetching data by @pdet in #19750
- No longer auto-enable profiling on DEBUG mode by @maiadegraaf in #19931
- Fix optimizer incorrectly remove ORDER BY clause from aggregates by @tianjq16 in #19925
- Backport revert append fixes by @Mytherin in #19941
- Bump: spatial by @Maxxen in #19943
- Issue #19916: WASM Time Zones by @hawkfish in #19918
- Fix correlated column binding in ConstantBinder by @d-justen in #19945
- [chore] Increase slow threshold by @taniabogatsch in #19965
- [Parquet] Fix prepared copy option parameter by @Tishj in #19966
- [TestConfig] Fix
verify_fetch_rowconfig, reduce duplication inskip_testsby @Tishj in #19967 - remove sha from artifacts by @c-herrewijn in #19957
- Unbound index binding with context by @artjomPlaunov in #19953
- [Fix] Bug in
FetchRowafter update on indexed table withdict_fsstcompression by @taniabogatsch in #19970 - Null assertion on denormalized_table argument by @Dtenwolde in #19947
- [Art][Wal]Unbound index allocations by @artjomPlaunov in #19901
- No sampling over 281TB by @Tmonster in #19978
- Take in consideration if the sniffer used unstrictness while selecting candidates by @pdet in #20005
- [CSV] Avoid throwing unnecessary errors on strict mode by @pdet in #20007
- [Compression] Fix an issue with NULL updates to a column compressed with DICT_FSST by @Tishj in #20009
- Bump: delta, ducklake, iceberg by @maiadegraaf in #20012
- add unity_catalog, update delta by @samansmink in #20019
- Bump spatial by @staticlibs in #20020
- Retag #19821 to v1.4 - Use PLAIN_DICTIONARY for Parquet version 1 by @pdet in #20024
- Fix INSERT OR REPLACE BY NAME with partial columns(#19845) by @henry8128 in #19989
- Bump MySQL scanner by @staticlibs in #20025
- Windows must sample less by @Tmonster in #20018
- Issue #20015: Streaming Window Sequence by @hawkfish in #20027
- bump iceberg by @Tmonster in #20032
- Fix #20014: correctly use numeric_limits::min for NumericLimits::Min so that stats are initialized to -infinity for floating points by @Mytherin in #20039
- Enable windows_arm64 arch for main extensions by @staticlibs in #20004
- Fix mark join decorrelation by @kryonix in #20033
- propagate gate status in Node4::DeleteChild by @artjomPlaunov in #20044
- Bump: httpfs by @samansmink in #20036
- Bump ducklake by @pdet in #20054
- bump azure to v1.4.3 by @benfleis in #20057
- Bump extensions by @samansmink in #20055
- Fix unnecessary dependent join rewrite by @kryonix in #20048
- Version tag for local files for robust external file cache validation by @lnkuiper in #20058
- backport runner cleanup action by @hannes in #20085
Full Changelog: v1.4.2...v1.4.3
v1.4.2 Bugfix Release
This is a bug fix release for various issues discovered after we released 1.4.1. PR #19716 addresses the issues raised in CVE-2025-64429. Users of the DuckDB encryption feature are encouraged to update.
What's Changed
- bump duckdb-azure ref for 1.4.1 by @benfleis in #19275
- Fix regex optimization to remove flags when converting to contains by @mlafeldt in #19290
- feat: Fix length of sort keys by @krlmlr in #19260
- Pass lambda bindings to next binder by @Dtenwolde in #19144
- bump Julia to v1.4.1 by @c-herrewijn in #19306
- Use cross-compilation for static libs on OSX by @hannes in #19304
- Allow multiple read-only attaches to the same database across database instances by @Mytherin in #19319
- hugeint_t fixes by @carlopi in #19318
- Use CMAKE__COMPILER_LAUNCHER by @evertlammerts in #19326
- Add settings field to test config by @Mytherin in #19330
- Keep track of which database managers have which databases attached in the DatabaseFilePathManager by @Mytherin in #19338
- Bugfixes by @lnkuiper in #19329
- skip several ci jobs on prs that only bump extensions by @samansmink in #19249
- [C API] bind_value out of range fix by @taniabogatsch in #19348
- Destroy TaskNotifier prior to calling FinishTask by @Mytherin in #19373
- Remove zip bombs by @Mytherin in #19380
- Avoid calling shared_from_this() and instead use the passed in ClientContext in buffered data by @Mytherin in #19379
- add test tag support [vfs integration tests p1] by @benfleis in #19331
- Throw if non-
VARCHARkey is passed tojson_objectby @lnkuiper in #19365 - BUGFIX: Silent failure to write row groups with large lists by @J-Meyers in #19376
- Fixes for CTE (de)serialization compatibility with older versions by @Mytherin in #19393
- Fix #18139: correctly initialize flush size in MemoryStream, and re-use writer states by @Mytherin in #19398
- Fixup Kalman fiter: bound it's (0, scale_factor), not (0, 1) by @carlopi in #19395
- Fix StringDecompress for hugeint_t values by @ywelsch in #19403
- Fix #19211: make INSERT OR IGNORE correctly handle multiple constraints by @Mytherin in #19409
- Issue #19386: ICU TZDEFAULT by @hawkfish in #19413
- Fix issue in MetadataManager triggered when doing concurrent reads while checkpointing, and rework concurrent attach / detach test by @Mytherin in #19424
- ADBC fix: escape schema, table and column identifiers by @evertlammerts in #19407
- v1.4: Only serialize CTE nodes when MATERIALIZED is specified by @Mytherin in #19420
- Add forwards compatibility tests to CI by @Mytherin in #19432
- Internal #6168: Unsupported Correlated Binds by @hawkfish in #19431
- Don't pull up filters through
DISTINCT ONand makeenumcasts in Parquet safe (bugfixes) by @lnkuiper in #19406 - Get table bindings for all operators under a logical get if the logical get function is an unnest by @Tmonster in #19467
- WASM #1897: UTC Offset Support by @hawkfish in #19464
- Add tests for nested lambda bindings by @Dtenwolde in #19453
- [Compression] Fix issue in ZSTD decompression related to mis-interpreted segment offsets by @Tishj in #19475
- vfs integration tests p2 - update (env) vars and tests by @benfleis in #19428
- CI Fix: clear benchmark cache between runs by @Mytherin in #19508
- Make
DatabaseInstance::log_manageraunique_ptrby @Flogex in #19471 - Support non-standard NULL in Parquet again by @Mytherin in #19523
- Bump: inet by @samansmink in #19526
- Bump multiple extensions by @samansmink in #19522
- add upcoming patch release to internal versions by @samansmink in #19525
- Add test that either 'latest' or 'vX.Y.Z' are supported STORAGE_VERSIONs by @carlopi in #19527
- Moving staging to cf and uploading to install bucket by @hannes in #19539
- Creating separate OSX cli binaries for each arch by @hannes in #19538
- [Dev] Disable the use of
ZSTDif the block_manager is theInMemoryBlockManagerby @Tishj in #19543 - Follow up to staging move by @hannes in #19551
- Bugfixes: Parquet JSON+DELTA_LENGTH_BYTE_ARRAY and sorting iterator by @lnkuiper in #19556
- [ported from main] Fix bug initializing std::vector for column names by @evertlammerts in #19555
- Disable jemalloc on BSD by @lnkuiper in #19560
- Fix race condition between
AppendandScanby @Captain32 in #19571 - Release relevant tests to still be run on all builds by @carlopi in #19559
- fix inconsistent behavior in remote read_file/blob, and prevent union… by @Maxxen in #19531
- [v1.4-andium] Add Profiler output to logger interface by @carlopi in #19572
- Fix edge case in uncompressed validity scan with offset and fix off-by-one in ArrayColumnData::Select by @Maxxen in #19567
- Skip compiling remote optimizer test when TSAN Is enabled by @Mytherin in #19590
- Wal index deletes by @artjomPlaunov in #19477
- add vortex external extension by @samansmink in #19580
- Avoid eagerly resolving the next on-disk pointer in the MetadataReader, as that pointer might not always be valid by @Mytherin in #19588
- [DevEx] Improve error message when FROM clause is omitted by @Tishj in #18995
- bump iceberg by @Tmonster in #19618
- Improve error message around compression type deprecation/availability checks by @Tishj in #19619
- Increase cast-cost of old-style implicit cast to string by @Maxxen in #19621
- Try to prevent overshooting of
FILE_SIZE_BYTESby pre-emptively increasing bytes written in Parquet writer by @lnkuiper in #19622 - Bump: spatial by @Maxxen in #19620
- Detect invalid merge into action and throw exception by @Mytherin in #19636
- Fix #19455: correctly extract root table in merge into when running ajoin that contains single-sided predicates that are transformed into filters by @Mytherin in #19637
- Remove
FlushAllfromDETACHby @lnkuiper in #19644 - Bump MySQL scanner by @staticlibs in #19643
- Enable running all extensions tests as part of the build step by @carlopi in #19631
- Always remember extra_metadata_blocks when checkpointing by @ywelsch in #19639
- duckdb_logs_parsed to do case-insensitive matching by @carlopi in #19669
- Categorize ParseLogMessage as CAN_THROW_RUNTIME_ERROR by @carlopi in #19672
- Fixup linking for LLVM by @carlopi in #19668
- Log total probe matches in hash join by @lnkuiper in #19683
- Fix InsertRelation on attached database by @evertlammerts in #19583
- Add request timing to HTTP log by @samansmink in #19691
- Add missing query location to blob cast by @Mytherin in #19689
- Bump: aws, ducklake, httpfs, iceberg by @samansmink in #19654
- Bump: delta, ducklake, httpfs by @samansmink in #19715
- Fix #19355: correctly resolve subquery in MERGE INTO action condition by @Mytherin in #19720
- Fix #19700: correctly sort output selection vector in nested selection operations by @Mytherin in #19718
- Bump httpfs and resume testing on Wind...
v1.4.1 Bugfix Release
This is a bug fix release for various issues discovered after we released 1.4.0.
What's Changed
- Fix attach to right DB when using DuckLake by @pdet in #19011
- set default value of MAIN_BRANCH_VERSIONING to false by @c-herrewijn in #19014
- ComplexJSON: parse all valid JSON correctly by @Mytherin in #19024
- Issue #19016: ICU Offset Parsing by @hawkfish in #19029
- Throw if we detect a quoted new line with the null padding set in parallel mode by @pdet in #19012
- Bump iceberg & ducklake by @carlopi in #19037
- Build Fix:
unordered_map<enum classis not supported in all compilers, usemap<instead by @Mytherin in #19046 - Disable emitting versioned libraries by default by @Mytherin in #19047
- Re-add aliased settings to duckdb_settings() view, and some fixes for aliased settings by @Mytherin in #19050
- Fix threading issues in metadata manager, and expand concurrent attach / detach fuzz test by @Mytherin in #19054
- Correctly re-align all child column segments of the ColumnData on Deserialize, and add logging to checkpoints by @Mytherin in #19055
- [unittest] Fixes so that '{BASE_TEST_NAME}' can be used within --on-new-connection by @carlopi in #19056
- add a bunch of expected error messages to old macro tests and fix iss… by @lnkuiper in #19042
- Always execute cast and try_cast if they are not invertible by @DinosL in #19010
- Switching core extension upload to dedicated credentials by @hannes in #19061
- Include BeginQuery in latency metric by @taniabogatsch in #19064
- [Dev] Bit of code cleanup in (parquet) ColumnWriter by @Tishj in #19063
- Add config: one_schema_per_test.json by @carlopi in #19059
- Change bucket name for core extensions by @hannes in #19083
- Moved test data into testing dir by @NiclasHaderer in #19102
- Bump httpfs by @carlopi in #19104
- Fix example syntax in
variant_typeof()function by @krlmlr in #18977 - Avoid throwing on unset extension setting by @Mytherin in #19117
- Fix internal issue 5975 by @lnkuiper in #19101
- Properly initialize
StringStatsin Parquet reader by @lnkuiper in #19139 - Remove HTTPFS tests and setup scripts by @Mytherin in #19140
- Validate JSON in Parquet reader by @lnkuiper in #19143
- Fix bug in merge into when condition is in parenthesis by @pdet in #19137
- Allow implicit casts from
JSON[]toJSONagain by @lnkuiper in #19141 - [ci] Change logic for saving caches: Github variable that decides what gets cached by @carlopi in #19150
- Fix handling of quotes in ToString() of search_path in current_setting by @Mytherin in #19162
- Delay throwing
NotImplementedExceptioninExpressionBinderby @lnkuiper in #19153 - Issue #18303: AsOf NLJ Nulls by @hawkfish in #19173
- HTTPUtil: response might be null, perform check by @carlopi in #19179
- Handle malformed schema index in Parquet reader by @Mytherin in #19191
- ATTACH IF NOT EXISTS: avoid looping waiting for DETACH to finish, wait only for an ATTACH operation to finish by @Mytherin in #19193
- Implement duckdb_connection_count table function by @taniabogatsch in #19187
- Disable ALP for non-default block sizes by @taniabogatsch in #19197
- Check for unresolved parameters when binding
CREATE MACRO ... AS TABLEby @lnkuiper in #19196 - https://duckdb-blobs.s3.amazonaws.com -> https://blobs.duckdb.org by @carlopi in #19206
- [chore] Attempt at restoring workflow for MinGW Static libs by @carlopi in #19205
- Simple no default region return 301 response by @Tmonster in #19087
- [Fix] Correctly reset the gate status during ART merging by @taniabogatsch in #19204
- build spatial extension for mingw by @c-herrewijn in #19207
- Fixup templated version of TryGetSecretKeyOrSetting by @carlopi in #19218
- Bump: delta by @samansmink in #19220
- Autoloading helper file system: allow either autoloading or proper errors in more file operations by @carlopi in #19198
- Eargerly destroy sort buffers in Window by @lnkuiper in #19224
- [Transaction] Delete and drop of a table can now happen in the same transaction without error by @Tishj in #18918
- PRAGMA's MissingEntry: Suggest CALL might be an option by @carlopi in #18815
- Bump: aws, ducklake, iceberg by @samansmink in #19228
- Issue 18603 by @Tmonster in #19227
- Bump DuckLake to latest of V1.4 by @pdet in #19237
- Bump mysql and sqlite by @staticlibs in #19240
- Don't write parquet-native
GEOMETRYby default, add option to control GeoParquet version by @Maxxen in #19244 - When executing a relation, generate a query to set if it is not a query relation by @Mytherin in #19234
- add support for writing geoparquet with v2 metadata too by @Maxxen in #19246
- Bump: iceberg by @samansmink in #19250
- Bump: avro, httpfs by @samansmink in #19248
- bump duckdb-azure ref for 1.4.1 by @benfleis in #19275
Full Changelog: v1.4.0...v1.4.1
DuckDB 1.4.0 "Andium"
This release of DuckDB is named "Andium" after Anas Andium, a species of duck that lives in the Andes mountains in South America.
Please also refer to the announcement blog post: https://duckdb.org/2025/09/16/announcing-duckdb-140.html
What's Changed
- Python package devexp improvements by @evertlammerts in #17483
- change exception type to not be an internal exception by @samansmink in #17551
- Remove redundant code path in the ConflictManager by @taniabogatsch in #17562
- Add support for ToSqlString for union types by @wmTJc9IK0Q in #17513
- Update function descriptions and examples by @c-herrewijn in #17132
- Move query profiler's EndQuery after commit/rollback by @taniabogatsch in #17595
- fix extension troubleshooting link by @simon0191 in #17616
- C API tidying by @taniabogatsch in #17623
- bump DuckDB_jll to v1.3.0 by @c-herrewijn in #17677
- Add rowsort in generate_series test #43 by @jeewonhh in #17675
- [C API] Expose duckdb_scalar_function_bind_get_extra_info by @taniabogatsch in #17666
- Enable profiling output for all operator types by @taniabogatsch in #17665
- Output hashes in unittest and fix order by @niykko in #17664
- New Sorting Implementation by @lnkuiper in #17584
- Merge v1.3-ossivalis into main by @Mytherin in #17690
- Issue #17040: FILL Window Function by @hawkfish in #17686
- ClientBufferManager wrapper to access the client context in the buffer manager by @taniabogatsch in #17699
- Revert "set default for MAIN_BRANCH_VERSIONING to false" by @carlopi in #17708
- Sorting followup by @lnkuiper in #17717
- Correctly setting the delim offset by @Damon07 in #17716
- fix linux extension ci by @samansmink in #17720
- Aggregation performance by @lnkuiper in #17718
- Fix windows-2025 build errors by @adsharma in #17726
- [SQLLogicTester] Introduce
reset label <query label>in the tester by @Tishj in #17729 - Adding additional authenticated data for encryption by @ccfelius in #17508
- csv_scanner: correct code comment by @Djfe in #17735
- Deprecate windows-2019 runners by @hannes in #17745
- re-add httpfs apply_patches by @samansmink in #17755
- Rename decorator from test_nulls to null_test_parameters by @Mytherin in #17760
- [CAPI] Expose ErrorData by @taniabogatsch in #17722
- Expose file_size_bytes and footer_size in parquet_file_metadata by @gijshendriksen in #17750
- Pass
ExtensionLoaderwhen loading extensions, change extension entry function by @Maxxen in #17772 - Support glibc 2.28 environments by @James-Gilbert- in #17776
- Mark Upper/LowerComparisonType as const by @JelteF in #17773
- [Indexes] Buffer-managed indexes part 1: segment handles by @taniabogatsch in #17758
- [Julia] api docs improvements by @tqml in #15645
- Ensure we use the same layout in
RadixPartitionedHashTableandGroupedAggregateHashTableby @lnkuiper in #17790 - [Profiling] Propagate the ClientContext into file handle writes by @taniabogatsch in #17754
- Fix propagatesNullValues for case expr by @suibianwanwank in #17796
- Add qualified parameter to Python GetTableNames API by @evertlammerts in #17797
- Merge v1.3 into main by @Mytherin in #17806
- Pushdown pivot filter by @flashmouse in #17801
- Replace string for const data ptr in encryption api by @ccfelius in #17825
- Merge130 by @carlopi in #17833
- fix: escape using_columns on JoinRef::ToString by @akoshchiy in #17839
- Fix ICE with Windows ARM64 by @staticlibs in #17844
- Merge v1.3 into main by @Mytherin in #17851
- Add
duckdb_typecolumn to parquet_schema by @Mytherin in #17852 - Internal #4991: Remove Epoch_MS(MS) by @hawkfish in #17816
- #17853 Enable flexible page sizes and update Android NDK to r27 in workflow. by @aprock in #17854
- [Indexes] Buffer-managed indexes part 2: segment handle for base nodes by @taniabogatsch in #17828
- Function Serialization: adapt to removal of overloads by explicitly casting if argument types have changed by @Mytherin in #17864
- julia: add missing methods from C-API by @tqml in #17733
- Issue #17153: Window Order Columns by @hawkfish in #17835
- Issue #17040: FILL Secondary Sorts by @hawkfish in #17821
- Add STRUCT to MAP cast function by @evertlammerts in #17799
- Issue #17849: Test FILL Duplicates by @hawkfish in #17869
- Add GenAI policy by @szarnyasg in #17882
- Update function descriptions and examples for list, array, lambda functions by @c-herrewijn in #17886
- Issue #17861: FILL Argument Types by @hawkfish in #17888
- Reword GenAI policy by @szarnyasg in #17895
- Use an arena linked list for the physical operator children by @taniabogatsch in #17748
- Make CTE Materialization the Default Instead of Inlining by @kryonix in #17459
- Merge v1.3 into main by @Mytherin in #17897
- Leverage
VectorTypeinColumnDataCollectionby @lnkuiper in #17881 - Fix empty BP block when writing parquet by @platypii in #17929
- fix use after free in adbc on invalid stmt by @ruslandoga in #17927
- Do not dispatch JDBC/ODBC jobs in release CI runs by @staticlibs in #17937
- Block based encryption by @ccfelius in #17275
- Unittester failures summary by @hmeriann in #16833
- Add v1.3-ossivalis to Cross version workflow by @hmeriann in #17906
- [CI Nightly Fix] Skip logging test if not standard block size by @taniabogatsch in #17957
- Visual Studio 17 (2022) fixes by @edouarda in #17948
- [Nested] Add
struct_positionandstruct_containsfunctions by @maiadegraaf in #17819 - Enable building spatial and encodings extensions by @staticlibs in #17960
- [Nested] Optimize structs in
LIST_VALUEby @maiadegraaf in #17169 - Unit Tester Configuration by @Mytherin in #17972
- [nested] Allow fixed-size arrays to be unnested by @maiadegraaf in #17968
- Merge v1.3-ossivalis into main by @Mytherin in #17973
- [CI] Skip some workflows when updating out of tree extensions SHA by @hmeriann in #17949
- Issue #5144: AsOf Join Threshold by @hawkfish in #17979
- [Fix] Reset profiling info before preparing a query by @taniabogatsch in #17940
- Flag to disable database invalidation by @taniabogatsch in #17938
- Issue #5123: make_timestamp_ms by @hawkfish in #17908
- Rework extension loading to go through thread-safe ExtensionManager by @Mytherin in #17994
- Implement consumption and production of Arrow Binary View by @pdet in #17975
- Add support to produce Polars Lazy Dataframes by @pdet in #17947
- c-api to copy vector with selection by @abramk in #17870
- Fix #18007: correctly execute expressions with pivot operator by @Mytherin in #18020
- [Chore] Minor conflict manager refactoring by @taniabogatsch in #18015
- Remove Linux (32 Bit) job by @hmeriann in https://gith...
v1.3.2 Bugfix Release
This is a bug fix release for various issues discovered after we released 1.3.1. There are no new major features, just bug fixes. Database files created by DuckDB versions all the way back to v0.9 can be read by this version.
What's Changed
- bump julia to v1.3.1 by @c-herrewijn in #17966
- [CI] adding DONT_LINK parameter to the test extension configuration for
inetextension by @hmeriann in #17967 - Eviction queue: Sort purged nodes and bulk re-add by @lnkuiper in #17913
- Grab lock before finalizing dynamic filters by @lnkuiper in #17964
- Issue #5144: AsOf Join Threshold by @hawkfish in #17978
- Fix for IsDenseRange check in filter_combiner by @pdet in #17988
- Use SharedLockTable in DataTable::Fetch by @Mytherin in #17983
- On Windows CI use zip from msys2 instead of choco by @staticlibs in #17993
- [FIX] Arrow ArrowBool8 Extension Type Add Validity Type Check by @rustyconover in #18005
- Make test more lenient by @lnkuiper in #18022
- Print internal exception stack traces on failed transaction rollback by @taniabogatsch in #18023
- More fixes around GetDatabases by @taniabogatsch in #18024
- [Fix] Binding error when resolving lambdas with a struct alias by @taniabogatsch in #18014
- Disable constexpr std::mutex on Windows by @staticlibs in #17991
- Use correct expression function after filter pushdown by @Tmonster in #17860
- Implement bulk enqueue for non-concurrent queue by @lnkuiper in #18032
- Bring back libduckdb-src.zip as release artifact by @mlafeldt in #18019
- Issue #18035: Zero Fill TIMESTAMP_NS by @hawkfish in #18045
- Fix handling dynamic table filters in RemoveUnusedColumns by @Damon07 in #18033
- Fix incorrect results in index scan by @taniabogatsch in #18058
- bump spatial (v1.3) by @Maxxen in #18059
- constant or null can be replaced when argument is a bound column reg by @Tmonster in #18018
- Issue #18047: TIMESTAMP_TZ Upcast Costs by @hawkfish in #18064
- Properly handle empty RHS in IE Join by @lnkuiper in #18067
- Avoid going too in-depth while computing join order by @carlopi in #17904
- Main.yml: Move very long job from debug to release with
-DDEBUGandFORCE_ASSERTby @carlopi in #18081 - Run Python workflow against both Python 3.9 and 3.13 on PR to ensure … by @evertlammerts in #18080
- fix statistics propagation for anti-joins on empty tables by @bradynwalsh in #17439
- OSX.yml: Move from using debug builds to release + DDEBUG + FORCE_ASSERT by @carlopi in #18102
- Fix copy constructor in SetVariableStatement by @staticlibs in #18101
- [Parquet] Add write_bloom_filter flag to allow disabling of bloom filters by @Mytherin in #18093
- Add Stack Trace marker to stack trace by @JelteF in #18089
- Add missing
INT128to decimal Parquet reader switch by @lnkuiper in #18104 - shared_ptr& must be reached from FileOpener by @carlopi in #18107
- Cleanup on correct branch (1.3-ossivalis) instead of v1.2-histrionicus by @carlopi in #18111
- Refactor extracting expressions for dynamic index scans by @lnkuiper in #18095
- Add v1.3.2 to version_map.json and generate storage_info.cpp by @hmeriann in #18112
- CI: Actually correctly skip building unnecessary extensions by @carlopi in #18119
- Absorb patch from #18107 by @carlopi in #18114
- Use unsigned hugeint for compressed materialization strings by @lnkuiper in #18128
- Throw internal exception on corrupted roaring bitmap offsets by @taniabogatsch in #18130
- Allow .tsv as an accepted db file by @pdet in #18133
- Fixing CSV Fuzzer issues by @pdet in #18134
- Bump iceberg for 1.3.2, from @Tishj, and bumping also httpfs by @carlopi in #18148
- Internal #5245: AsOf NLJ Comparisons by @hawkfish in #18159
- Add internal exceptions to compression paths to prevent segmentation violations by @taniabogatsch in #18151
- Fix for arrow.json production extension type by @pdet in #18132
- partially restore deprecated http logging settings by @samansmink in #18150
- Bump for wasm fixes (excel and httpfs) and test fixes (iceberg) by @carlopi in #18167
- bump spatial by @Maxxen in #18161
- Bump sqlsmith and aws by @Tmonster in #18155
- bump ducklake for v1.3.2. by @c-herrewijn in #18156
- Bump spatial again: include re-linking for handling global objects in Wasm by @carlopi in #18170
Full Changelog: v1.3.1...v1.3.2
v1.3.1 Bugfix Release
This is a bug fix release for various issues discovered after we released 1.3.0 "Ossivalis". There are no new major features, just bug fixes. Database files created by DuckDB versions all the way back to v0.9 can be read by this version.
What's Changed
- MultiFileReader: Fix for handling nested list/map default values by @Mytherin in #17589
- Signed to Unsigned is not reversible by @Tmonster in #17571
- [Dev][CLI] Use an unused bit for
DUCKDB_LATEST_STORAGE_VERSIONby @Tishj in #17598 - minor restructure MAIN_BRANCH_VERSIONING by @c-herrewijn in #17601
- Main branch versioning set to false by @c-herrewijn in #17602
- Generate correct UUID v7 by @Mytherin in #17612
- Issue #17606: Disable TIMESTAMPTZ Casts by @hawkfish in #17614
- Bugfixes by @lnkuiper in #17543
- Improve Windows lock conflict error by @Mytherin in #17622
- Actually initialize in batch copy to file by @Mytherin in #17627
- Issue #17621: Streaming Window Reset by @hawkfish in #17649
- CLI: Print codename for '-version' by @szarnyasg in #17637
- [Python][Dev] Ignore
DYNAMIC_FILTERTableFilters in filter pushdown by @Tishj in #17657 - [Dev] Throw if
dbis not available yet in setting certain configuration options by @Tishj in #17659 - chore: Fix initialization by @krlmlr in #17643
- chore: Fix initialization by @krlmlr in #17644
- DefaultSecretGenerator: require lock for modifying persistent_secrets by @Mytherin in #17650
- initialize the read with the OpenFile info and not just the path by @Tmonster in #17652
- Don't bail on TopN optimization if we don't have a cardinality by @lnkuiper in #17654
- Fixes for CSV fuzzer tests by @pdet in #17678
- chore: Fix strict aliasing warning on GCC by @krlmlr in #17641
- Partitioned copy: don't check if file exists for remote files by @Mytherin in #17689
- Fix version detection for sdist builds without git info by @evertlammerts in #17605
- Bugfixes by @lnkuiper in #17695
- Do not get file handle unnecessarily. by @Tmonster in #17698
- Allow table functions to disable statement caching by @Mytherin in #17702
- [SQLLogicTester] Replace keywords in
<FILE>:patternresult for thequerystatement by @Tishj in #17710 - Parquet Reader: only read strings as fixed length strings if the type is FIXED_LEN_BYTE_ARRAY by @Mytherin in #17723
- Internal #5022: IN Pushdown Equalities by @hawkfish in #17731
- Internal #4995: Commutative INTERVAL Multiply by @hawkfish in #17730
- Issue #17725: Quantile NaN Compare by @hawkfish in #17761
- Backport CI fixes by @Mytherin in #17763
- Have the skip_rows option consider empty csv lines by @pdet in #17756
- Fix wrong assertion in Parquet DBP encoder by @lnkuiper in #17746
- CLI: make -f always bail on error by @Mytherin in #17768
- Don't Flatten() then Reference() by @hannes in #17769
- [Dev] Fix
TRYexpression crash on literals by @Tishj in #17753 - Pop up ICU errors to the csv sniffer by @pdet in #17719
- [Nested] Fix incorrect type casting in list_reduce lambda expressions by @maiadegraaf in #17581
- [chore] Avoid caching msys artifacts on PRs by @carlopi in #17777
- Skip encodings and spatial extensions on PRs by @carlopi in #17775
- Add
FileBufferType::EXTERNAL_FILEand add to same queue asFileBufferType::BLOCKby @lnkuiper in #17771 - Storge the argument and value of arg_min_max in the state as a unique_ptr by @pdet in #17749
- Physical operator logging by @lnkuiper in #17752
- Take string size into account in
GetRowSizeinParquetWriterby @lnkuiper in #17793 - [CSV Sniffer] Consider if null_padding is set to true when detecting midcomment line during sniffing by @pdet in #17751
- Revert "Avoid early-out when catalog lookup fails - instead finish all look-ups" by @Mytherin in #17805
- Support file rotation with WRITE_EMPTY_FILE false by @Mytherin in #17804
- Check page filtered out flag before reading it by @Damon07 in #17786
- Avoid saving ccache on pull_requests by @carlopi in #17810
- Support glibc 2.28 environments in 1.3.x by @taniabogatsch in #17814
- arrow_output_version option to produce arrow depending on a format version. by @pdet in #17791
- Internal #5069: Win32 Cast Simplification by @hawkfish in #17820
- local_agnostic::isspace to avoid spaces be depending on locale by @carlopi in #17808
- MultiFileReader: Make column mapping mode configurable per-file, instead of requiring it to be set globally by @Mytherin in #17817
- Fixup confict by @carlopi in #17831
- [Fix] Throw serialisation error when encountering invalid row IDs in WAL delete by @taniabogatsch in #17832
remap_struct: Correctly reserve list vectors to deal with remapping larger-than-vector-size lists/maps by @Mytherin in #17836- Fixup
unique_ptr<T[], deleter>, now with working custom deleters by @carlopi in #17840 - Fix ICE with Windows ARM64 (1.3) by @staticlibs in #17846
- [Fix] WAL replay catalog error in AddForeignKeyConstraint by @taniabogatsch in #17830
- Add ducklake sha by @c-herrewijn in #17818
- add ducklake to internal_extensions by @c-herrewijn in #17811
- Use boolean operators instead of bitwise operators for (
u)hugeintby @lnkuiper in #17862 - Bump httpfs by @Tmonster in #17863
- Function Serialization: adapt to removal of overloads by explicitly casting if argument types have changed by @Mytherin in #17867
- Add FinalizeLoad callback to catalogs, which can be called after the database is fully instantiated by @Mytherin in #17868
- Set query location for interval constants in all cases by @Mytherin in #17876
- Add support for
option.scheduler_process_partialto the Executor by @bleskes in #17877 - Add array indices to nested paths in
json_each/json_treeby @lnkuiper in #17878 - Add urllib3 dependency and improve fixture download reliability by @evertlammerts in #17880
- Add option to control parquet NaN pruning by @Maxxen in #17883
- Issue #17781: ASOF Predicate Binding by @hawkfish in #17889
duckdb_base_std::plus compile time test on discontinued functions by @carlopi in #17866- Early-out on catalog errors to prevent GetDatabases by @taniabogatsch in #17865
- Fix mismatch between idx_t (64 bits) and size_t (implementation dependent) by @carlopi in #17898
- Skipping failing on OSX Release part by @hmeriann in #17899
- [Fix] Serialisation error for invalid block ID in index deserialization by @taniabogatsch in #17900
- Update
MemoryTagwhen converting block to persistent by @lnkuiper in #17893 - Add support for deserializing multiple json-serialized statements by @Maxxen in #17902
- Deprecated std::stringstream by @carlopi in #17894
- Bump extensions by @Mytherin in #17905
- http_log.test: solve non-determinism at the test level by @carlopi in #17914
- Fixup #17775, correct boolean logic by @carlopi in #17912
- Make sure distance is always an int when doing version b...
DuckDB 1.3.0 "Ossivalis"
This release of DuckDB is named "Ossivalis" after Bucephala Ossivalis, an ancestor of the Goldeneye duck that lived Millions of years ago.
Please also refer to the announcement blog post: https://duckdb.org/2025/05/21/announcing-duckdb-130
What's Changed
- V1.2 histrionicus by @Mytherin in #16070
- V1.2 histrionicus by @Mytherin in #16072
- unittests: clear test directory after every test by @Mytherin in #16053
- Benchmark runner: catch and log errors + add support for
retry load Nsyntax by @Mytherin in #16054 - Throw an error when unsupported commands are used in concurrentloop by @Mytherin in #16009
- Remove extension definitions to prevent re-compilation of the entire system on commit by @Mytherin in #15955
- Display schema information of currently selected database only by @ashwaniYDV in #15815
- Issue #14366: Average Intervals by @hawkfish in #15864
- Internal #2176: Temporal AVG by @hawkfish in #15661
- discussions #15981: remove confusing comment in "duckdb/tools/shell/shell.cpp" by @komainu8 in #15984
- Fix #15466 Transform LIMIT or OFFSET first based on order specified in prepared statement by @ashwaniYDV in #15484
- Bitpacking mode info by @arjenpdevries in #15623
- Sniff Timestamp_TZ from CSV FIles by @pdet in #15730
- [no-op] Add documentation for filesystem read behavior by @dentiny in #15937
- Accept "Auto" as date/timestamp format by @pdet in #15808
- Parquet Reader Cleanup: Move ColumnReaders to separate files by @Mytherin in #16092
- Parquet Reader: Move decoding logic into separate Decoder classes by @Mytherin in #16100
- BundleStaticLibs to be also triggered by InvokeCI by @carlopi in #16107
- Parquet Reader: Split DeltaLengthByteArray decoder from DeltaByteArray, and read the strings in a streaming manner by @Mytherin in #16105
- Parquet Dictionary reader: set NULL values as the last value in the dictionary by @Mytherin in #16106
- Parquet Reader: Share ResizeableBuffers across decoders, and unify Plain/PlainReference by @Mytherin in #16113
- Using GitHub ARM runners for Linux CLI builds by @hannes in #16119
- Parquet Reader: Implement dedicated Skip method by @Mytherin in #16117
- Use ColumnSegment::FilterSelection and SelectionVector for filtering in Parquet scans by @Mytherin in #16126
- [Dev] Fix output (long lines > 333 characters) getting truncated in shell by @Tishj in #16128
- Adaptive table filter: initialize filter order based on heuristics by @Mytherin in #16127
- Feature #16044: TimeZone Offset Seconds by @hawkfish in #16048
- ATTACH OR REPLACE database to allow swapping of new data. by @xevix in #15355
- [Dev] Remove
upsert_conflict_in_different_chunk.testby @Tishj in #15980 - [Dev] Fix issue related to unpacked columns and the NOT operator by @Tishj in #15534
- [Julia] Add support for named params in prepared statements by @tqml in #15621
- Use Adaptive Filters in the Parquet reader by @Mytherin in #16133
- Parquet reader: push table filters directly into dictionaries by @Mytherin in #16136
- Parquet reader: Plain templates - make CHECKED a template parameter, and use memcpy/bulk skip when reading/skipping without defines by @Mytherin in #16141
- Parquet reader: only set invalid entry in the dictionary when the column has defines by @Mytherin in #16144
- Add uniq_ptr_cast for interpreted benchmark. by @Tmonster in #16151
- Hopefully fixing ci runs by @hannes in #16150
- Removed the last CI job that used the Ubuntu 18 setup by @hannes in #16155
- Parquet Reader: Split
CreateReaderinto two separate stages -ParseSchemaandCreateReaderby @Mytherin in #16161 - Have CSV Parellel tests on CI again by @pdet in #16164
- [Python][Dev] Bump the minimum pybind11 version from
2.6to2.9by @Tishj in #16159 - Add StackTraces to FatalExceptions by @NiclasHaderer in #16158
- Rework invoke by @carlopi in #16108
- Adds pre-optimization hooks for DuckDB by @NiclasHaderer in #16115
- Unify behavior of
range/generate_serieswith PostgreSQL by @kryonix in #15935 - [CI] Avoid Linux CLI jobs to fail-fast by @carlopi in #16173
- Parquet: Add dedicated Select method that can be used to push selection vectors into the read by @Mytherin in #16174
- Unvendor ICU by @m-kuhn in #16176
- Parquet reader: batch check if buffer is available in RLEBpDecoder by @Mytherin in #16185
- Parquet Reader: for DeltaLengthByteArray encoding, directly refer to strings from the block without copying by @Mytherin in #16186
- unified names for duckdb-extensions by @hmeriann in #16179
- Only delete test directory when
--test-temp-diris not specified by @Mytherin in #16192 - Fix #16163: COLUMNS should not treat identifiers as strings by @Mytherin in #16193
- Parquet reader: Avoid applying bloom filters if we are casting columns by @Mytherin in #16194
- Pretty print sniffer values by @pdet in #16182
- V1.2 histrionicus by @Mytherin in #16191
- Bump Julia by @Mytherin in #16199
- Ensure that dependent targets are present after find_package. by @BillyONeal in #16197
- Concurrency groups for R and Wasm by @hmeriann in #16201
- Parquet Writer Cleanup: Move ColumnWriters to separate files by @Mytherin in #16202
- [fix] Use bigobj when building with MSVC by @m-kuhn in #16200
- Improve performance of
UNNEST/UNPIVOTby using selection vectors to unnest multiple lists at once by @Mytherin in #16210 - Add the
TRYexpression by @Tishj in #15939 - [Python][Dev] Replace the default connection when it's closed by @Tishj in #16160
- Use steady clock for profiler by @dentiny in #16198
- Add parallel memset when building hash join table by @hehezhou in #16172
- Avoid unnecessarily projecting the input columns of the UNPIVOT operator in the UNNEST by @Mytherin in #16221
- Left join push down optimization by @Damon07 in #15881
- Do In-Filter pushdown in PyArrow by @pdet in #16224
- Use _win32 with MSVC by @cfis in #16235
- Fix Python 3 executable name on Windows by @cfis in #16236
- Fix -std=c++11 by @cfis in #16237
- Issue #8265: AsOf Nested Loop by @hawkfish in #16218
- Include extension_util.hpp in libduckdb by @mlafeldt in #16255
- Generalize
rowidinto the concept of virtual columns, and makefilenamea virtual column in the Parquet/CSV/JSON readers by @Mytherin in #16248 - Modify histogram test to more fuzzily check boundaries since the test can be inconsistent on different platforms by @Mytherin in #16261
- [Dev] Fix issue in
TRYexpression withdictionary_expressionVector verification by @Tishj in #16262 - [Python Dev] Add the correct variant of the
-std=c++11flag based on the compiler (MSVC or not) by @Tishj in #16267 - Fix extension install mode null by @samansmink in #16268
- A little cleanup. by @JasonPunyon in #16028
- Improve Parquet writer performance by @lnkuiper in #16243
- Merge v1.2-histrionicus into main by @Mytherin in #16284
- Many reclaim storage fixes by @taniab...
v1.2.2 Bugfix Release
This is a bug fix release for various issues discovered after we released 1.2.1. There are no new major features, just bug fixes. Database files created by DuckDB versions all the way back to v0.9.* can be read by this version.
What's Changed
- [Python] Fix deadlock in
from_parquetwithfile_globscaused by not releasing the GIL by @Tishj in #16522 - [Python] Fix some stubs issues by @Tishj in #16523
- [Dev] Fix issues in the benchmark runner related to the serialized
storage_versionby @Tishj in #16533 - Set estimated cardinality to newly created logical operators by @jeewonhh in #16528
- custom_extension_repository to also be the default autoinstall_extension_repository by @carlopi in #16459
- [Python Dev] No longer trigger a DeprecationWarning when using a UDF by @Tishj in #16433
- Fixup problems connected to prep to 1.2.1 by @carlopi in #16578
- Give preference to quote=escape if we can't do better by @pdet in #16584
- Max ART key length by @taniabogatsch in #16588
- Issue #16617: Window Constant Finalize by @hawkfish in #16628
- [Fix] Index scan - Move the table scan state into the local state by @taniabogatsch in #16650
- [Fix] Perform eager constraint checking during physical batch insert by @taniabogatsch in #16651
- [Python] Don't push down
OPTIONAL_FILTERintopyarrowfor the arrow scan. by @Tishj in #16657 - move OrderPreservationRecursive to physical_plan_generator.hpp by @jeewonhh in #16656
- Add libs folder to bundled static libs by @taniabogatsch in #16655
- Avoid UMA by @krlmlr in #16641
- Avoid UMA by @krlmlr in #16643
- Avoid UMA by @krlmlr in #16642
- Several fixes for CSV fuzzer tests by @pdet in #16636
- FSST Fix: Correctly detect the situation where we have 3 bytes remaining by @Mytherin in #16688
- Fix #16627: handle DISTINCT FROM and NOT DISTINCT FROM in zone-map pushdown by @Mytherin in #16691
- Fix #16554: emit blobs as part of .dump by @Mytherin in #16693
- add avro by @samansmink in #16708
- Issue #16652: Implicit Ordered Aggregation by @hawkfish in #16718
- Issue #16649: SelectNth Remainders by @hawkfish in #16705
- [Dev] Fix
EXPORT DATABASEmissing semicolons in producedschema.sqlby @Tishj in #16723 - Always throw the error that happens the earliest in the CSV Reader by @pdet in #16728
- Fix #16662 by @lnkuiper in #16732
- [Test] Add missing test for eager-constraint checking fix by @taniabogatsch in #16745
- Clamp reported memory in duckdb_memory by @Mytherin in #16753
- CLI
-help: Fix default value for -nullvalue by @carlopi in #16755 - Apply RLE fix to v1.2 by @Mytherin in #16756
- [Arrow] Setting schema of the keys to not null for maps and properly write Null columns to arrow by @pdet in #16711
- Fix min/max values in numeric cast exception message by @abramk in #16777
- [ADBC] Add wrapper to connection new, set options at connection init, if any. by @pdet in #16748
- Remove delta from extensions built on a nightly basis (vs 1.2-histrionicus) by @carlopi in #16794
PhysicalTopN: Buffer-allocatedStringHeapby @lnkuiper in #16770- [chore] Add v1.2.2 to storage versions, preparation for upcoming patch release by @carlopi in #16799
- Fix issue when line is empty by @pdet in #16823
- Add extra check in normalize for commit, rollback and abort by @pdet in #16802
- Fix #16783: Fix DistributivityRule by @flashmouse in #16804
- Internal #4492: Ignore Nulls Orrification by @hawkfish in #16837
- Fix #16836: rewrite main column data in case of an update that only modifies the validity by @Mytherin in #16851
- Initialize type by @pdet in #16848
- [CSV Reader] Add check on ever quoted for candidate selection by @pdet in #16868
- Clean up partial deletes when encountering a transaction conflict in a vector by @Mytherin in #16905
- Fix clang-tidy: add missing include by @Mytherin in #16920
- Backport "Adding gzip version of shell for linux/osx install script" by @carlopi in #16917
- Backport "Adding gzip version of shell for linux/osx install script"/2 by @carlopi in #16924
- Skip tests for Python 3.8, already end-of-life by @carlopi in #16923
- Bump to avro version that now support wasm by @carlopi in #16916
- Unwrap DUCKDB_EXTENSION__LINKED_LIBS via CMake by @carlopi in #16914
- Skip manylinux-extensions-x64 job by @carlopi in #16932
- [Fix] Disable recursion for invalidated database error by @taniabogatsch in #16907
- bump excel (on v1.2) by @Maxxen in #16915
- [Fix] Flush local storage before any deletes by @taniabogatsch in #16951
- [Python] Fix PyArrow filter pushdown for NaN by @Tishj in #16952
- Allow overriding the Printer::Print output destination by @bleskes in #16909
- bump out-of-tree extensions to v1.2.2 by @Tmonster in #16913
- Allow SQL prepared statements to be rebound by @NiclasHaderer in #16820
- Fix #16959: binding constraints should not modify the original constraints by @Mytherin in #16971
- Add back manylinux extensions [on v1.2-histrionicus] AND bump extensions by @carlopi in #16976
- Skip TPC-DS Q67 on DOUBLE by @Mytherin in #16987
- Re-enable iceberg by @carlopi in #16992
- Try enabling and bump delta by @Maxxen in #16990
- Backport @szarnyasg's PR 16999 by @carlopi in #17013
Full Changelog: v1.2.1...v1.2.2
v1.2.1 Bugfix Release
This is a bug fix release for various issues discovered after we released 1.2.0. There are no new major features, just bug fixes. Database files created by DuckDB versions all the way back to v0.9.* can be read by this version.
What's Changed
- [Dev] MultiFileReader fix InternalError in CreateFilterMap by @Tishj in #16114
- [bug-fix] Avoid throwing in catch block for failed commits by @Vegetable26 in #15903
- Issue #16098: ValidEnd Parallel Vectorisation by @hawkfish in #16140
- Adding an extension option shouldn't delete a set value and promote unrecognized options by @bleskes in #15919
- Parquet writer: Re-implement GetRowSize for Strings by @Mytherin in #16178
- Fix #16157: correctly get the first row when reading hive partitions from a dictionary vector by @Mytherin in #16180
- Fix #16122: bind default values in a sub-binder to avoid modifying the catalog search path of the current binder by @Mytherin in #16181
- Fix #16134: when a catalog/schema/table has the same name, we prefer to suggest the table name by @Mytherin in #16189
- Ensure MergeCollectionTask has a writer by @ywelsch in #16207
- Backport #16115 by @NiclasHaderer in #16227
- Deleted copy constructor of pending query by @NiclasHaderer in #16242
- Correctly report errors caused by get_database in C extensions by @mlafeldt in #16253
- use random seeds for bernoulli sample when parallel is enabled by @Tmonster in #16223
- Parquet Reader: avoid caching the compressed buffer in the ColumnReader by @Mytherin in #16263
- Fix #16260: correctly handle parameters in getvariable by @Mytherin in #16264
- Avoid calling SetFilterAlwaysTrue multiple times in RowGroup::CheckZonemap by @Mytherin in #16269
- [Fix] Scanning from normal leaf to nested leaf by @taniabogatsch in #16270
- Fix #16231: refer to order by condition in ARRAY(SUBQUERY) by alias instead of by index by @Mytherin in #16272
- Fix #16257 by @lnkuiper in #16275
- AFL Tests for the CSV reader by @pdet in #16280
- Issue #16250: Window Range Performance by @hawkfish in #16276
- Fix #16278: late materialization should not trigger on very large limits, and it should never trigger on limits without offsets when preserve_insertion_order = false by @Mytherin in #16282
- Overflow strings allocations: avoid rounding up memory allocated per overflow string - when reading "small" overflow strings place them directly into the vector instead by @Mytherin in #16283
- Use ordered map to preserve expressions order by @Damon07 in #16111
- [Dev]
register_filesystemstubs, usefsspec.AbstractFileSystem, notstrby @Tishj in #16266 - [Python Dev] Fix crash with empty args for
isin| Fix transformation forisnotinby @Tishj in #16271 - Fix issue related to hang when all candidates are eliminated in refinement by @pdet in #16288
- [Fix] Early-out on CREATE INDEX (IF NOT EXISTS) by @taniabogatsch in #16093
- [Python] Fix the reverse binary expressions in the Expression API by @Tishj in #16300
- Bugfixes by @lnkuiper in #16244
- [Fix] MinGW bundle static libs by @taniabogatsch in #16292
- Fix heap buffer oveflow sampling by @Tmonster in #16279
- Expose STRING_LITERAL in C API by @taniabogatsch in #16293
- Allow accessing profiler query tree under lock by @ywelsch in #16314
- bump extensions by @Maxxen in #16313
- Inline virtual list lambda bind functions by @Maxxen in #16327
- Update shell.cpp to fix #16333 by @teaguesterling in #16335
- Add the suggestion to verify the nullstring as part of the cast error message by @pdet in #16336
- Take
NULLs into account forDELTA_BINARY_PACKEDby @lnkuiper in #16317 - PhysicalTableScan: Adapt to allow async behaviour by @carlopi in #16310
- Allow querying attached catalog from detached catalog by @jeewonhh in #16289
- Reduce minimum expected memory usage in
RadixPartitionedHashTableby @lnkuiper in #16332 - Backport ebb4dcc to v1.2-, adding missing include by @carlopi in #16369
- Excecption load on mismatched ABI: Use '%d' to print ABI type by @carlopi in #16367
- Aggregation: For dictionaries without an id - use the correct threshold to bail-out on using the dictionary by @Mytherin in #16364
- Bump excel to the same version distributed on
corerepository by @carlopi in #16375 - Accept valid dialects with escape set into the refinement phase by @pdet in #16387
- Push the correct casts for values of different types in (X, Y) IN (SELECT X, Y) by @Mytherin in #16392
- Add support for autoload and autoinstall for
uiextension by @carlopi in #16393 - Add twine_upload option to Python.yml to trigger upload by @carlopi in #16410
- Only select options that generate more columns with null_padding, if they at least hold 50% of consistency by @pdet in #16404
- Use checkpoint bind in DuckTableEntry::Copy to avoid re-validating default values (and potentially causing issues during WAL replay) by @Mytherin in #16398
- Rename
DUCKDB_APItoDUCKDB_C_APIforduckdb.hby @Mytherin in #16397 - Issue #16407: Try_Strptime Invalid TimeZone by @hawkfish in #16416
- Internal #4303: Windowed DISTINCT Leaks by @hawkfish in #16417
- Internal #4258: MODE Spooling Stability by @hawkfish in #16415
- Fix PyPi upload also for branches, when twine_upload is provided by @carlopi in #16421
- [Fix] Throw constraint violation for FK constraint checking by @taniabogatsch in #16399
- Add storage and serialization version for v1.2.1 by @Flogex in #16403
- Update flaky return_files.test by @jeewonhh in #16432
- Add python version to duckdb_api by @guenp in #15599
- Do not accept null values in lists for column parameters by @pdet in #16425
- Use seed for system sample when it is provided. by @Tmonster in #16408
- Bump delta to working commit by @carlopi in #16442
- Adding windows code signing using azure by @hannes in #16444
- CSV small code Improvements + initialising boolean variable. by @pdet in #16454
- [Python Dev] Make
pandasnot required in a couple places, check if it's installed in others by @Tishj in #16414 - fix passing a null path to the C API instance cache by @jraymakers in #16474
- Add methods to retrieve current task scheduler busyness by @NiclasHaderer in #16465
- Fixed reading piped JSON by @lnkuiper in #16480
- [Python Dev]
pyproject.tomlshould not useoldest-supported-numpyanymore by @Tishj in #16486 - [tests] Multiple FORMAT in copy, only last one matters by @carlopi in #16493
- Bump
postgres_scannerandftsextensions by @lnkuiper in #16492 - bump sqlsmith extension tag by @Tmonster in #16488
- [BugFix]: Swap join children, not left and right set by @Tmonster in #16487
- [tests] Add allow_unsigned_extensions require by @carlopi in #16499
- Provide callback when tasks are starting / stopping by @ywelsch in #16451
- CodeQuality: ubuntu-20 to ubuntu-22, lock black to version 24 and trick clang_format detection by @carlopi in #16513
- Move from ubuntu-20:04 to ubuntu-22:04, part I by @carlopi in #16510
- [chore] Build Linux releases also on PRs AND ubuntu-20 to 22 by @carlopi in https://github.com/duckdb/duckd...
DuckDB 1.2.0 "Histrionicus"
This release of DuckDB is named "Histrionicus" after the good-looking Harlequin duck (Histrionicus Histrionicus) that inhabits "cold fast moving streams in North America, Greenland, Iceland and eastern Russia".
Please also refer to the announcement blog post: https://duckdb.org/2025/02/05/announcing-duckdb-120
What's Changed
- Optimise division by a constant at runtime for integer division by @JAicewizard in #10348
- Add cross join to Python Relational and PySpark API by @khalidmammadov in #13519
- Fix #13805: throw a more descriptive error message when an on-disk file is referenced using a replacement scan for an unsupported file format by @Mytherin in #13871
- Make sampling accept parameters at the parser/transformer layer by @Mytherin in #13903
- Fix #13867: use 64-bit random numbers to generate random numbers for
random()by @Mytherin in #13920 - Fix #13769: when binding views, always first search in the schema that the view is defined in by @Mytherin in #13921
- Rework table bindings to be components (
catalog,schema,table) instead of flat strings by @Mytherin in #14017 - Add auto-loadable extension settings to duckdb_config_count and duckdb_get_config_flag by @Mytherin in #14021
- Fix #10961 - in the HAVING clause - in case of column name conflicts, bind to aliases instead of to ungrouped columns by @Mytherin in #14023
- Enable filter pushdown through Logical Unnest by @Tmonster in #14008
- Allow duplicate table aliases in the table binder by @Mytherin in #14035
- Unify DESCRIBE [query] and DESCRIBE [table] by @Mytherin in #14039
- Support qualified identifiers in the
EXCLUDEclause by @Mytherin in #14043 - Add
SMALLER_BINARYflag to reduce binary size by @Mytherin in #14057 - Smaller Binary: remove more templates from arg_min_max by @Mytherin in #14071
- Unify entropy and mode aggregates - and skip specialized implementations for entropy with smaller binary by @Mytherin in #14080
- [Python] Add
set_default_connectionto theduckdbmodule by @Tishj in #13442 - Provide workaround for prefetching parquet files with incorrect page offsets by @samansmink in #13697
- Move
core_functionsto a separate extension by @Mytherin in #14149 - PySpark df.drop() to support expressions by @khalidmammadov in #14059
- add some RealNest benchmarks by @hmeriann in #13345
- feed table function into multifilereader initialization by @samansmink in #14112
- [Dev] Fix an issue causing ExecuteTask to do much more work than intended by @Tishj in #14034
- Overhaul Parquet dictionary handling by @hannes in #14194
- [Feature] Allow passing the catalog (database name) to appender by @taniabogatsch in #13692
- Add Taxi Dataset Benchmark by @pdet in #14197
- Feature #3036: Window Spooling by @hawkfish in #14181
- Small C Extension API changes by @samansmink in #13987
- Add HTML and Graphviz support for explain analyze by @abramk in #13942
- Fix #13064: offer more suggestions with same score by @Damon07 in #14048
- New Algorithm to find a new line on parallel execution by @pdet in #14260
- Making client context lock optional for relation binding by @pdet in #14093
- [Feature] Allow passing the catalog during C API appender creation by @taniabogatsch in #14256
- Make test random output ordered by @Damon07 in #14267
- Skip test_window_distinct by @Mytherin in #14309
- Taxi Benchmark by @pdet in #14301
- Switch to shared pointer for multfilelists by @samansmink in #14291
- Push #14298 to feature branch by @flashmouse in #14311
- Implement PullUp Empty Results optimizer by @Tmonster in #13524
- [Export/Import] Use the DependencyManager to (stable) sort the entries before export by @Tishj in #14196
- Partitioning-Aware Aggregation and Partitioning-Aware Infrastructure by @Mytherin in #14329
- Add df.unionByName to PySpark API by @khalidmammadov in #14063
- Or filter pushdown into zone maps by @Tmonster in #14313
- Get the current setting in the database file opener by @Mytherin in #14361
- [Feature + Fix] Support ALTER TABLE tbl ALTER col TYPE USING and fix null handling in struct_insert by @taniabogatsch in #14359
- [C API] Add table_description_create_ext and table_description_get_column_name by @taniabogatsch in #14285
- Move _rtools platform to be equivalent to _mingw by @carlopi in #14368
- Fix for accidental like skip in the CSV Buffer by @pdet in #14380
- Table locks - always grab table locks through the transaction interface by @Mytherin in #14379
- Implementing array_slice and [] for BLOB by @hannes in #14358
- Rework settings handling and implement auto-generation for new ones by @Mytherin in #14383
- Rework settings handling and implement auto-generation for new ones by @chrisiou in #14018
- Arrow list buffer - suggest setting
arrow_large_buffer_sizeto true when regular list buffer size is exceeded by @Mytherin in #14384 - Fix incorrect merge conflict resolution in workflow file by @Mytherin in #14390
- Update Parquet Thrift to latest version by @hannes in #14258
- Reformat list functions by @c-herrewijn in #14372
- Tidy Check to do complete run also on feature by @carlopi in #14394
- [Python] Use an
ArrowQueryResultinFetchArrowTablewhen possible. by @Tishj in #14319 - Make mysql_scanner auto-loadable, and add mysql/postgres secrets by @Mytherin in #14392
- Improvement the speed of table sample systems by @continue-revolution in #12631
- Support defining column names in CTAS by @douenergy in #14327
- Fix pointer indirection in pyrelation.cpp by @carlopi in #14403
- Fix idx_t to int64_t implicit conversion flagged by clang-tidy by @carlopi in #14402
- Storage: make
ROW_GROUP_SIZEconfigurable by @Mytherin in #14406 - [Dev] Update vendored ZSTD to v1.5.6 by @Tishj in #14360
- Top-N: Rework to use heap of sort keys by @Mytherin in #14424
- reformat string functions by @c-herrewijn in #14400
- Prefix Aliases in SQL by @hannes in #14436
- [Dev] Optimize
ValidityMaskwhen reading from aColumnDataCollectionby @Tishj in #14416 - [Dev] Further optimize the CDC ValidityMask deserialization by @Tishj in #14448
- Reformat date and map functions by @c-herrewijn in #14425
- Reformat generic functions by @c-herrewijn in #14423
- Push dynamically generated join filters through
UNION,UNNESTandAGGREGATEby @Mytherin in #14453 - Try auto-casting for mismatching data chunks in the Appender API by @taniabogatsch in #14433
- Implement
DELTA_BINARY_PACKEDcompression in Parquet writer by @lnkuiper in #14257 - Eviction Queue Partitioning by @lnkuiper in #14375
- Implement
map_extract_firstby @lnkuiper in #14175 - RowGroup no longer lives in format namespace by @Mytherin in #14469
- Convert the shell from C to C++ by @Mytherin in #14473
- Fixing an issue with parquet dictionary reading by @hannes in #14438
- Strip down unused/unsupported options from the CLI by @Mytherin in #14478
- [PySpark] Add withColumns, withColumnsRenamed, cos, acos, any_value, approx_count_distin...