Improve Parquet writer performance#16243

Merged

Mytherin merged 21 commits intoduckdb:mainfrom

lnkuiper:parquet_stuff

Feb 18, 2025

Commits on Feb 12, 2025

improve performance of hashing longer strings
Laurens Kuiper
committed
implement bit-packing for RleBpEncoder and allow for larger dictionaries
Laurens Kuiper
committed
init primitive dictionary
Laurens Kuiper
committed

Commits on Feb 13, 2025

integrate PrimitiveDictionary into Parquet writer and improve writing levels
Laurens Kuiper
committed
make string_dictionary_page_size_limit for Parquet writer configurable
Laurens Kuiper
committed
some more fast paths/optimizations for parquet writer
Laurens Kuiper
committed
add another fast path
Laurens Kuiper
committed

Commits on Feb 14, 2025

use cast operator for src/target types in primitive dictionary, and add fast path to TemplatedWritePlain
Laurens Kuiper
committed
take vectors larger than standard into account
Laurens Kuiper
committed
take parent nulls into account in fast path writing define levels
Laurens Kuiper
committed
prefer allocator over unique array
Laurens Kuiper
committed
cast and directly write to memorystream in primitive dictionary so we can deal with uuid/interval
Laurens Kuiper
committed
all parquet tests working again
Laurens Kuiper
committed
slightly tweak hash function
Laurens Kuiper
committed
add clickbench write benchmark
Laurens Kuiper
committed
codequality fixes and buffer-manage parquet columndatacollections
Laurens Kuiper
committed

Commits on Feb 17, 2025

format/test fixes for parquet writer
Laurens Kuiper
committed
Merge branch 'main' into parquet_stuff
Laurens Kuiper
committed
change result order now that string hash has changed
Laurens Kuiper
committed
improve performance of boolean column writer too
Laurens Kuiper
committed
even faster boolean writing
Laurens Kuiper
committed