Release Note 2.0.0

Thanks to our community users and developers, 383 improvements and bug fixes have been made in Doris 2.0.1.

Behavior Changes

• [fix](create table) modify varchar default length 1 to 65533 #21302

Improvements

functionality and stability of array and map datatypes

• [new-feature](complex-type) support read nested parquet and orc file with complex type #22793
• [Improve](complex-type) update for array/map element_at with nested complex type with local tvf #22927
• [Improve](serde) support text serde for nested type-array/map #22738
• [feature](hive)append support for struct and map column type on textfile format of hive table #22347
• [FIX](map)fix column map for offset next_array_item_rowid order #23250
• [vectorized](udf) java udf support with return map type #22300

performance for inverted index query

• [fix](inverted index) check all pushdown for match predicate #22836
• [Enhancement](inverted index) remove time-consuming log prints in bkd reader #23381
• [Optimize](search) Optimize implement the new query interface #23389
• [fix](invert index) supports utf8 and non-utf8 strings #22570

performance for bitmap, like, scan, agg functions

• [improvement](agg) Do not serialize bitmap to string #23172
• [improvement](function) do not use hyperscan for non-const partterns in like function #23495
• [Improvement](scan) Remove redundant predicates on scan node #23476
• [Improvement](pipeline) Terminate early for short-circuit join (#23378) #23396
• [Enhancement](scanner) allocate blocks in scanner_context on demand  #23182
• [Improvement](aggregate) optimization for AggregationMethodKeysFixed::insert_keys_into_columns #22216

functionality and stability of CCR

• [improvement](binlog) Gc BE binlog metas when tablet is dropped #22447
• [Enhancement](binlog) Add Barrier log into BinlogManager #22559
• [Enhancement](binlog) Add binlog enable/disable check && filter in Binlog* #22173
• [Improvement](binlog) Bring the binlog when do the full/incremental engine clone task #22678

merge on write unique table

• [Enhancement](merge-on-write) add correctness check for the calculation of delete bitmap #22282
• [enhancement](merge-on-write) Add more log info when delete bitmap correctness check failed #22984
• [improvement](delete) support delete predicate on value column for merge-on-write unique table #21933
• [Enhancement](partial update) Support including delete sign column in partial update stream load #22874

optimizer table stats and analyze

• [improvement](nereids) Refactor analysis task #22658
• [enhancement](nereids) Execute sync analyze task with multi-thread #22211
• [improvement](stats) No more sync unknown stats #22775
• [improvement](stats) Catch exception properly #22896
• [improvement](external statistics)Fix external stats collection bugs. #22788
• [Improvement](statistics)Improve show analyze performance. #22484 #22882

functionality and performance of multi catalog

• [feature](multi-catalog)support iceberg hadoop catalog external table query #22949
• [Improvement] Optimize count operation for iceberg #22923
• [Improvement] Add iceberg metadata cache and support manifest file content cache #22336
• [Enhancement](tvf) Table value function support reading local file #22915
• [feature](jdbc catalog) support doris jdbc catalog array type #23056
• [improve](jdbc catalog) support sqlserver uniqueidentifier data type #23297
• [improve](jdbc catalog) Improve the speed of writing to postgresql databases #23279

Important Bug fixes

• [Fix](planner)support delete conditions contain non-key columns and add check in analyze phase for delete. #22673
• [fix](time) fix error in time_to_sec #22656
• [Bug](pipeline) fix pipeline jdbc coredump in regression test #22892
• [Fix](inverted index) fix non-index match function core #22959
• [fix](Nereids) type check could not work when root node is table or file sink #22902
• [Bug](exchange) init _instance_to_rpc_ctx on register_sink #22976
• [Fix](Nereids)fix insert into return npe from follower node. #22734
• [Fix](Planner) fix multi phase analysis failed in multi instance environment #22840
• [fix](nereids)bind slot failed because of "default_cluster" #23008
• [fix](Nereids) scan should output all invisiable column #23003
• [fix](nereids)dphyper join reorder use wrong method to get hash and other conjuncts #22966
• [Fix](load) fix partiotion Null pointer exception #22965
• [fix](muti-catalog)convert to s3 path when use aws endpoint #22784
• [Fix](statistics)Fix update cached column stats bug. #23049
• [Bug](scan) fix core dump due to store_path_map #23084
• [Fix](Planner) fix arguments type changed to array type and implicitly cast child error #22947
• [enhancement](file-cache) limit the file cache handle num and init the file cache concurrently #22919
• [fix](Nereids) non-inner join should not merge dist info #22979
• [Fix](orc-reader) Fix filling partition or missing column used incorrect row count. #23096
• [fix](multi-catalog) conversion of compatible numerical types #23113
• [fix](json)Fix the bug that does not stop when reading json files #23062
• [fix](show-table-status) fix priv error on show table status stmt #22918
• [fix](catalog) fix hive partition prune bug on nereids #23026
• [fix](multi-catalog)fix compatibility issue for s3 endpoint when use incompatible client #23175
• [Fix](RoutineLoad)Fix when Unique (MoW) RoutineLoad imports unspecified Sequence column #23167
• [fix](multi-catalog)fix all avro,hudi and jdbc scanner class loader dependencies conflicts #23015
• [Fix](multi-catalog) Fix NPE when file cache is enabled. #23165
• [FIX](array) fix array() with cast in if #23264
• [bugfix](vertical-compaction) Only can init the SegmentCacheHandle once #23246
• [fix](parquet) parquet reader confuses logical/physical/slot id of columns #23198
• [fix])(nereids)Support select catalog.db.table.column from xxx for nereids planner. #23221
• [fix](parquet) A row of complex type may be stored across more pages #23277
• [Fix](Nereids)add nereids load function in read fields of GlobalFunctionMgr and Database #23249
• [fix](planner)convert to double if div decimal overflow #23272
• [fix](Nereids) use Stopwatch as timeout checker #23383
• [fix](common) implement the move assignment operator for Status #23372
• [FIX](array-func) fix array index func with decimal  #23399
• [fix](auth)Disable colauth #23295
• [fix](planner) fix now function param type error #23446
• [fix](catalog)add custom jar #23406
• [fix](Outfile/Nereids) fix that csv_with_names and csv_with_names_and_types file format could not be exported on nereids  #23387
• [opt](Nereids) remove between expression to simplify planner #23421
• [fix](Nereids) infer predicates generate wrong result #23456
• [Fix](multi-catalog) Fix hive incorrect result by disable string dict filter if exprs contain null expr. #23361
• [fix](iceberg) fix iceberg count(*) short circuit read bug #23402
• [Fix](orc-reader) Fix incorrect result if null partition fields in orc file. #23369
• [fix](hive) do not split compress data file and support lz4/snappy block codec #23245
• [fix](planner)only forbid substitute literal expr in function call expr #23532
• [Fix](multi-catalog) Fix broker load reader and hdfs reader issue. #23529
• [fix](view) The parameter positions of timestamp diff function to sql are reversed #23601

See the complete list of improvements and bug fixes on github .

Big Thanks

Thanks all who contribute to this release:

@adonis0147
@airborne12
@amorynan
@AshinGau
@BePPPower
@BiteTheDDDDt
@bobhan1
@ByteYue
@caiconghui
@CalvinKirs
@csun5285
@DarvenDuan
@deadlinefen
@DongLiang-0
@Doris-Extras
@dutyu
@englefly
@freemandealer
@Gabriel39
@GoGoWen
@HappenLee
@hello-stephen
@HHoflittlefish777
@hubgeter
@hust-hhb
@JackDrogon
@jacktengg
@jackwener
@Jibing-Li
@kaijchen
@kaka11chen
@Kikyou1997
@Lchangliang
@LemonLiTree
@liaoxin01
@LiBinfeng-01
@lsy3993
@luozenglin
@morningman
@morrySnow
@mrhhsg
@Mryange
@mymeiyi
@shuke987
@sohardforaname
@starocean999
@TangSiyang2001
@Tanya-W
@ucasfl
@vinlee19
@wangbo
@wsjz
@wuwenchi
@xiaokang
@XieJiann
@xinyiZzz
@yujun777
@Yukang-Lian
@Yulei-Yang
@zclllyybb
@zddr
@zenoyang
@zgxme
@zhangguoqiang666
@zhangstar333
@zhannngchen
@zhiqiang-hhhh
@zxealous
@zy-kkk
@zzzxl1993
@zzzzzzzs