What's Changed

• chore: switch to dev mode by @Chillax-0v0 in #1324
• refactor(config): remove config "s3.object.tagging" by @Chillax-0v0 in #1326
• revert: revert to 1.0.6-rc1 by @Chillax-0v0 in #1376
• fix(KAFKA-14644): Process should crash after failure in Raft IO thread (#13140) by @SCNieh in #1361
• fix(KAFKA-14693): Kafka node should halt instead of exit (#13227) by @SCNieh in #1378
• fix(KAFKA-14704): Follower should truncate before incrementing high watermark by @SCNieh in #1379
• fix(metadata): fix broken logic on catching up check by @SCNieh in #1385
• fix(KAFKA-14857): Fix some MetadataLoader bugs by @SCNieh in #1390
• fix(kafka): merge OSS Kafka 3.4 latest by @SCNieh in #1391
• build(gradle): fix s3stream repo by @SCNieh in #1409
• chore(version): release 1.0.6-rc2 by @SCNieh in #1415

Full Changelog: 1.0.6-rc1...1.0.6-rc2

What's Changed

• feat(auto_balancer): support dynamic configs for AutoBalancer by @SCNieh in #1076
• fix(issues1078): fix RequestChannel shutdown blocking by @superhx in #1080
• feat(auto_balancer): optimzed distribution goal action score calculation by @SCNieh in #1083
• docs(config): add config usage suggestion by @superhx in #1084
• fix(auto_balancer): avoid broker load affected by excluded topics by @SCNieh in #1086
• fix(maven): correct the maven build system for s3stream by @daniel-y in #1089
• fix(auto_balancer): add missing variable initialization by @SCNieh in #1090
• feat(auto_balancer): record network metrics by throttle priority by @SCNieh in #1093
• feat(MM2): allow customizing MM2 offset-syncs/checkpoints/heartbeats topic easily by @aaron-ai in #1095
• build(issues1033): switch log appender from DailyRollingFileAppender to RollingFileAppender by @warr99 in #1092
• feat(issues1097): increase api call timeout after timeout by @superhx in #1098
• fix(gradle): specify the classifier for netty-tcnative by @Chillax-0v0 in #1103
• fix(s3stream): fail fast if openssl not available by @Chillax-0v0 in #1105
• feat(issues1087): add data block cache by @superhx in #1107
• fix(docker): use specific localstack version; update aws-cli version by @mooc9988 in #1108
• feat(auto_balancer): split and merge swap actions by @SCNieh in #1109
• chore(util): print commit id when calling scripts wich "--version" as it used to be by @Chillax-0v0 in #1110
• fix(bin): support to run in Windows by @Chillax-0v0 in #1112
• feat(issues1087): add stream reader by @superhx in #1114
• feat(auto_balancer): support broker rack in cluster model by @SCNieh in #1115
• fix(s3): ControllerKVClient putKV should return result by @yangl in #1113
• feat(issues1087): add stream readers by @superhx in #1116
• fix(issues1087): event loop reject task after a task fail by @superhx in #1117
• fix(issues1087): fix long running issues by @superhx in #1118
• fix(issues1087): fix object not exist handle by @superhx in #1119
• fix(core): fix race issue in transaction verification by @ShadowySpirits in #1121
• feat(issues1087): add cold down time to reset readahead by @superhx in #1123
• feat(issue1120): support slow broker detect in controller by @SCNieh in #1122
• docs(config): refine metrics related config documentations by @SCNieh in #1125
• feat(issue1120): support reporting broker metrics in AutoBalancerMetricsReporter by @SCNieh in #1124
• feat(issues1087): allow readahead when cache is free by @superhx in #1126
• fix(metrics): fix method name by @SCNieh in #1129
• feat(issues1087): set s3.object.block.size default from 1MB to 512KB by @superhx in #1130
• chore: update README.md by @KaimingWan in #1131
• feat(metrics): use delta value on reporting percentile metrics by @SCNieh in #1132
• feat(issues1087): increase readahead size when cache miss by @superhx in #1133
• fix(issues1127): fix 1.0 to 1.1-rc0 launch fail by @superhx in #1128
• fix(metrics): remove redundant error log by @SCNieh in #1135
• chore(grafana): fix broken panels by @SCNieh in #1136
• fix(issues1142): retry get index when object not exist by @superhx in #1143
• feat(auto_balancer): support client ACL configuration by @SCNieh in #1146
• fix(core): compose future result when writing stream object by @SCNieh in #1147
• feat(issue1144): start next compaction in advance when needed by @SCNieh in #1148
• fix(issues1097): specify cleanup/minor/major stream object compaction… by @superhx in #1149
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1151
• fix(auto_balancer): suppress error log for OutOfOrderSequenceException by @SCNieh in #1154
• Revert "fix(issues1140): fix S3ObjectControlManager NPE" by @superhx in #1157
• feat(s3stream): change the checksum algorithm to crc32c by @ShadowySpirits in #1155
• chore: update readme and add comparision by @KaimingWan in #1141
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1158
• chore: adjust table and cloud supported badge by @KaimingWan in #1162
• feat(s3stream): reduce stream object compaction write amplification by @superhx in #1159
• feat(throttle): subtract extra token from total available token by @SCNieh in #1163
• feat(throttle): add constructor for compatibility by @SCNieh in #1164
• Revert "feat(s3stream): change the checksum algorithm to crc32c (#1155)" by @ShadowySpirits in #1166
• fix(issues1072): fix MultiPartWriterTest unstable test by @warr99 in #1165
• fix(s3stream): precise record get objects metrics by @superhx in #1170
• chore: support one line quick start by @KaimingWan in #1168
• feat(issues1171): improve objects deletion performance by @superhx in #1172
• feat(s3stream): speed up block cache get objects by @superhx in #1167
• feat(auto_balancer): reduce massive small objects copy during scheduling by @SCNieh in #1176
• fix(auto_balancer): add client ACL to load retriever consumer by @SCNieh in #1182
• feat(issue1179): accelerate partition reassignments on load balancing by @SCNieh in #1181
• chore: update readme by @KaimingWan in #1183
• chore: Update README.md for more accurate descriptions by @daniel-y in #1185
• feat(issues1169): optimize partition reassignment logic to avoid NotLeaderOrFollowerException by @superhx in #1180
• feat(s3stream): active evict expired data by @superhx in #1186
• feat(auto_balancer): change log level to warning when failed on sendi… by @SCNieh in #1188
• style(auto_balancer): remove unused import by @SCNieh in #1189
• feat(issue668): disable swap action temporarily by @SCNieh in #1190
• feat(auto_balancer): add missing action logging by @SCNieh in #1194
• chore: Update README.md for explanations for Kafka compatibility (#1191) by @amos201600 in #1193
• chore(readme): remove controller from the comparison table, and mark proxy of pulsar as optional by @daniel-y in #1197
• feat(issue1195): introduce goal group on score calculation for goals on different scale by @SCNieh in #1196
• fix(issue1198): use timeline structure to avoid incorrect s3 object size by @SCNieh in #1199
• feat(s3stream): add tags to stream by @superhx in #1200
• chore(controller): more logs by @Chillax-0v0 in #1201
• feat(test): add e2e tests for s3stream by @ShadowySpirits in #1204
• feat(auto_balancer): optimize resource type for better scalability by @SCNieh in #1206
• chore(cicd): removing site-docs file in GitHub release workflow by @Gezi-lzq in #1207
• feat(issues1169): prevent re-open partition when the broker is shutting down by @superhx in #1210
• feat: update AWS S3 bucket and destination directory in GitHub releas… by @Gezi-lzq in #1211
• feat(#1212): update readme，add alibaba cloud by @amos201600 in #1214
• fix(s3stream): fix stream reader busy loop when metadata replay delay by @superhx in #1215
• feat(issues1174): cleanup deleted topic by controller by @superhx in #1213
• chore(lo...

What's Changed

• chore: switch to dev mode by @Chillax-0v0 in #1290
• fix(issues1222): fix stream advance check by @superhx in #1307
• chore(version): release 1.0.6-rc1 by @superhx in #1310

Full Changelog: 1.0.6-rc0...1.0.6-rc1

What's Changed

• fix(issues1222): fix #openStream resp nextOffset by @superhx in #1226
• chore: switch to dev mode by @Chillax-0v0 in #1230
• fix: compatible with s3-Stream by @warr99 in #1251
• fix(issues1248): fix partition reassignment deadlock by @superhx in #1250
• feat(metadata): fix data race when close and reopen coordinator partition by @ShadowySpirits in #1255
• fix(metadata): ensure that the consumer offset is deleted along with the deletion of the topic by @ShadowySpirits in #1263
• fix(metadata): clean up the consumption offset for the deleted partitions by @ShadowySpirits in #1265
• fix(issues1276): group the batch request by nodeId and apiKey (#1277) by @superhx in #1278
• ci: bump version to 1.0.6-rc0 by @Chillax-0v0 in #1289

Full Changelog: 1.0.5...1.0.6-rc0
S3Stream Changelog: 1.0.5-s3stream...1.0.6-s3stream-rc0

What's Changed

• fix(issues1078): fix RequestChannel shutdown blocking by @superhx in #1079
• build(issues1033): switch log appender from DailyRollingFileAppender to RollingFileAppender #1092 by @warr99 in #1111
• feat(auto_balancer): support client auth config by @SCNieh in #1138
• feat: bump docker version to 1.0.5-rc0 by @superhx in #1139
• feat(s3stream): switch to branch main by @superhx in #1145
• feat: 1.0 use the old block cache impl by @superhx in #1150
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1152
• Revert "fix(issues1140): fix S3ObjectControlManager NPE" by @superhx in #1156
• feat(config): change s3.object.delete.retention.minutes default to 5min by @superhx in #1160
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1161
• feat(issues1171): improve objects deletion performance by @superhx in #1173
• feat(release): 1.0.5-rc1 by @superhx in #1175
• fix(core): fix leader fast ABA reassign cause epoch outdated by @superhx in #1187
• chore(controller): more logs by @Chillax-0v0 in #1202
• feat(MM2): allow customizing MM2 offset-syncs/checkpoints/heartbeats topic easily by @aaron-ai in #1221
• ci: bump version to 1.0.5 by @Chillax-0v0 in #1223

Full Changelog: 1.0.4...1.0.5
S3Stream Changelog: 1.0.4-s3stream...1.0.5-s3stream

What's Changed

• feat(s3stream): switch to branch main by @superhx in #1145
• feat: 1.0 use the old block cache impl by @superhx in #1150
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1152
• Revert "fix(issues1140): fix S3ObjectControlManager NPE" by @superhx in #1156
• feat(config): change s3.object.delete.retention.minutes default to 5min by @superhx in #1160
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1161
• feat(issues1171): improve objects deletion performance by @superhx in #1173
• feat(release): 1.0.5-rc1 by @superhx in #1175

Full Changelog: 1.0.5-rc0...1.0.5-rc1
S3Stream Changelog: 1.0.4-s3stream...1.0.5-s3stream-rc0

What's Changed

• fix(issues1078): fix RequestChannel shutdown blocking by @superhx in #1079
• build(issues1033): switch log appender from DailyRollingFileAppender to RollingFileAppender #1092 by @warr99 in #1111
• feat(auto_balancer): support client auth config by @SCNieh in #1138
• feat: bump docker version to 1.0.5-rc0 by @superhx in #1139

Full Changelog: 1.0.4...1.0.5-rc0

Highlights

• Upgrade Apache Kafka from version 3.4.0 to 3.7.0

What's Changed

• chore: move s3stream module back to automq kafka repo by @daniel-y in #929
• feat(others): migrate other files by @superhx in #930
• fix(README): add README image by @superhx in #931
• feat(s3stream): add s3stream into gradle build system by @ShadowySpirits in #932
• fix(core): fix optimize condition check by @SCNieh in #934
• fix(core): fix configuration for test cluster by @ShadowySpirits in #936
• feat(s3stream): policies of allocating memory by @Chillax-0v0 in #938
• fix(core): fix ElasticReplicaManagerTest by @ShadowySpirits in #940
• chore(s3stream): remove allocator policy "UNPOOLED_HEAP" by @Chillax-0v0 in #942
• chore: optimize README file by @KaimingWan in #945
• feat(metadata): add automq metadata to metadata shell by @superhx in #947
• chore(workflow): ignore tags like "1.0.2-s3stream" by @Chillax-0v0 in #955
• perf(ReplicaManager): release unused permits after reads by @Chillax-0v0 in #935
• feat: tagging S3 objects by @Chillax-0v0 in #956
• feat(config): auto memory adaptation by @Chillax-0v0 in #949
• fix(s3stream): set tagging only when not null by @Chillax-0v0 in #957
• fix(controller): fix partition migration fail after broker fence by @superhx in #958
• fix(core): fix adjusting s3 cache related configuration by @ShadowySpirits in #959
• fix(controller): fix reassignment fail by @superhx in #960
• fix(core): fix ElasticLog#formatStreamKey by @ShadowySpirits in #961
• fix(core): fix repeatedly opening the partition by @ShadowySpirits in #962
• fix(core): implement parallelism in the creating log process by @ShadowySpirits in #963
• feat: add hook point by @superhx in #964
• refactor(config): adjust configs in KafkaConfig rather than Kafka by @Chillax-0v0 in #970
• fix(bin): fix logs variable by @SCNieh in #972
• feat(metrics): support kraft related metrics by @SCNieh in #974
• fix(tools): controllerOnlyMode is false no extra commands will be output by @funky-eyes in #967
• feat(tool): next node id by @superhx in #976
• fix(test): fix new group coordinator load by @superhx in #977
• fix(test): disable E2E zookeeper related test by @superhx in #980
• fix(test): update E2E base image by @superhx in #981
• fix(common): get cause NPE by @superhx in #984
• fix(network): adapt the callback queue by @superhx in #985
• chore(E2E): split main E2E test to 3 tasks by @superhx in #986
• fix(issues987): fix StreamMetadataManager NPE by @superhx in #988
• fix(metadata): avoid getObjects return invalid when endOffset is -1 by @SCNieh in #991
• fix(s3stream): fix potential concurrent modification on size during tryRealFree by @SCNieh in #990
• fix(s3stream/wal): fail fast if the path does not exist in recovery mode by @Chillax-0v0 in #993
• fix(test): increase heap size to 1GiB by @superhx in #994
• fix(issues996): fix fetch NPE by @superhx in #997
• chore(test): move transactions_test to seperate test suite by @superhx in #998
• fix(s3stream): fix cf combination by @SCNieh in #1000
• refactor(s3stream): rename tagging key from "clusterId" to "automq:ClusterID" by @Chillax-0v0 in #999
• refactor(s3stream): rename tagging key from "automq:ClusterID" to "s3stream:namespace" by @Chillax-0v0 in #1001
• test(s3stream/wal): check file existence on startup rather than init by @Chillax-0v0 in #1004
• chore(workflow): run s3stream tests by @Chillax-0v0 in #1003
• feat(core): optimize block cache performance for high-spec node by @SCNieh in #1005
• fix(s3stream): fix pending fetches modification order by @SCNieh in #1008
• fix(s3stream): temporarily disable throttle for slow read by @SCNieh in #1009
• perf(s3stream): reuse ObjectMapper by @lifepuzzlefun in #1012
• chore(core): upgrade "netty-tcnative-boringssl-static" from 2.0.48 to… by @Chillax-0v0 in #1010
• revert(s3stream): revert bypass slow read throttle by @SCNieh in #1015
• fix: order the transaction requests in request pipeline to adapt KIP-890 by @superhx in #1016
• feat(auto_balancer): use nw bandwidth capacity as score limit by @SCNieh in #1018
• feat(s3stream): provide extra tokens to prevent throttle starvation by @SCNieh in #1020
• fix(test): disable useless E2E test case by @superhx in #1019
• feat(auto_balancer): log expected dist bound on each detection by @SCNieh in #1022
• chore(all): checkstyle by @superhx in #1026
• chore(all): double check the merge by @superhx in #1029
• feat: merge apache kafka trunk by @superhx in #1030
• fix(test): E2E add consumer rebalance protocols by @superhx in #1032
• perf(s3stream): only read the first record from log cache before reading block cache by @Chillax-0v0 in #1011
• fix(s3stream): use thread safe data structures to prevent race condition by @SCNieh in #1036
• fix(core): enhancement AnomalyDetector and LoadRetriever by @Wzy19930507 in #1035
• revert(s3stream): "perf(s3stream): only read the first record from log cache before reading block cache" by @Chillax-0v0 in #1038
• chore(action): trigger build for any changes by @superhx in #1040
• feat(s3stream): add S3 API compatibility for BOS DeleteObjects by @lifepuzzlefun in #1028
• fix(metrics): fix jmx rules by @SCNieh in #1041
• fix(stream): fix compaction interval check by @superhx in #1046
• fix(dashboard): present latency avg to timed range by @SCNieh in #1045
• fix(jmx_rule): remove unused rules that could lead to confusion by @SCNieh in #1048
• chore(s3stream/wal): upgrade metrics level of wal_write and wal_await by @Chillax-0v0 in #1047
• fix(s3stream): limit the expired objects delete count by @superhx in #1050
• fix(config): add elasticstream.replication.factor for compability by @superhx in #1051
• fix(issues1060): only allow one cleanup task infight by @superhx in #1062
• fix(controller): check continuity after redundancy in commitStreamSetObject by @Chillax-0v0 in #1056
• fix(test): disable new consumer coordinator unstable consumer protoco… by @superhx in #1066
• test(controller): commit the same stream set object or stream object by @Chillax-0v0 in #1064
• fix(core): ensure partition closed before metadata listener close by @superhx in #1068
• fix(issues1069): stream client shutdown await opening stream close by @superhx in #1071
• fix(test): ignore unstable new consumer coordinator e2e test by @superhx in #1073
• chore(workflow): ignore tags like "1.0.3-s3stream" by @Chillax-0v0 in #1043
• feat(release): bump version to 1.1.0-rc0 by @superhx in #1077

New Contributors

• @lifepuzzlefun made their first contribution in #1012
• @Wzy19930507 made their first contribution in #1035

Full Changelog: 1.0.4...1.1.0-rc0

What's Changed

• ci: bump version to 1.0.3 by @Chillax-0v0 in #1042
• chore: switch to dev mode by @Chillax-0v0 in #1044
• fix(jmx_rule): remove unused rules that could lead to confusion by @SCNieh in #1049
• ci: bump version to 1.0.4-rc0 by @Chillax-0v0 in #1052
• chore: switch to dev mode by @Chillax-0v0 in #1053
• chore(release): tag in the main branch manually rather than the PR branch by @Chillax-0v0 in #1054
• fix(issues1058): clear offline mark before reopen by @superhx in #1059
• fix(issues1060): only allow one cleanup task infight by @superhx in #1061
• fix(controller): check continuity after redundancy in commitStreamSetObject by @Chillax-0v0 in #1057
• test(controller): commit the same stream set object or stream object by @Chillax-0v0 in #1065
• fix(core): ensure partition closed before metadata listener close by @superhx in #1067
• ci: bump version to 1.0.4 by @superhx in #1075

Full Changelog: 1.0.3...1.0.4
S3Stream Changelog: 1.0.3-s3stream...1.0.4-s3stream

What's Changed

• ci: bump version to 1.0.3 by @Chillax-0v0 in #1042
• chore: switch to dev mode by @Chillax-0v0 in #1044
• fix(jmx_rule): remove unused rules that could lead to confusion by @SCNieh in #1049
• ci: bump version to 1.0.4-rc0 by @Chillax-0v0 in #1052

Full Changelog: 1.0.3...1.0.4-rc0
S3Stream Changelog:1.0.3-s3stream...1.0.4-s3stream-rc0