Tags: minio/zipindex
Tags
Add layered zip indexes (#17) * Add layered zip indexes - Add LayeredIndex[T] for combining multiple zip indexes with generic references - Implement file override semantics: newer layers override older ones - Add delete layers to remove files from previous layers - Automatic empty directory cleanup when files are deleted - Binary search optimization: files sorted by name on layer addition - Find operations use O(log n) binary search instead of O(n) linear search - Concurrent serialization/deserialization with user-provided RefSerializer[T] - Manual msgpack format: version + layers with ref data as binary arrays - Comprehensive test suite with 15+ test scenarios and benchmarks - Iterator support for Go 1.23 range syntax - Complete API: layer management, file access, conversion, serialization - Updated README with usage examples and API documentation * Upgrade Go versions and dependencies. * Upgrade ci-lint
Add chunked encoding of large directories (#16) When warm, it provides a nice speedup: ``` BEFORE: BenchmarkFindSerialized/100-32 84632 14027 ns/op 24.28 b/file 84632 files/s 5062 B/op 4 allocs/op BenchmarkFindSerialized/1000-32 13371 88087 ns/op 23.32 b/file 13371 files/s 49398 B/op 4 allocs/op BenchmarkFindSerialized/10000-32 1491 769408 ns/op 20.66 b/file 1491 files/s 451106 B/op 4 allocs/op BenchmarkFindSerialized/100000-32 163 7350855 ns/op 20.25 b/file 163.0 files/s 4499216 B/op 5 allocs/op BenchmarkFindSerialized/1000000-32 15 175437187 ns/op 20.16 b/file 7.500 files/s 44978624 B/op 6 allocs/op AFTER: BenchmarkFindSerialized/100-32 89673 13144 ns/op 24.28 b/file 89673 files/s 216 B/op 4 allocs/op BenchmarkFindSerialized/1000-32 15356 78862 ns/op 23.32 b/file 15356 files/s 220 B/op 4 allocs/op BenchmarkFindSerialized/10000-32 1663 719424 ns/op 20.66 b/file 1663 files/s 228 B/op 4 allocs/op BenchmarkFindSerialized/100000-32 170 6894286 ns/op 20.25 b/file 170.0 files/s 26760 B/op 4 allocs/op BenchmarkFindSerialized/1000000-32 16 67916169 ns/op 20.16 b/file 16.00 files/s 2812044 B/op 4 allocs/op ``` * Add new chunked format for faster FindSerialized. It kicks in at 25000 dir entries. Ensures that performance will be somewhat linear. Bumps index limit to 1B. ``` BenchmarkFindSerialized/100-32 88186 13926 ns/op 24.28 b/file 88186 files/s 216 B/op 4 allocs/op BenchmarkFindSerialized/1000-32 14157 82875 ns/op 23.32 b/file 14157 files/s 220 B/op 4 allocs/op BenchmarkFindSerialized/10000-32 1562 757547 ns/op 20.66 b/file 1562 files/s 225 B/op 4 allocs/op BenchmarkFindSerialized/100000-32 567 2101872 ns/op 21.66 b/file 567.0 files/s 1164419 B/op 10 allocs/op BenchmarkFindSerialized/1000000-32 583 2035905 ns/op 21.73 b/file 583.0 files/s 1181472 B/op 47 allocs/op ```
Sync upstream changes (#15) Notably includes golang/go#66869 and `RemoveInsecurePaths()` for golang/go#55356