Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Block splitter #4136

Merged
merged 44 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
a5bce4a
XP: add a pre-splitter
Cyan4973 Sep 3, 2024
9e52789
fixed strict C90 semantic
Cyan4973 Sep 3, 2024
586ca96
do not use `new` as variable name
Cyan4973 Sep 3, 2024
e2d7d08
use ZSTD_memset()
Cyan4973 Sep 3, 2024
6021b66
minor C++-ism
Cyan4973 Sep 3, 2024
fa147cb
more ZSTD_memset() to apply
Cyan4973 Sep 3, 2024
83a3402
fix overlap write scenario in presence of incompressible data
Cyan4973 Oct 17, 2024
f83ed08
fixed RLE detection test
Cyan4973 Oct 17, 2024
8b3887f
fixed kernel build
Cyan4973 Oct 17, 2024
dd38c67
fixed single-library build
Cyan4973 Oct 17, 2024
0d4b520
only split full blocks
Cyan4973 Oct 17, 2024
20c3d17
fix assert
Cyan4973 Oct 17, 2024
6dc5212
fixed c90 comment style
Cyan4973 Oct 17, 2024
80a912d
fixed zstreamtest
Cyan4973 Oct 17, 2024
6939235
fixed meson build
Cyan4973 Oct 17, 2024
cdddcaa
new Makefile target mesonbuild
Cyan4973 Oct 17, 2024
76ad1d6
fixed VS2010 solution
Cyan4973 Oct 17, 2024
31d48e9
fixing minor formatting issue in 32-bit mode with logs enabled
Cyan4973 Oct 17, 2024
7f015c2
replaced uasan32 test by asan32 test
Cyan4973 Oct 18, 2024
73a6653
ZSTD_splitBlock_4k() uses externally provided workspace
Cyan4973 Oct 18, 2024
433f459
fixed minor conversion warnings on Visual
Cyan4973 Oct 18, 2024
4685eaf
fix alignment test
Cyan4973 Oct 18, 2024
cae8d13
splitter workspace is now provided by ZSTD_CCtx*
Cyan4973 Oct 18, 2024
4ce91cb
fixed workspace alignment on non 64-bit systems
Cyan4973 Oct 18, 2024
dac26ea
updated regression test results
Cyan4973 Oct 21, 2024
1c62e71
minor split optimization
Cyan4973 Oct 21, 2024
a167571
added a faster block splitter variant
Cyan4973 Oct 21, 2024
7bad787
made ZSTD_isPower2() an inline function
Cyan4973 Oct 22, 2024
5ae34e4
ensure `lastBlock` is correctly determined
Cyan4973 Oct 22, 2024
ea85dc7
conservatively estimate over-splitting in presence of incompressible …
Cyan4973 Oct 22, 2024
4662f6e
renamed: FingerPrint => Fingerprint
Cyan4973 Oct 22, 2024
1ec5f9f
changed loop exit condition so that there is no need to assert() with…
Cyan4973 Oct 22, 2024
16450d0
rewrite penalty update
Cyan4973 Oct 22, 2024
06b7cfa
rewrote ZSTD_cwksp_initialAllocStart() to be easier to read
Cyan4973 Oct 22, 2024
0be334d
fixes static state allocation check
Cyan4973 Oct 22, 2024
d2eeed5
updated compression results
Cyan4973 Oct 22, 2024
18b1e67
fixed extraneous return
Cyan4973 Oct 22, 2024
57239c4
fixed minor strict pedantic C90 issue
Cyan4973 Oct 23, 2024
b68ddce
rewrite fingerprint storage to no longer need 64-bit members
Cyan4973 Oct 23, 2024
7d3e5e3
split all full 128 KB blocks
Cyan4973 Oct 23, 2024
c80645a
stricter limits to ensure expansion factor with blind-split strategy
Cyan4973 Oct 23, 2024
bbda1ac
update regression results
Cyan4973 Oct 23, 2024
90095f0
apply limit conditions for all splitting strategies
Cyan4973 Oct 24, 2024
70c77d2
update regression results
Cyan4973 Oct 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
made ZSTD_isPower2() an inline function
  • Loading branch information
Cyan4973 committed Oct 23, 2024
commit 7bad787d8bcae57cf0bdcbca00b3f106ae557b75
6 changes: 5 additions & 1 deletion lib/common/compiler.h
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,11 @@
* Alignment check
*****************************************************************/

#define ZSTD_IS_POWER_2(a) (((a) & ((a)-1)) == 0)
/* @return 1 if @u is a 2^n value, 0 otherwise
* useful to check a value is valid for alignment restrictions */
MEM_STATIC int ZSTD_isPower2(size_t u) {
return (u & (u-1)) == 0;
}

/* this test was initially positioned in mem.h,
* but this file is removed (or replaced) for linux kernel
Expand Down
6 changes: 3 additions & 3 deletions lib/compress/zstd_cwksp.h
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ MEM_STATIC void ZSTD_cwksp_assert_internal_consistency(ZSTD_cwksp* ws) {
*/
MEM_STATIC size_t ZSTD_cwksp_align(size_t size, size_t align) {
size_t const mask = align - 1;
assert(ZSTD_IS_POWER_2(align));
assert(ZSTD_isPower2(align));
return (size + mask) & ~mask;
}

Expand Down Expand Up @@ -266,7 +266,7 @@ MEM_STATIC size_t ZSTD_cwksp_slack_space_required(void) {
MEM_STATIC size_t ZSTD_cwksp_bytes_to_align_ptr(void* ptr, const size_t alignBytes) {
size_t const alignBytesMask = alignBytes - 1;
size_t const bytes = (alignBytes - ((size_t)ptr & (alignBytesMask))) & alignBytesMask;
assert(ZSTD_IS_POWER_2(alignBytes));
assert(ZSTD_isPower2(alignBytes));
assert(bytes < alignBytes);
return bytes;
}
Expand Down Expand Up @@ -536,7 +536,7 @@ MEM_STATIC void* ZSTD_cwksp_reserve_object_aligned(ZSTD_cwksp* ws, size_t byteSi
void* const start = ZSTD_cwksp_reserve_object(ws, byteSize + surplus);
if (start == NULL) return NULL;
if (surplus == 0) return start;
assert(ZSTD_IS_POWER_2(alignment));
assert(ZSTD_isPower2(alignment));
return (void*)(((size_t)start + surplus) & ~mask);
}

Expand Down