Releases: zlib-ng/zlib-ng
2.2.2
This release fixes a corruption bug with the inflateback implementation, this was
detected by Docker using pigz w/zlib-ng to decompress a 25GB image file and failing CRC.
Since this is so far the only known way to trigger the bug, it seems to be hard to hit.
Most of the rest are minor changes to avoid triggering warnings in MSVC or optional
warnings in other compilers, as well as a few minor fixes to the buildsystem and CI.
Changes
Important Fixes
- Don't use chunkunroll for inflateBack #1773
Buildsystem
- Enable warning C4242 and treat warnings as errors for Visual C++. #1768
- Fixed false positive HAVE_ARMV6_INTRIN value on old ARM platforms #1774
CI/Test
- Upgrade MacOS ARM64 UBSAN to use gcc-13. #1763
Misc
- Fix compiler warnings #1762 #1764 #1765
- Fix new Windows SDK build break #1771
- Prepare to make use of unaligned loads on big endian in insert_string #1695
S390x
- IBM zSystems: Hardcode HWCAP_S390_VXRS #1766
RISC-V
- Better run-time detection of RVV vector instruction support #1770
2.1.8
This backport release fixes a corruption bug with the inflateback implementation, this was
detected by Docker using pigz w/zlib-ng to decompress a 25GB image file and failing CRC.
Since this is so far the only known way to trigger the bug, it seems to be hard to hit.
Most of the rest are minor changes to avoid triggering warnings in MSVC or optional
warnings in other compilers, as well as a few minor fixes to the buildsystem and CI.
Other changes
Important Fixes
- Don't use chunkunroll for inflateBack #1773
Buildsystem
- Enable warning C4242 and treat warnings as errors for Visual C++. #1768
- Fixed false positive HAVE_ARMV6_INTRIN value on old ARM platforms #1774
- don't use zlib-ng's -Wl,--version-script in tests #1750
CI/Test
- Upgrade MacOS ARM64 UBSAN to use gcc-13. #1763
Misc
S390x
- IBM zSystems: Hardcode HWCAP_S390_VXRS #1766
RISC-V
- Better run-time detection of RVV vector instruction support #1770
2.2.1
This is the first stable release of the 2.2.x branch
Please read the changelog for the 2.2.0 Release Candidate if you didn't already, especially if your software gives zlib-ng a custom allocator.
No bugreports came in during 2.2.0 RC testing, so the only change in 2.2.1 is a small fix for Configure that was already in the pipe:
- Configure: Don't use zlib-ng's -Wl,--version-script in tests #1750
2.2.0 Release Candidate
This release contains several larger changes and optimizations. On x86-64 for example, this leads to a compression speedup of ~12% on default level.
We also have a major reorganization of memory alloc/free to always happen during init, this allows applications to potentially do the init early and be finished with the malloc system calls before it needs to process latency sensitive compression/decompression. It also ensures that zlib-ng can not fail due to memory pressure after having run the init functions successfully. We also now only do a single memory allocation deflate or inflate, ensuring we do less system calls and the allocated buffers live close together in memory.
Compression or decompression of very small buffers will now also be faster due to spending less time doing malloc/free.
The downside to this is that decompression will now always allocate the maximum required memory (~42KB total on 64-bit platforms), previously it would allocate (and potentially free) memory as needed during decompression.
It also means that applications that replace the alloc/free functions with their own can potentially have some issues (Yes I am looking at you Nginx).
Changes
Buildsystem
- Generate CMake package configuration files #1647
- Relocate CMake target export definitions #1657
- Allow overwrite NATIVEFLAG value. #1662 #1684
- Fix xsave intrinsic test for clang, and gcc 8.2 or newer, and icc #1664
- Disable Intel Compiler diagnostic message 10441 #1666
- Add missing checks for 64bit arm/intel with msvc compiler #1667
- Don't export git/github-related files in tar/zip archives #1688
- Cleanup and update NMake Makefiles #1673
- Add more result variables to the cmake package configuration #1671
- Fix building with NVHPC #1698
- CMake: Replace ; by $ in generator-expression #1707
- Bump max CMake policy version to 3.29.0 #1709
- make darwin cross compilation possible #1714
CI/Test
- Improve code coverage handling #1640 #1642 #1675 #1729
- Add VPCLMULQDQ crc32 tests to Google benchmarks #1651
- Add small compress() benchmark #1721 #1730
- Add back-and-forth inflateCopy() test #1731
- Enable orphaned unit tests for compare256_rle family of functions #1739
- Fix MSAN error in test_dict #1726
- CI workflows
- Add dependabot for github actions #1687
- Upgrade ilammy/msvc-dev-cmd to v1.13.0 #1665
- Upgrade codecov/codecov-action to v4. #1676
- Upgrade github/codeql-action from 2 to 3 #1691
- Upgrade actions/upload-artifact from 3 to 4 #1692
- Upgrade mymindstorm/setup-emsdk to v14. #1677
- Update dependencies for 32-bit MinGW CI run #1711
- Use windows-2019 for build with toolset v141 #1725
- Fix macOS Github Actions #1720
Cleanup
Refactoring and Optimizations
- Move C fallback functions into arch/generic [Part 1] #1630 #1631 #1658 #1668
- Remove unneeded pointer for functable.longest_match in deflate_slow #1633
- Improve x86 intrinsics dependency handling #1643
- Split cpu_features.h by architecture #1644
- Speed up crc32_[v]pclmulqdq on small strings. #1650
- Cleanup of bi_flush() #1660
- Split cpu features and arch functions #1685 #1696
- Inline CHUNKCOPY and CHUNKUNROLL #1669
- Remove update_hash and insert_string implementations from functable #1681
- Disable dynamic function dispatching for native or arch-specific builds #1659 #1701
- Clean up insert_match() in deflate_medium #1682
- Prefer HAVE_ALIGNED_ALLOC when available in zng_alloc #1635
- Rewrite deflate memory allocation #1713 #1736
ARM
- Add test for checking if -march=native needs -mfpu=neon for 32-bit ARM. #1683
- Override Clang x4 NEON intrinsics for Android #1694
- Add AArch64 feature detection support for OpenBSD #1732
- Improved Configure ACLE check #1727
Power
- Fix regression in Power8/9 detection #1649
RVV
- Optimized rvv slide_hash #1704
- arch/riscv/riscv_features.c: fix uclibc build #1700
- Disable CodeCov for RISC-V as the toolchain doesn't support generating code coverage #1679
S390x
- Update s390x dockerfile #1716
- IBM zSystems DFLTCC: Extend sanitizer checks #1717
- IBM zSystems DFLTCC: Inline DLFTCC states into zlib states #1718
- Remove unused function dfltcc_alloc_state #1728
x86
- Fix PCLMULQDQ, AVX512VNNI and VPCLMULQDQ feature tests for Intel LLVM compiler (icx) #1672
- Fix invalid instruction usage in Xeon Phi x200 processors #1723
Misc
2.1.7
Due to the high amount of refactoring changes in develop, I have decided to target those to a new version branch 2.2.x.
There is also a lot of fixes and minor improvements, so those will be backported and released as 2.1.7.
To work around the #1708 issue (incompatibility with applications mis-using the zlib zalloc/zfree API), #1710 is merged instead of backporting the much bigger #1713.
Backported Changes
Buildsystem
- Generate CMake package configuration files #1647
- Relocate CMake target export definitions #1657
- Fix xsave intrinsic test for clang, and gcc 8.2 or newer, and icc #1664
- Disable Intel Compiler diagnostic message 10441 #1666
- Add missing checks for 64bit arm/intel with msvc compiler #1667
- Don't export git/github-related files in tar/zip archives #1688
- Add more result variables to the cmake package configuration #1671
- Fix building with NVHPC #1698
- CMake: Replace ; by $ in generator-expression #1707
- Bump max CMake policy version to 3.29.0 #1709
- make darwin cross compilation possible #1714
CI/Test
- Improve code coverage handling #1640 #1642 #1675 #1729
- Add VPCLMULQDQ crc32 tests to Google benchmarks #1651
- Add small compress() benchmark #1721 #1730
- Add back-and-forth inflateCopy() test #1731
- Enable orphaned unit tests for compare256_rle family of functions #1739
- Fix MSAN error in test_dict #1726
- CI workflows
- Add dependabot for github actions #1687
- Upgrade ilammy/msvc-dev-cmd to v1.13.0 #1665
- Upgrade codecov/codecov-action to v4. #1676
- Upgrade github/codeql-action from 2 to 3 #1691
- Upgrade actions/upload-artifact from 3 to 4 #1692
- Upgrade mymindstorm/setup-emsdk to v14. #1677
- Update dependencies for 32-bit MinGW CI run #1711
- Use windows-2019 for build with toolset v141 #1725
- Fix macOS Github Actions #1720
Cleanup
Refactoring and Optimizations
- Remove unneeded pointer for functable.longest_match in deflate_slow #1633
ARM
- Add test for checking if -march=native needs -mfpu=neon for 32-bit ARM. #1683
- Override Clang x4 NEON intrinsics for Android #1694
- Add AArch64 feature detection support for OpenBSD #1732
Power
- Fix regression in Power8/9 detection #1649
RVV
- arch/riscv/riscv_features.c: fix uclibc build #1700
- Disable CodeCov for RISC-V as the toolchain doesn't support generating code coverage #1679
S390x
x86
- Fix PCLMULQDQ, AVX512VNNI and VPCLMULQDQ feature tests for Intel LLVM compiler (icx) #1672
- Fix invalid instruction usage in Xeon Phi x200 processors #1723
Misc
2.1.6
This is a stable release, with several minor improvements and one corruption fix for inflateCopy().
This release also improves on the functable implementation, and also moves its initialization to happen in deflateInit() and inflateInit(). We also have some optimizations for RVV and ARM.
Notes for packagers:
- FAR macro has been added back to zlib-compat mode in this release, please remember to remove downstream patches that add it.
- Please consider removing CMake
INSTALL_LIB_DIRworkarounds, this should not be needed since v2.0.2 (2021), but packagers seem to keep copying the workaround from each other. Please see cmake/detect-install-dirs.cmake.
Changes
- Fix inflateCopy corruption caused by change in 2.1.4 #1628
- This is a regression caused by a change introduced in 2.1.4
-
Functable
-
API
- Add FAR macro to zlib-compat headers to improve compatibility #1637
-
ARM
-
RVV
- Optimize adler32_fold_copy using RVV #1597
-
x86
- Simplify AVX2 and AVX512 adler32_fold_copy by removing templates #1599
-
Buildsystem
- Don't attempt ARMv6 detection on AARCH64 #1617
- Prevent tests writing into source directory #1604
- CMake: Fix clang-cl warnings #1591
- CMake: Export cmake target #1601 #1611
- CMake: Remove duplicate enable tests option #1610
- CMake: Fix reading version information from zlib.h.in #1614
- CMake: Check whether compiler supports -march=native or -mcpu=native #1618
- CMake: Always run compiler feature tests without LTO #1622
- CMake: Make sure uqsub16 check doesn't get optimized away with LTO #1619
- CMake: Update to GoogleTest 1.12.1 #1623
- CI: Add linter workflow for whitespace errors #1625 #1632
- CI: Cancel outdated running CI jobs for PR or branch #1629
- CI: Added CI instance for WITH_NATIVE_INSTRUCTIONS #1634
- Tests: Fix buffer overflow in compare256_rle benchmark #1612
-
Misc
- Update copyright to sync with zlib 1.3 #1615
2.1.5
This is a hotfix release, fixing an issue where certain applications would fail with a checksum error during inflate (decompression).
A few minor fixes and improvements are also included.
2.1.4
This is a stable release, with several minor improvements and one fix for a possible buffer overrun while using inflateCopy().
Zlib-ng's zlib-compat mode is now targeting zlib 1.3 compatibility.
Of note, we have new optimizations for ARM and Risc-V RVV, and a lot of fixes and improvements to the buildsystem.
-
Fix: inflateCopy() allocate window with padding #1583
-
Pull zlib 1.3 changes #1563
-
API
- Deprecate ZLIBNG_VER_STATUS, use ZLIBNG_VER_STATUSH #1581
-
MacOS
- Relocatable pkg config files, @rpath/ install name on macOS #1546
-
MinGW
-
ARM
-
Loongarch
- Initial loongarch port #1537
-
PowerPC
- Fix building benchmarks on 32-bit PowerPC #1588
-
RVV
-
x86
-
Buildsystem
- Improve intrinsics detection: ensure intrinsics are not optimized out #1562
- CMake: Fix cross-compiling benchmarks and libpng #1589
- CMake: Fix examining value of GENERATOR_IS_MULTI_CONFIG #1575
- CMake: Fix Match CMAKE_GENERATOR_TOOLSET #1577
- CMake: Cleanup handling of march=native #1544 #1578
- CMake: Add CPack capability #1556 #1579
- Configure: Remove march=native support #1555
- Configure: use dash not bash #1561
- Configure: Fix disabling deflate_quick and deflate_medium #1545
- Configure: Fix distclean #1530
-
Misc
Thanks to all the contributors this release looks to be the best and most stable one so far. 🎉
2.1.3
This is a stable release, with several minor improvements and one fix for a possible endless loop during inflate.
The endless loop bug was detected using unpigz, and is likely a rare corner case that was exposed by pigz threading.
We also have optimizations for the upcoming Risc-V RVV instruction set.
Changes since 2.1.2:
2.1.2
This is the first stable release of the 2.1.x branch.
The changes since beta2 are minor, no changes to code, only buildsystem and tests.
Changes since 2.1.0-Beta2:
- Stop using
COMMAND_ECHOin ctests, it is not supported in older than CMake 3.15. - Add MIPS/MIPS64 CI tests.
- Fix
make distcleancommand with configure/Makefile. - Fix using configure/Makefile on architecture without a directory under
arch.
Full release notes for the first 2.1 stable release:
This release contains two years of development and improvements to zlib-ng, as well as fixes and changes inherited from zlib.
The 2.1.x version series has new targeted minumum buildsystem versions, as detailed on the Wiki https://github.com/zlib-ng/zlib-ng/wiki
Buildsystem:
- Many improvements to the CMake scripts.
- Improved support for detecting memory alignment functions.
- Improved support for unaligned access by letting the compiler promote code to unaligned if supported by the CPU.
- Remove x86 cpu feature detection for TZCNT, safely fallback to BSF.
- Enable using AVX512 intrinsics with GCC <9.
Optimizations and Enhancements:
- Decompression is a lot faster (56% faster measured on AVX2-capable x86-64)
- Compresson is improved for Level 9, at the cost of a little performance.
- Compression is improved for Level 3, by switching from deflate_fast to deflate_medium.
- Levels 3 and 4 have been reconfigured to provide a better gradual tradeoff for speed/compression between levels 2 and 5.
- Deflate_quick (Level 1) has been improved to default to a bigger windowsize and support changing the window size like the other levels.
- Deflate_rle has been optimized with its own compare_256 implementation.
New instruction set optimizations:
- Adler32 implementation using AVX512, AVX512-VNNI, VMX.
- CRC32-B implementation using VPCLMULQDQ & IBM-Z.
- Slide hash implementation using VMX.
- Compare256 implementations using SSE2, Neon, & POWER9.
- Inflate chunk copying using SSSE3 & VSX.
Compatibility and Porting:
- CRC-32 computation changes from madler/zlib. zlib-ng/zlib-ng#a6155234
- Compatible and up-to-date with zlib 1.2.13.
- Removed the usage of macros in zlib-ng.h, making life easier for languages that want to call the C functions without having the C preprocessor (Python, etc).
Improved support more environments:
- Apple M1
- vcpkg
- Emscripten
Testing:
- Tests have been converted to use GTest. Many new tests have also been added.
- Gbench support has been added to easily benchmark changes to performance-critical functions.
- Many new tests added
Misc:
- Several pieces of core code has been restructured or rewritten.
- Too many changes to list here, see the git commit log for the full list of changes.
Deprecations:
- Configure no longer has the full range of tests.
- NMake is no longer actively supported and tested, it is now community supported.
- See the wiki for minimum build system versions and deprecations https://github.com/zlib-ng/zlib-ng/wiki