Skip to content

Move verification out into separate script to fix ARM64 use#294

Open
ell1e wants to merge 1 commit into
richgel999:masterfrom
ell1e:amalgamate_fix_arm64
Open

Move verification out into separate script to fix ARM64 use#294
ell1e wants to merge 1 commit into
richgel999:masterfrom
ell1e:amalgamate_fix_arm64

Conversation

@ell1e

@ell1e ell1e commented Nov 25, 2023

Copy link
Copy Markdown
Contributor

Some platforms don't have the clang sanitizer checks available that are used in the verification tests, currently on these the output files can't even be amalgamated. To fix this, separate amalgamation and verification so that the later can be skipped. Fixes #273

@ell1e ell1e mentioned this pull request Nov 25, 2023
Open
@ell1e ell1e force-pushed the amalgamate_fix_arm64 branch from c090d7a to 68ad084 Compare November 25, 2023 14:04
@ell1e
Move verification out into separate script to fix ARM64 use
b8f5bb2 
Some platforms don't have the clang sanitizer checks available
that are used in the verification tests, currently on these
the output files can't even be amalgamated. To fix this, separate
amalgamation and verification so that the later can be skipped.
@ell1e ell1e force-pushed the amalgamate_fix_arm64 branch from 68ad084 to b8f5bb2 Compare November 25, 2023 14:12
@ell1e

ell1e commented Nov 26, 2023

Copy link
Copy Markdown
Contributor Author

If anyone knows why the fuzzing workflow suddenly can't find miniz_export.h anymore lemme know, I thought I hadn't moved anything around that could possibly break that (this file should still be generated by test.sh after my pull request is applied) so I don't understand what's going on there. Or is the workflow just generally broken? I can't quite make sense of it.

I would really love for this whole ARM64 musl-libc situation to get fixed soon, because it's just quite cumbersome to work around and I'm starting to shift a lot of my projects to more heavily support various ARM64 variants.

@ell1e ell1e mentioned this pull request Nov 26, 2023
Open
@ell1e

ell1e commented Jun 4, 2024

Copy link
Copy Markdown
Contributor Author

If a77af68 was the reason why CIFuzz / Fuzzing failed then maybe this is still good to go in. Sadly I don't know how to trigger the build again to test if it works now. Nevertheless, the underlying problem with platforms without libsanitize support seems to remain.

@ell1e

ell1e commented Feb 9, 2025

Copy link
Copy Markdown
Contributor Author

I just tested the latest commit and the basic task of creating the amalgamated header file continues to be broken on ARM64 if clang happens to be installed.

@ell1e

ell1e commented Oct 29, 2025

Copy link
Copy Markdown
Contributor Author

This is still broken for 174573d:

$ ./amalgamate.sh 
-- The C compiler identification is GNU 14.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Zip files: miniz.c;miniz.h;examples/example1.c;examples/example2.c;examples/example3.c;examples/example4.c;examples/example5.c;examples/example6.c;ChangeLog.md;readme.md;LICENSE
-- The CXX compiler identification is GNU 14.2.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done (3.5s)
-- Generating done (0.2s)
-- Build files have been written to: /home/ellie/Develop/miniz/_build
Test compile with GCC...
_build/amalgamation/miniz.c:3203:9: note: '#pragma message: Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.'
 3203 | #pragma message("Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.")
      |         ^~~~~~~
In file included from _build/amalgamation/miniz.c:1:
_build/amalgamation/miniz.h:584:24: warning: 'zError' defined but not used [-Wunused-function]
  584 |     static const char* zError(int err)
      |                        ^~~~~~
_build/amalgamation/miniz.h:576:21: warning: 'adler32' defined but not used [-Wunused-function]
  576 |     static mz_ulong adler32(mz_ulong adler, const unsigned char *ptr, size_t buf_len)
      |                     ^~~~~~~
_build/amalgamation/miniz.h:571:21: warning: 'crc32' defined but not used [-Wunused-function]
  571 |     static mz_ulong crc32(mz_ulong crc, const unsigned char *ptr, size_t buf_len)
      |                     ^~~~~
_build/amalgamation/miniz.h:565:16: warning: 'uncompress2' defined but not used [-Wunused-function]
  565 |     static int uncompress2(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong* pSource_len)
      |                ^~~~~~~~~~~
_build/amalgamation/miniz.h:560:16: warning: 'uncompress' defined but not used [-Wunused-function]
  560 |     static int uncompress(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong source_len)
      |                ^~~~~~~~~~
_build/amalgamation/miniz.h:555:16: warning: 'inflateEnd' defined but not used [-Wunused-function]
  555 |     static int inflateEnd(mz_streamp pStream)
      |                ^~~~~~~~~~
_build/amalgamation/miniz.h:550:16: warning: 'inflate' defined but not used [-Wunused-function]
  550 |     static int inflate(mz_streamp pStream, int flush)
      |                ^~~~~~~
_build/amalgamation/miniz.h:545:16: warning: 'inflateReset' defined but not used [-Wunused-function]
  545 |     static int inflateReset(mz_streamp pStream)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:540:16: warning: 'inflateInit2' defined but not used [-Wunused-function]
  540 |     static int inflateInit2(mz_streamp pStream, int window_bits)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:535:16: warning: 'inflateInit' defined but not used [-Wunused-function]
  535 |     static int inflateInit(mz_streamp pStream)
      |                ^~~~~~~~~~~
_build/amalgamation/miniz.h:527:21: warning: 'compressBound' defined but not used [-Wunused-function]
  527 |     static mz_ulong compressBound(mz_ulong source_len)
      |                     ^~~~~~~~~~~~~
_build/amalgamation/miniz.h:523:16: warning: 'compress2' defined but not used [-Wunused-function]
  523 |     static int compress2(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len, int level)
      |                ^~~~~~~~~
_build/amalgamation/miniz.h:519:16: warning: 'compress' defined but not used [-Wunused-function]
  519 |     static int compress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len)
      |                ^~~~~~~~
_build/amalgamation/miniz.h:515:21: warning: 'deflateBound' defined but not used [-Wunused-function]
  515 |     static mz_ulong deflateBound(mz_streamp pStream, mz_ulong source_len)
      |                     ^~~~~~~~~~~~
_build/amalgamation/miniz.h:511:16: warning: 'deflateEnd' defined but not used [-Wunused-function]
  511 |     static int deflateEnd(mz_streamp pStream)
      |                ^~~~~~~~~~
_build/amalgamation/miniz.h:507:16: warning: 'deflate' defined but not used [-Wunused-function]
  507 |     static int deflate(mz_streamp pStream, int flush)
      |                ^~~~~~~
_build/amalgamation/miniz.h:503:16: warning: 'deflateReset' defined but not used [-Wunused-function]
  503 |     static int deflateReset(mz_streamp pStream)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:499:16: warning: 'deflateInit2' defined but not used [-Wunused-function]
  499 |     static int deflateInit2(mz_streamp pStream, int level, int method, int window_bits, int mem_level, int strategy)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:495:16: warning: 'deflateInit' defined but not used [-Wunused-function]
  495 |     static int deflateInit(mz_streamp pStream, int level)
      |                ^~~~~~~~~~~
Test compile with GCC ANSI...
_build/amalgamation/miniz.c:3203:9: note: '#pragma message: Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.'
 3203 | #pragma message("Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.")
      |         ^~~~~~~
In file included from _build/amalgamation/miniz.c:1:
_build/amalgamation/miniz.h:584:24: warning: 'zError' defined but not used [-Wunused-function]
  584 |     static const char* zError(int err)
      |                        ^~~~~~
_build/amalgamation/miniz.h:576:21: warning: 'adler32' defined but not used [-Wunused-function]
  576 |     static mz_ulong adler32(mz_ulong adler, const unsigned char *ptr, size_t buf_len)
      |                     ^~~~~~~
_build/amalgamation/miniz.h:571:21: warning: 'crc32' defined but not used [-Wunused-function]
  571 |     static mz_ulong crc32(mz_ulong crc, const unsigned char *ptr, size_t buf_len)
      |                     ^~~~~
_build/amalgamation/miniz.h:565:16: warning: 'uncompress2' defined but not used [-Wunused-function]
  565 |     static int uncompress2(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong* pSource_len)
      |                ^~~~~~~~~~~
_build/amalgamation/miniz.h:560:16: warning: 'uncompress' defined but not used [-Wunused-function]
  560 |     static int uncompress(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong source_len)
      |                ^~~~~~~~~~
_build/amalgamation/miniz.h:555:16: warning: 'inflateEnd' defined but not used [-Wunused-function]
  555 |     static int inflateEnd(mz_streamp pStream)
      |                ^~~~~~~~~~
_build/amalgamation/miniz.h:550:16: warning: 'inflate' defined but not used [-Wunused-function]
  550 |     static int inflate(mz_streamp pStream, int flush)
      |                ^~~~~~~
_build/amalgamation/miniz.h:545:16: warning: 'inflateReset' defined but not used [-Wunused-function]
  545 |     static int inflateReset(mz_streamp pStream)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:540:16: warning: 'inflateInit2' defined but not used [-Wunused-function]
  540 |     static int inflateInit2(mz_streamp pStream, int window_bits)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:535:16: warning: 'inflateInit' defined but not used [-Wunused-function]
  535 |     static int inflateInit(mz_streamp pStream)
      |                ^~~~~~~~~~~
_build/amalgamation/miniz.h:527:21: warning: 'compressBound' defined but not used [-Wunused-function]
  527 |     static mz_ulong compressBound(mz_ulong source_len)
      |                     ^~~~~~~~~~~~~
_build/amalgamation/miniz.h:523:16: warning: 'compress2' defined but not used [-Wunused-function]
  523 |     static int compress2(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len, int level)
      |                ^~~~~~~~~
_build/amalgamation/miniz.h:519:16: warning: 'compress' defined but not used [-Wunused-function]
  519 |     static int compress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len)
      |                ^~~~~~~~
_build/amalgamation/miniz.h:515:21: warning: 'deflateBound' defined but not used [-Wunused-function]
  515 |     static mz_ulong deflateBound(mz_streamp pStream, mz_ulong source_len)
      |                     ^~~~~~~~~~~~
_build/amalgamation/miniz.h:511:16: warning: 'deflateEnd' defined but not used [-Wunused-function]
  511 |     static int deflateEnd(mz_streamp pStream)
      |                ^~~~~~~~~~
_build/amalgamation/miniz.h:507:16: warning: 'deflate' defined but not used [-Wunused-function]
  507 |     static int deflate(mz_streamp pStream, int flush)
      |                ^~~~~~~
_build/amalgamation/miniz.h:503:16: warning: 'deflateReset' defined but not used [-Wunused-function]
  503 |     static int deflateReset(mz_streamp pStream)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:499:16: warning: 'deflateInit2' defined but not used [-Wunused-function]
  499 |     static int deflateInit2(mz_streamp pStream, int level, int method, int window_bits, int mem_level, int strategy)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:495:16: warning: 'deflateInit' defined but not used [-Wunused-function]
  495 |     static int deflateInit(mz_streamp pStream, int level)
      |                ^~~~~~~~~~~
/usr/bin/clang
Test compile with clang...
main.c:1:9: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
    1 | int main() { return 0; }
      |         ^
      |          void
1 warning generated.
_build/amalgamation/miniz.c:3203:9: warning: Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files. [-W#pragma-messages]
 3203 | #pragma message("Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.")
      |         ^
In file included from _build/amalgamation/miniz.c:1:
_build/amalgamation/miniz.h:495:16: warning: unused function 'deflateInit' [-Wunused-function]
  495 |     static int deflateInit(mz_streamp pStream, int level)
      |                ^~~~~~~~~~~
_build/amalgamation/miniz.h:499:16: warning: unused function 'deflateInit2' [-Wunused-function]
  499 |     static int deflateInit2(mz_streamp pStream, int level, int method, int window_bits, int mem_level, int strategy)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:503:16: warning: unused function 'deflateReset' [-Wunused-function]
  503 |     static int deflateReset(mz_streamp pStream)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:507:16: warning: unused function 'deflate' [-Wunused-function]
  507 |     static int deflate(mz_streamp pStream, int flush)
      |                ^~~~~~~
_build/amalgamation/miniz.h:511:16: warning: unused function 'deflateEnd' [-Wunused-function]
  511 |     static int deflateEnd(mz_streamp pStream)
      |                ^~~~~~~~~~
_build/amalgamation/miniz.h:515:21: warning: unused function 'deflateBound' [-Wunused-function]
  515 |     static mz_ulong deflateBound(mz_streamp pStream, mz_ulong source_len)
      |                     ^~~~~~~~~~~~
_build/amalgamation/miniz.h:519:16: warning: unused function 'compress' [-Wunused-function]
  519 |     static int compress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len)
      |                ^~~~~~~~
_build/amalgamation/miniz.h:523:16: warning: unused function 'compress2' [-Wunused-function]
  523 |     static int compress2(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len, int level)
      |                ^~~~~~~~~
_build/amalgamation/miniz.h:527:21: warning: unused function 'compressBound' [-Wunused-function]
  527 |     static mz_ulong compressBound(mz_ulong source_len)
      |                     ^~~~~~~~~~~~~
_build/amalgamation/miniz.h:535:16: warning: unused function 'inflateInit' [-Wunused-function]
  535 |     static int inflateInit(mz_streamp pStream)
      |                ^~~~~~~~~~~
_build/amalgamation/miniz.h:540:16: warning: unused function 'inflateInit2' [-Wunused-function]
  540 |     static int inflateInit2(mz_streamp pStream, int window_bits)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:545:16: warning: unused function 'inflateReset' [-Wunused-function]
  545 |     static int inflateReset(mz_streamp pStream)
      |                ^~~~~~~~~~~~
_build/amalgamation/miniz.h:550:16: warning: unused function 'inflate' [-Wunused-function]
  550 |     static int inflate(mz_streamp pStream, int flush)
      |                ^~~~~~~
_build/amalgamation/miniz.h:555:16: warning: unused function 'inflateEnd' [-Wunused-function]
  555 |     static int inflateEnd(mz_streamp pStream)
      |                ^~~~~~~~~~
_build/amalgamation/miniz.h:560:16: warning: unused function 'uncompress' [-Wunused-function]
  560 |     static int uncompress(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong source_len)
      |                ^~~~~~~~~~
_build/amalgamation/miniz.h:565:16: warning: unused function 'uncompress2' [-Wunused-function]
  565 |     static int uncompress2(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong* pSource_len)
      |                ^~~~~~~~~~~
_build/amalgamation/miniz.h:571:21: warning: unused function 'crc32' [-Wunused-function]
  571 |     static mz_ulong crc32(mz_ulong crc, const unsigned char *ptr, size_t buf_len)
      |                     ^~~~~
_build/amalgamation/miniz.h:576:21: warning: unused function 'adler32' [-Wunused-function]
  576 |     static mz_ulong adler32(mz_ulong adler, const unsigned char *ptr, size_t buf_len)
      |                     ^~~~~~~
_build/amalgamation/miniz.h:584:24: warning: unused function 'zError' [-Wunused-function]
  584 |     static const char* zError(int err)
      |                        ^~~~~~
20 warnings generated.
/usr/lib/gcc/aarch64-alpine-linux-musl/14.2.0/../../../../aarch64-alpine-linux-musl/bin/ld: cannot find /usr/lib/llvm20/lib/clang/20/lib/aarch64-alpine-linux-musl/libclang_rt.ubsan_standalone.a: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

./amalgamate.sh doesn't seem to be usable on ARM64 musl-libc Linux systems

1 participant

@ell1e