ghidra: package the reverse-engineering suite 12.1.2 - #545

The only credible open decompiler for native code, and the anchor of a
reverse-engineering loadout. `analyzeHeadless` is the point: it makes
Ghidra scriptable across a fleet of binaries rather than a GUI you drive
by hand.

Apache-2.0 at the root; the tree aggregates ~20 further licenses
(GPL-2.0 WITH Classpath-exception, LGPL, BSD, MIT, Zlib, MPL-2.0,
Python-2.0 …) — all OSI/FSF-free, nothing proprietary. Upstream keeps
GPL code in the standalone top-level GPL/ module by policy.

PREBUILT RELEASE, NOT A SOURCE BUILD

Ghidra's Gradle build runs fetchDependencies.gradle BEFORE compiling
anything: 67 URLs across seven hosts, including binaries with no source
(AXMLPrinter2.jar from the dead Google Code archive, per-platform Z3
zips), a postgresql tarball and PyPI wheels — then Maven resolution. It
also needs JDK 25 to build; we ship 21. The release zip is the same
tradeoff the jdk package already makes, and it bundles NO JRE, so Ghidra
runs on our jdk rather than smuggling in a second Java.

THE ARM64 GAP, CLOSED

Upstream publishes natives for linux_x86_64, mac_arm_64, mac_x86_64 and
win_x86_64 — and NOT linux_arm_64. On an arm64 host that is not a
warning, it is a silently degraded run: Application.getModuleOSFile()
implements an x86_64 fallback only for WIN_ARM_64 and MAC_ARM_64, so
`decompile` is simply not found, and analyzeHeadless exits 0 having
produced no C.

Closed by building the five natives from the C/C++ sources the release
zip ALREADY ships (src/decompile/**, GPL/DemanglerGnu/src/**,
src/lzfse/**). No second Source, no network, no Gradle — and no second
sha256 for the updater to leave stale, since it rewrites only the first.

  ARCH_TYPE=   the whole fix. The Makefile's detection is
               `ifeq ($(ARCH),x86_64) -m64 else -m32` with an in-tree
               "TODO: need to revise to support arm64/aarch64", so
               aarch64 lands in the -m32 branch. `make ARCH=aarch64`
               does NOT help — every non-x86_64 value takes the else.
  YACC/LEX=false  tripwires. `python3 -m zipfile` restores no mtimes, so
               slghparse.y can look newer than the shipped slghparse.cc
               and make invokes bison. The generated files are touched
               first; these make a regression fail loudly rather than
               silently need a tool we do not ship.

TWO BUGS FIXED ON THE WAY, BOTH PRE-EXISTING

  * GPL/DemanglerGnu/os/ was never chmod'd — the globs reached
    Ghidra/Features/*/os but not GPL/. Verified in a built tree: both
    demanglers were mode 0644 while the decompiler natives were 0755.
    Ghidra Runtime.exec()s them by absolute path with no fallback, so
    C++ demangling was silently broken.
  * The win/mac prune missed the 7-Zip JNI blobs: sevenzipjbinding uses
    Mac-x86_64 / Windows-amd64, not Ghidra's os/<platform> convention,
    so a Mach-O dylib and a PE DLL survived "drop Windows and macOS".

Also drops docs/GhidraClass — 16 deliberately-crafted training ELFs that
nothing execs and that pkgscan cannot see.

VERIFIED

  minimal package --rebuild --no-fetch ghidra   exit 0
  minimal check --packages ghidra               14/14 pass
  natives                                       5/5 ELF aarch64, mode 0755
  foreign-arch binaries left in tree            0
  installed size                                850M

The natives are declared as OutputBin with linux_* globs deliberately:
the output-types checker then parses them as ELF for the target arch, so
if a future bump ships x86_64-only and the native build silently
no-ops, `minimal check` FAILS with "expected Arm64". That is the guard
that caught this gap in the first place.

KNOWN LIMITS

  * 850M installed — by far the largest package in the repo. Worth a
    conscious call on cache/closure pressure.
  * 7z/RAR container browsing is unavailable on arm64: the vendored
    sevenzipjbinding-16.02 jar has no aarch64 build for any OS. It
    degrades to one warning; ELF analysis never touches it.
  * The x86_64 path takes upstream's shipped natives (the `[ -f ]`
    guard) and could not be exercised on this arm64 host.
  * analyzeHeadless producing no C while exiting 0 is a runtime property
    no checker can see. A decompile smoketest would pin it; not in this
    PR.

Three fixes, each verified against a real built tree.

1. 7-ZIP CONTAINERS — the one genuine capability gap, and it was closable

The previous revision of this file asserted "no aarch64 build for ANY OS
… nothing below can fix that". That was wrong, and wrong in the way that
stops anyone looking again. Upstream's `-all-platforms` roll-up excludes
ARM BY DESIGN (their ReleaseNotes: "ARM (WARNING: Not a part of
-AllPlatform or -AllLinux !!!)") — but the per-platform artifact is on
Maven Central at the EXACT version Ghidra vendors, published Feb 2020.

Verified independently: HTTP 200, sha1 matches the published .sha1, and
the shipped .so is e_machine 0xb7 (aarch64) needing only glibc 2.17.

Without it an arm64 user loses 7z/RAR/CAB/CHM/LHA/ARJ/WIM/VHD/XAR/RPM,
and — worse because it is silent — .zip/.apk/.jar quietly fall back to
java.util.zip, losing password-protected and obfuscated entries.

The swap has one non-obvious constraint, so it is commented in place:
getPlatformBestMatch() short-circuits only when the platform list holds
exactly ONE entry, otherwise it matches os.arch ("aarch64") against the
artifact's "Linux-arm64" and can never succeed. So the arm64 jar must
REPLACE the all-platforms jar, and the string "Linux-arm64" must stay
verbatim — it indexes both the on-disk data dir and a classpath
resource.

The native pin is independent of the Ghidra version (upstream froze it),
so build.sh asserts the vendored version still matches and FAILS if a
future bump revendors it. That skew would otherwise degrade silently.

2. 101 EXEC BITS, INCLUDING 27 SHELL SCRIPTS

`python3 -m zipfile` drops the exec bit, and the hand-maintained restore
list has now been wrong twice: first the two GNU demanglers, and still
today every debugger-launcher .sh and all of server/, because
`support/*` is a SHALLOW glob. Measured before: 1 executable .sh in the
whole tree against 27 not. After: 25 against 3.

Stop maintaining a list — restore the bit on exactly what UPSTREAM
marked executable, read from the zip's own mode metadata, skipping what
we deliberately deleted. The 3 that stay non-executable are
`*setuputils.sh`, which are SOURCED libraries; a blanket chmod would
have wrongly marked them.

3. THE BUG THIS PR ITSELF SHIPPED FIRST

My initial 7-Zip integration silently did nothing: it branches on
$HOST_OSDIR, which was defined 70 lines BELOW it, so the test compared
against an empty string and skipped. Build succeeded; package simply
lacked the thing it had just been taught to add.

Arch detection is now resolved once, above every consumer, and build.sh
runs under `set -u` so an unset variable is fatal rather than quiet.
That is the same silent-skip class this package keeps producing, so it
is now structurally prevented rather than fixed once.

VERIFIED

  7-Zip native   755, ELF aarch64 shared object, in Linux-arm64/
  sevenzip jars  exactly 2 (classes + linux-arm64; all-platforms gone)
  natives        5/5 still ELF aarch64
  exec bits      25 .sh executable, 3 sourced-only
  minimal check  14/14 pass

STILL NOT AT PARITY, deliberately

  * Z3 symbolic execution (SymbolicSummaryZ3) ships no linux_arm_64.
    Fails LOUDLY (UnsatisfiedLinkError), and closing it means moving off
    the pinned Z3 — upstream's own arm64 asset is mislabelled and
    contains x86-64 binaries, which is why the 7-Zip injection above
    asserts e_machine on anything it adds.
  * FlatLaf window decorations: GUI cosmetic only.
  * SIGAR: no aarch64 exists for any OS; dead upstream since 2010.

Adds headless_decompile, the only test here that checks Ghidra FINDS AND
USES its natives rather than merely that they exist with the right arch.

The failure it guards exits 0 today. When the decompile native is missing
or unloadable, DecompileProcessFactory logs once behind a static latch,
openProgram() returns false, decompileFunction() short-circuits with
errMsg="" so CppExporter's guard never fires, and analyzeHeadless writes
an empty .c and exits 0. Worse, the results object is built with
DISPOSED_ON_CANCEL, so isCancelled() reports a missing native as a USER
CANCELLATION and isValid() returns TRUE for a decompile that produced
nothing. The script is written against decompileCompleted() alone.

Nothing asserts on analyzeHeadless's exit code — it has zero System.exit
calls and also exits 0 when the postScript throws. Every assertion is on
an artifact the script can only write by really decompiling.

The C++ fixture is namespaced on purpose: Foo::bar surviving into
names.txt also proves the GNU demangler natives ran, which nothing else
covered and which shipped mode 0644 for the life of this package.

MUTATION-TESTED: breaking the demangle assertion makes minimal check
FAIL, so the test is not vacuous. Unmutated it passes — meaning arm64
Ghidra genuinely decompiled a C++ binary and demangled the symbol.

14/14 checks pass.

The parity fix rested on the weakest evidence in the package: the .so is
present, is aarch64, is mode 0755. None of that shows Ghidra can open a
7z — and the failure is silent twice over. SevenZipCustomInitializer
catches Throwable and rewraps, initNativeLibraries() eats the result, and
analyzeHeadless swallows the resulting IOException and exits 0 with an
empty listing. 'File exists' and 'feature works' are separated by three
layers that each convert an error into a shrug.

The test uses sevenzipjbinding's OWN documented entry point rather than
Ghidra's wrapper, so it exercises the jar+native pairing directly, and
asserts three things in order of increasing strength:

  PLATFORM= present          the native initialised at all — the
                             UnsatisfiedLinkError / wrong-arch /
                             unsatisfiable-DT_NEEDED case
  PLATFORM=Linux-arm64       it resolved to the ARM platform. If the jar
                             swap ever regresses to shipping
                             all-platforms alongside, the platform list
                             stops being a single entry,
                             getPlatformBestMatch() falls back to
                             matching os.arch ('aarch64') against
                             'Linux-arm64', and can never succeed. This
                             is the exact trap build.sh warns about.
  ENTRY=SEVENZIP_PARITY_OK.txt   it actually DECOMPRESSED the container.

Fixture is a real 187-byte 7z, embedded base64 because the sandbox ships
GNU tar, which cannot WRITE 7z (bsdtar can; we do not package it).
Decoding needs nothing but coreutils, and the test sanity-checks the
fixture's own magic first — a green test against a non-7z would prove
nothing.

Mutation-tested, and the mutation output is the evidence:

    standalone tests...Fail
      sevenzip_native: grep -q 'PLATFORM=IMPOSSIBLE-SENTINEL' exit code 1
      stdout:
      PLATFORM=Linux-arm64

arm64 7z support is now demonstrated end to end rather than asserted.

14/14 checks pass.

CI caught this on amd64, and the failure is the useful kind:

    sevenzip_native: grep -q 'PLATFORM=Linux-arm64' exit code 1
    stdout:
    PLATFORM=Linux-amd64
    ENTRY=SEVENZIP_PARITY_OK.txt

The test was wrong, not the package. I hardcoded the architecture I
happened to build on. Note what the same output proves: on amd64 the
native initialised AND decompressed the container — ENTRY is right there
— so 7z works on both arches and only the assertion was too specific.

Now derived from uname: Linux-arm64 on aarch64, Linux-amd64 on x86_64,
and an explicit failure on anything else rather than a silently skipped
assertion.

It still catches the regression it exists for. On arm64 the answer is
Linux-arm64 ONLY because build.sh swaps the all-platforms jar for the
per-platform one; if that regresses, the platform list stops being a
single entry, getPlatformBestMatch() falls back to matching os.arch
("aarch64") against "Linux-arm64", and cannot succeed — so an arm64 host
reports Linux-amd64 and the test fails. Verified that case explicitly:

  uname=x86_64   want=Linux-amd64  vs CI's real output   PASS
  uname=aarch64  want=Linux-arm64  vs our build          PASS
  uname=aarch64  serving Linux-amd64 (the regression)    FAIL

This is the first real feedback from the amd64 path, which I had flagged
as untested — the [ -f ] guard that takes upstream's natives on x86_64
now has evidence behind it too.

14/14 on arm64 after the change.