Add 6 foundational libraries imported from Wolfi (brotli, jsoncpp, jemalloc, re2, zeromq, stress-ng) - #367
Conversation
…malloc, re2, zeromq, stress-ng) The first batch from `pkgmgr import-wolfi` — a new tool that seeds Minimal build definitions from Wolfi (wolfi-dev/os) melange recipes, then refines outputs + runtime_deps from a verified build. Chosen by dependency fan-in (foundational) + GitHub stars (demand); all build-proven and reproducibly sourced. | package | upstream | build | runtime_deps (from ELF DT_NEEDED) | |---|---|---|---| | brotli 1.2.0 | google/brotli | cmake, C | glibc | | jsoncpp 1.9.8 | open-source-parsers/jsoncpp | cmake, C++ | glibc, gcc[libgcc,libstdcpp] | | jemalloc 5.3.1 | jemalloc/jemalloc | autotools | glibc, gcc[libgcc,libstdcpp] | | re2 2025-11-05 | google/re2 | cmake, C++ | glibc, gcc[..], abseil-cpp, icu | | zeromq 4.3.5 | zeromq/libzmq | cmake, C++ | glibc, gcc[..], libsodium | | stress-ng 0.21.03 | ColinIanKing/stress-ng | Makefile | glibc, gcc[libatomic], acl, gmp, libjpeg-turbo, mpfr, xz, zlib | How they were produced: - Sources mirrored to gs://minimal-staging-archives/<owner>/<repo>/<tag>.tar.gz (reproducible; sha256-pinned) and referenced via gs:// URLs. - outputs derived from the real install manifest; runtime_deps resolved from every installed object's ELF DT_NEEDED (verified exact against `ldd`). - Each builds clean and passes every `minimal check` checker (parse/fmt/imports/ output-naming/enumerate-bins/missing-runtime_deps/standalone-tests) at 0 TODOs. - Reviewed adversarially in 3 rounds (~120 findings) — provenance verified correct for vuln-scanning on all 6; the CMake exports resolve for downstream find_package(); tests compile-link-run real consumers. Reviewer notes (surfaced, not hidden): - jemalloc: gcc-16 build fix vendored as a .patch (upstream cherry-pick 1a15fe33, matching Wolfi) — inert on gcc-15, prevents FTBFS once gcc bumps to 16. jemalloc.pc reports `Version: 5.3.1_` (Wolfi/upstream parity — jemalloc's own pkg-config template; passes every pkg-config version gate; header + config script report clean 5.3.1). - stress-ng: a *different* (not strictly smaller) stressor set than Wolfi — 8 optional feature libs (keyutils/kmod/libaio/libsctp/…) aren't packaged in Minimal so those stressors compile out as skip-stubs; Minimal adds gmp/mpfr/xz stressors Wolfi omits. Reduced-scope by construction, builds fail-open. - Follow-up (not blocking): no package in pkgs (imports or production siblings abseil-cpp/protobuf/gtest) exercises a find_package() consumer in its test — worth a `minimal check` step that auto-generates one for any cmake_data output. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 3 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (10)
📝 WalkthroughWalkthroughThis PR adds new Wolfi-imported package build specifications and shell build scripts for six packages: brotli, jemalloc, jsoncpp, re2, stress-ng, and zeromq. Each package defines build/runtime dependencies, outputs, metadata, and tests. Jemalloc and zeromq additionally include patches (gcc-16 C++ exceptions fix, and a curve_keygen CMake build/install addition). ChangesBrotli Package
Jemalloc Package
Jsoncpp Package
RE2 Package
Stress-ng Package
ZeroMQ Package
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related issues
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 9
🧹 Nitpick comments (2)
packages/brotli/build.ncl (2)
4-9: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winImport order violates alphabetical ordering guideline.
glibc(line 9) should be ordered alphabetically betweencmakeandmake, not placed last.♻️ Proposed fix
let base = import "../base/build.ncl" in let cmake = import "../cmake/build.ncl" in +let glibc = import "../glibc/build.ncl" in let make = import "../make/build.ncl" in let pkgconf = import "../pkgconf/build.ncl" in let toolchain = import "../toolchain/build.ncl" in -let glibc = import "../glibc/build.ncl" inAs per coding guidelines, "Import package dependencies... with lowercase identifiers ordered alphabetically before uppercase identifiers."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/brotli/build.ncl` around lines 4 - 9, The import list in build.ncl is out of alphabetical order, with glibc placed after toolchain instead of between cmake and make. Reorder the top-level imports in the package dependency block so the lowercase identifiers are sorted alphabetically, keeping the existing import names like base, cmake, glibc, make, pkgconf, and toolchain in the correct sequence.Source: Coding guidelines
13-20: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
makebuild dep used instead ofninjafor a CMake build.Coding guidelines require CMake builds to use the Ninja generator, but this package pulls in
makeandbuild.shinvokes cmake without-G Ninja. See paired comment inbuild.sh.As per coding guidelines, "For CMake builds, use Ninja generator with '-DCMAKE_BUILD_TYPE=Release' and '-DCMAKE_INSTALL_PREFIX=/usr'."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/brotli/build.ncl` around lines 13 - 20, The Brotli CMake package is using make instead of the required Ninja-based CMake flow. Update the build definition and related build script so the package depends on ninja rather than make, and make sure build.sh invokes cmake with the Ninja generator plus the required Release build type and /usr install prefix. Use the existing build_deps list and build.sh entry points to locate the changes.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/brotli/build.sh`:
- Around line 5-8: The CMake build in build.sh is missing the required Ninja
generator and reproducibility toolchain flags. Update the cmake configure step
to use the Ninja generator in the existing CMake invocation, and ensure the
build script exports CFLAGS/CXXFLAGS, LDFLAGS, and ARFLAGS with the required
reproducibility values before running cmake --build and cmake --install. Keep
the existing build flow centered around the current cmake and DESTDIR install
commands, and align the flags with the package build guidelines for reproducible
C/C++ builds.
In `@packages/jemalloc/build.sh`:
- Around line 9-13: The jemalloc autotools build is missing the required
reproducibility settings; update the build logic in build.sh to export the
standard C/C++ reproducibility flags before configure/make, including
CFLAGS/CXXFLAGS with the file-prefix map and no-record-gcc-switches options,
LDFLAGS with build-id disabled, and ARFLAGS set for deterministic archives. Also
pass --enable-deterministic-archives to the ./configure invocation, and after
make install remove any generated .la files from "$OUTPUT_DIR" with the existing
post-install cleanup pattern so the output stays reproducible.
In `@packages/jsoncpp/build.sh`:
- Around line 12-13: The build script for the JSONCPP package is using the
default CMake generator instead of the required Ninja generator, and the related
package manifest is missing the Ninja build dependency. Update the build flow in
build.sh to invoke CMake with the Ninja generator while keeping the existing
Release and /usr install settings, and adjust build.ncl so build_deps includes
ninja or ninja-build. Use the existing cmake invocation in build.sh and the
package metadata in build.ncl as the places to fix.
- Line 12: The cmake invocation in the jsoncpp build script is missing the
reproducibility flags required by the coding guidelines. Update the build
configuration to pass the C/C++ compiler flags and linker flags through the
CMake variables used by the build, and ensure the archive tool flags are set as
well; locate the change in the cmake command itself so the build applies the
same reproducibility settings consistently. Use the existing build script entry
point to add the required flag values rather than changing unrelated build
behavior.
In `@packages/re2/build.ncl`:
- Line 10: The import binding in the top-level let expression uses an invalid
Nickel identifier, so rename the binding in the import statement to a valid bare
name and update any references to that binding in the same file. Locate the
issue in the let/import declaration for the abseil-cpp dependency and choose an
identifier that does not contain hyphens.
In `@packages/re2/build.sh`:
- Around line 6-7: The build script currently relies on CMake’s default
Makefiles generator instead of Ninja, so update the cmake invocation in build.sh
to explicitly use the Ninja generator while keeping the existing Release build
type and install prefix settings. Make sure the build pipeline is aligned with
this change by adjusting the related build dependency from make to ninja in
build.ncl so cmake --build uses Ninja consistently.
- Around line 6-8: The RE2 build script is missing the required reproducibility
flags for its C/C++ build. Update the build setup in build.sh so the cmake
invocation or surrounding environment exports CFLAGS and CXXFLAGS with
-ffile-prefix-map=$(pwd)=/builddir -gno-record-gcc-switches, LDFLAGS with
-Wl,--build-id=none, and ARFLAGS='Drc'. Keep the changes near the existing cmake
-S . -B build and cmake --build build steps so the flags are applied throughout
the build and install flow.
In `@packages/stress-ng/build.sh`:
- Around line 1-6: The build script is missing the reproducibility flags
required for C/C++ builds. Update the package build flow in build.sh so the make
invocation for stress-ng exports CFLAGS and CXXFLAGS with the file-path mapping
and no-record-switches options, sets LDFLAGS to disable build IDs, and sets
ARFLAGS to Drc before calling make/install. Keep the changes anchored around the
existing CC/CXX setup and make commands so the compiled binary does not leak
build paths or nondeterministic metadata.
In `@packages/zeromq/build.sh`:
- Around line 11-13: The Zeromq CMake build is using the default generator
instead of Ninja, so update the build flow in build.sh to invoke cmake with the
Ninja generator while keeping the existing Release build type and install prefix
settings. Also make sure the package metadata in build.ncl includes ninja in
build_deps alongside make so the generator is available during builds.
---
Nitpick comments:
In `@packages/brotli/build.ncl`:
- Around line 4-9: The import list in build.ncl is out of alphabetical order,
with glibc placed after toolchain instead of between cmake and make. Reorder the
top-level imports in the package dependency block so the lowercase identifiers
are sorted alphabetically, keeping the existing import names like base, cmake,
glibc, make, pkgconf, and toolchain in the correct sequence.
- Around line 13-20: The Brotli CMake package is using make instead of the
required Ninja-based CMake flow. Update the build definition and related build
script so the package depends on ninja rather than make, and make sure build.sh
invokes cmake with the Ninja generator plus the required Release build type and
/usr install prefix. Use the existing build_deps list and build.sh entry points
to locate the changes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: a6fc55af-341c-4803-a998-f587400a2809
📒 Files selected for processing (14)
packages/brotli/build.nclpackages/brotli/build.shpackages/jemalloc/build.nclpackages/jemalloc/build.shpackages/jemalloc/jemalloc-gcc16-cxx-exceptions.patchpackages/jsoncpp/build.nclpackages/jsoncpp/build.shpackages/re2/build.nclpackages/re2/build.shpackages/stress-ng/build.nclpackages/stress-ng/build.shpackages/zeromq/0001-cmake-add-curve_keygen-binary.patchpackages/zeromq/build.nclpackages/zeromq/build.sh
| cmake -S . -B build -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_BUILD_TYPE=Release -DCMAKE_POLICY_VERSION_MINIMUM=3.5 -DBUILD_STATIC_LIBS=OFF -DBUILD_OBJECT_LIBS=OFF | ||
| cmake --build build -j"$(nproc)" |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Not using the Ninja generator required by coding guidelines.
As per coding guidelines, "For CMake builds, use Ninja generator with '-DCMAKE_BUILD_TYPE=Release' and '-DCMAKE_INSTALL_PREFIX=/usr'". This script omits -G Ninja (defaulting to Unix Makefiles) and doesn't include ninja/ninja-build in build_deps in the corresponding build.ncl.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/jsoncpp/build.sh` around lines 12 - 13, The build script for the
JSONCPP package is using the default CMake generator instead of the required
Ninja generator, and the related package manifest is missing the Ninja build
dependency. Update the build flow in build.sh to invoke CMake with the Ninja
generator while keeping the existing Release and /usr install settings, and
adjust build.ncl so build_deps includes ninja or ninja-build. Use the existing
cmake invocation in build.sh and the package metadata in build.ncl as the places
to fix.
Source: Coding guidelines
| let pkgconf = import "../pkgconf/build.ncl" in | ||
| let toolchain = import "../toolchain/build.ncl" in | ||
| let glibc = import "../glibc/build.ncl" in | ||
| let abseil-cpp = import "../abseil-cpp/build.ncl" in |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check if hyphenated let-identifiers are used elsewhere in the repo and whether they parse.
rg -n 'let [a-z0-9_]+-[a-z0-9_-]+ = import' --type-add 'ncl:*.ncl' -t nclRepository: gominimal/pkgs
Length of output: 152
🌐 Web query:
Nickel language identifier syntax hyphen unquoted let binding variable names
💡 Result:
In the Nickel language, standard identifiers must follow a specific naming convention: they start with zero or more underscores, followed by an alphabetic character (a–z or A–Z) [1]. Because of this requirement, identifiers cannot contain hyphens (e.g., -) when written in their standard, unquoted form. If you need to use an identifier that includes a hyphen or other characters not permitted by the standard identifier syntax, you must use dynamic record access or similar workarounds, as the language does not currently support unquoted raw identifiers containing hyphens in let bindings [2]. For let bindings specifically, Nickel supports standard let expressions (e.g., let x = 1 in x) as well as a block syntax for multiple bindings that uses commas as separators (e.g., let x = 1, y = 2 in x + y) [3][4]. Neither of these syntaxes allows the use of hyphens within the unquoted variable name itself.
Citations:
- 1: https://nickel-lang.org/user-manual/syntax/
- 2: Syntax for raw identifiers nickel-lang/nickel#2534
- 3: The Syntax Bikeshedding Dojo, round 5: Let bindings nickel-lang/nickel#218
- 4: Block syntax for let-binding nickel-lang/nickel#494
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the relevant build file and search for any Nickel-style bindings with hyphenated names.
sed -n '1,80p' packages/re2/build.ncl
printf '\n---\n'
rg -n 'let [^=]+- [^=]+ =|let [^=]+-[^=]+ =' packages -g '*.ncl'Repository: gominimal/pkgs
Length of output: 13007
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read the target file and inspect nearby Nickel import bindings for naming patterns.
cat -n packages/re2/build.ncl | sed -n '1,40p'
printf '\n---\n'
# Look for any let-bindings with hyphens or quoted identifiers in build.ncl files.
rg -n '^\s*let\s+[^=]*-[^=]*=' packages -g '*.ncl' || true
printf '\n---\n'
rg -n '^\s*let\s+["'\''][^"'\''"]+["'\'']\s*=' packages -g '*.ncl' || trueRepository: gominimal/pkgs
Length of output: 12138
🌐 Web query:
Nickel language syntax let binding identifiers hyphen variable names official docs
💡 Result:
In the Nickel configuration language, standard identifiers (used for variable names in let bindings or field names) cannot contain hyphens [1]. According to the Nickel user manual, identifiers must start with zero or more underscores, followed by an alphabetic character [1]. Consequently, hyphens are not valid characters for standard Nickel identifiers. If you need to use a key containing a hyphen (for example, when generating JSON or YAML that requires hyphens in keys), you must use string-based record fields or dynamic identifiers [2]. While standard identifiers do not support hyphens, string literals used as record fields do [2]. Note: Ensure you do not confuse the Nickel configuration language (nickel-lang.org) with the unrelated language Nickle (nickle.org), which has different syntax rules [3].
Citations:
- 1: https://nickel-lang.org/user-manual/syntax/
- 2: Syntax for raw identifiers nickel-lang/nickel#2534
- 3: https://nickle.org/manual.html
Rename this binding to a valid Nickel identifier. abseil-cpp is not a valid bare let name, so this line will not parse as written.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/re2/build.ncl` at line 10, The import binding in the top-level let
expression uses an invalid Nickel identifier, so rename the binding in the
import statement to a valid bare name and update any references to that binding
in the same file. Locate the issue in the let/import declaration for the
abseil-cpp dependency and choose an identifier that does not contain hyphens.
AGENTS.md requires C/C++ builds to strip build paths / build-id and to build CMake with Ninja; the importer omitted these (now fixed at the template in pkgmgr, this applies the same to the already-imported packages): - all six: export CFLAGS/CXXFLAGS (-ffile-prefix-map=$(pwd)=/builddir -gno-record-gcc-switches), LDFLAGS (-Wl,--build-id=none), ARFLAGS=Drc. - brotli, jsoncpp, re2, zeromq (cmake): pass -G Ninja; build_deps make -> ninja. - jemalloc (autotools): --enable-deterministic-archives + drop *.la post-install. - stress-ng (make): drop *.la post-install. All six rebuilt clean (minimal package --rebuild --no-fetch). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addressed CodeRabbit review (pushed)The dominant finding was the AGENTS.md C/C++ reproducibility flags. Fixed on all six, and root-caused into the importer so it stops recurring (pkgmgr-rs #471):
All six rebuilt clean with One dismissed as a false positive
|
First batch from
pkgmgr import-wolfi— a new tool that seeds Minimal build definitions from Wolfi (wolfi-dev/os) melange recipes, then refines outputs + runtime_deps from a verified build. These 6 were chosen by dependency fan-in (foundational) + GitHub stars (demand).DT_NEEDED)How they were produced
gs://minimal-staging-archives/<owner>/<repo>/<tag>.tar.gz(reproducible, sha256-pinned) and referenced viags://URLs.DT_NEEDED(verified exact againstldd).minimal checkchecker (parse/fmt/imports/output-naming/enumerate-bins/missing-runtime_deps/standalone-tests) at 0 TODOs.find_package(); the tests compile-link-run real consumers.Reviewer notes (surfaced, not hidden)
.patch(upstream cherry-pick1a15fe33, matching Wolfi) — inert on gcc-15, prevents FTBFS once gcc bumps to 16.jemalloc.pcreportsVersion: 5.3.1_(Wolfi/upstream parity — it's jemalloc's own pkg-config template; passes every pkg-config version gate; header +jemalloc-configreport clean5.3.1).find_package()consumer in its test block. That's the exact seam a lib64 export bug slipped through during review. Worth aminimal checkstep that auto-generates afind_package(<pkg> REQUIRED)smoke test for any package shipping acmake_dataoutput, closing it fleet-wide.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
curve_keygencommand-line tool.Bug Fixes
Tests