test: prove the in-sandbox min add path via interactive PTY attach on every lane - #760

…ox proof

The native-e2e lane dropped its userns sysctl step on the rationale that
session-e2e.sh forks no sandbox — true when written, falsified by the
in-sandbox `min add` proof (#760) merged since: on a restricted runner
the proof's sandbox dies at uid_map and surfaces as the misleading
'jq was already present' error (the absent-marker never prints).

Load the shipped profile from the script itself, attached to the
minimald the run will spawn — dogfooding the documented remediation on
CI and on any restricted host running the e2e locally. Native Linux
only: VM lanes create their userns in-guest as root.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ox proof

The native-e2e lane dropped its userns sysctl step on the rationale that
session-e2e.sh forks no sandbox — true when written, falsified by the
in-sandbox `min add` proof (#760) merged since: on a restricted runner
the proof's sandbox dies at uid_map and surfaces as the misleading
'jq was already present' error (the absent-marker never prints).

Load the shipped profile from the script itself, attached to the
minimald the run will spawn — dogfooding the documented remediation on
CI and on any restricted host running the e2e locally. Native Linux
only: VM lanes create their userns in-guest as root.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(minimald): ship an AppArmor profile granting minimald userns

Ubuntu 24.04+ defaults kernel.apparmor_restrict_unprivileged_userns=1,
which bars unconfined programs from creating unprivileged user
namespaces. Every minimald session is one (hakoniwa forks and the child
writes /proc/self/uid_map), so on a stock 24.04 host no session can
start: the child dies with EPERM before it runs, surfacing only as an
opaque uid_map DIAG in the daemon log.

Ship the accommodation Ubuntu intends, the same shape the distro ships
for rootlesskit, runc and podman: an unconfined-mode profile whose sole
purpose is to give minimald a named label carrying the `userns`
permission. It confines nothing, and unlike setting the sysctl to 0 it
does not hand user namespaces back to every other program on the host.

minimald has no single install path (the installer puts it in
${MINIMAL_BIN:-~/.local/bin}, a package would use /usr/bin), and AppArmor
attaches profiles by executable path, so the attachment set lives in a
tunable; --path extends it for dev builds via tunables/minimald.d.

Verified on Ubuntu with the restriction on: without the profile an
interactive attach yields no sandbox shell; with it loaded the daemon
runs as `minimald (unconfined)` and the shell comes up in its own user
namespace. Unconfined processes remain unable to unshare.

Refs: #610

* fix(scripts): assign SRC_DIR before readonly in the apparmor installer

shellcheck SC2155: `readonly VAR=$(cmd)` masks the command's exit
status, so a missing packaging/apparmor dir would go unnoticed instead
of failing the script.

* ci: drop dead userns sysctl from the native e2e job

native-daemon-e2e lifted Ubuntu's unprivileged-userns restriction
(`sysctl ...=0`) on the theory that the session sandbox needs it. It does
not: session-e2e.sh drives activate/list/destroy/stop only and never
forks a session sandbox, so the job needs no user namespace — verified by
running the e2e green with the restriction left on. Remove the step and
leave a comment so it is not re-added; correct the job header, which
wrongly claimed the e2e execs inside the sandbox (it does not — that path
is proven in Rust, per session-e2e.sh).

Sandbox-forking coverage that genuinely needs userns stays put: the
workspace tests and the minimald-root-integration job (whose own sysctl
step is unaffected — its userns comes from the cargo test binary, not
minimald).

Refs: #610

* feat(installer): ship and advise the minimald AppArmor profile

The curl|sh installer had no path to the AppArmor profile that lets
minimald create the unprivileged user namespace every session sandbox
needs: the profile and its loader lived only in the source tree, so
`sudo scripts/install-apparmor-profile.sh` was a dead reference for
anyone who installed via curl|sh. On stock Ubuntu 24.04+ that left every
session dying at uid_map with an opaque EPERM and no in-product hint.

Plumb it in without ever elevating from the installer:

- stage-release.sh ships packaging/apparmor/{minimald,tunables/minimald}
  and install-apparmor-profile.sh as data-prefix components, installed to
  ~/.local/share/minimal/apparmor/.
- install.sh prints a gated, Linux-only advisory when it detects
  apparmor_restrict_unprivileged_userns=1, pointing at the shipped loader
  (sudo bash ...). Advice only; it never runs sudo.
- On --uninstall, offer to remove the system profile too: prompt on a
  terminal, advise the root command otherwise, run before the record walk
  deletes the loader. Never touches /etc/apparmor.d unattended.
- install-apparmor-profile.sh resolves its source dir from either the
  checkout layout or the shipped-flat layout.
- The install loop now creates each component's parent dir, so a
  component whose subpath nests directories installs correctly.

install_test.sh covers placement, the advisory on/off, darwin-skip, and
the uninstall advisory (no elevation, loader removed).

Refs: #610

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(installer): skip the userns advisory when the profile is installed

The install-time advisory checked only the sysctl and the shipped
loader, so a reinstall on an already-remediated host still claimed the
sandbox "cannot start" — the profile is loaded and sessions work, yet
the sysctl stays 1. Gate the advisory on the absence of the installed
profile at /etc/apparmor.d/minimald (via the same MINIMAL_OVERRIDE_-
APPARMOR_DIR seam the uninstall path uses), so it fires only on a host
that is restricted and not yet remediated.

Also show the loader's output in the curl-installed docs example, so the
command block carries output like its sibling (satisfies markdownlint
MD014).

Both raised by CodeRabbit on #740.

* feat(sandbox2,minimald): preflight the userns restriction at daemon startup

Every session sandbox starts by unsharing an unprivileged user
namespace, forked from the daemon with no exec in between. On a host
that refuses it (stock Ubuntu 24.04+ with an unconfined daemon, or user
namespaces disabled outright) the denial previously surfaced only when
the first attach died writing /proc/self/uid_map, with nothing useful
in the daemon log.

sandbox2 gains user_namespaces_restriction() — a read-only probe of the
userns quota, the apparmor_restrict_unprivileged_userns sysctl, euid,
and the process's own AppArmor label, mirroring the crate's existing
net-namespace probe — and minimald warns at startup naming the
restriction and the exact fix. Root (the in-guest microVM pid-1) and
confined daemons stay silent, so the common case logs nothing.

Refs: #610

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(installer): point the userns advisory at docs.minimal.dev

The reference docs ship to https://docs.minimal.dev/ with this release,
so the advisory's details link should point there rather than at a
GitHub blob URL. The build-specs link to minimal.ncl stays on GitHub:
it targets source, not a hosted doc page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(minimald,paths): derive the advisory's loader path and attach the running binary

The startup warning hardcoded ~/.local/share for the shipped loader, so
the advised command was wrong whenever XDG_DATA_HOME was set, and it
omitted --path, so a daemon outside the tunable's stock attachment set
(a custom MINIMAL_BIN, a dev build) was told to load a profile that
would never attach to it. paths gains minimal_data_dir(), resolving
exactly like the installer's data prefix, and the fix string now carries
--path <current_exe>.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(scripts): preserve arguments in the loader's sudo hint

The parse loop consumes $@ before the root check, so the hinted
`sudo $0 $*` always printed bare — a copy-pasted retry of
--uninstall would install, and one of --path would skip the attachment.
Capture the original argv up front and print that.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(installer): cover custom MINIMAL_BIN in advisory; stop the daemon only for bin/lib swaps

Two review findings against the advisory and the R5.5 stop:

- The stock tunable attaches only /usr/bin, /usr/local/bin, and
  ~/.local/bin, so for a custom MINIMAL_BIN the advised command now
  carries --path "$bindir/minimald", and a loaded profile counts as
  remediation only when the tunables actually name that binary —
  previously the advisory suppressed itself while sessions stayed
  broken.
- stop_running_daemon now fires only for bin/lib component swaps:
  replacing a data file (the shipped apparmor text) must not kill live
  sessions to swap a file the running daemon does not serve from.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(release): stage repo files into the workdir so --dry-run touches nothing

stage_repo_file wrote the three apparmor files into ARTIFACTS_DIR before
the dry-run branch, mutating a directory the caller owns (and dying
outright when it is read-only). Stage into the ephemeral workdir and let
the manifest loop prefer staged files over same-named artifacts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(scripts): load the minimald AppArmor profile before the e2e sandbox proof

The native-e2e lane dropped its userns sysctl step on the rationale that
session-e2e.sh forks no sandbox — true when written, falsified by the
in-sandbox `min add` proof (#760) merged since: on a restricted runner
the proof's sandbox dies at uid_map and surfaces as the misleading
'jq was already present' error (the absent-marker never prints).

Load the shipped profile from the script itself, attached to the
minimald the run will spawn — dogfooding the documented remediation on
CI and on any restricted host running the e2e locally. Native Linux
only: VM lanes create their userns in-guest as root.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

#760 deepened session-e2e.sh with the in-sandbox min add proof, whose
session mint clones the pkgs upstream inside the guest, and added
fetch-gvproxy + MINVMD_GVPROXY_BIN to the KVM and macOS lanes — but not
to nightly-tests' session-e2e-soak. With no gvproxy staged, minvmd
boots the VM switchless (resolution is best-effort and never errors),
the guest has no egress or DNS, and every one of the soak's 10
iterations dies with 'Could not resolve host: github.com' (first seen
on the Jul 16 scheduled run, the first after #760 merged). Stage the
pinned gvproxy and point MINVMD_GVPROXY_BIN at it, exactly as the KVM
lane does.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…efore pushing unstable (#759)

* ci(release): smoke-test the shipped artifacts before publish

Prove the just-built RELEASE binaries actually run before publishing any of
them. Three new jobs download the uploaded artifacts by name, put the SHIPPED
binaries on PATH (never a fresh target/ build), and run the ONE shared session
e2e (scripts/session-e2e.sh — the same proof the CI lanes run):

- smoke-linux-amd64 (ubuntu): DM2 native minimald — the static-musl
  min/minimald boot a host daemon and pass the session e2e.
- smoke-linux-kvm (KVM-capable ubuntu): DM3 — boot a real microVM from the
  shipped native-glibc minvmd + shipped guest kernel/rootfs/initramfs +
  pinned gvproxy; libkrun is materialized from the upstream package (not
  shipped on Linux) via the shipped static mip.
- smoke-macos (self-hosted mini): DM1 — lay the SIGNED min/minvmd/libkrun.1.dylib/
  gvproxy + arm64 guest payload into the installer layout (libkrun.1.dylib a
  lib/ sibling of bin/, resolved via @loader_path/../lib) and boot the microVM.

A new skip-tolerant smoke-success aggregator (if: always(), fails only on
failure/cancellation) is added to the release job's needs, so a smoke failure
blocks every publish step while a skipped smoke-macos (RUN_MACOS_CI kill-switch)
reads as pass. Smoke has no !inputs.dry_run guard: a dry run builds, signs,
assembles, and SMOKES, then publishes nothing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): grant nightly the checks scope its release call needs

release.yml's verify-ci gate (#757) requests checks: read at the job
level. When release.yml runs as a workflow_call from nightly.yml, it
cannot hold any scope the caller's permissions block doesn't grant, and
nightly.yml granted only contents: write and id-token: write — so every
scheduled run since #757 merged died at workflow compile time with
startup_failure (Jul 15 and Jul 16), cutting no nightly release. Manual
release.yml dispatches kept working because a directly-dispatched
workflow uses its own permissions, which is why this went unnoticed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci(nightly): smoke the shipped artifacts before advancing the channel

Move the three smoke jobs and their skip-tolerant aggregator from
release.yml into nightly.yml. The workflow_call jobs are part of the
caller's run, so each smoke job still downloads the just-built artifacts
by name and runs the one shared session e2e against the shipped bytes;
what changes is what the smoke gates. In release.yml it gated the
publish step of every release; here it gates promote-nightly, so a
failed smoke leaves the nightly channel pointing at yesterday's build
instead of blocking the release machinery itself. On a no-op night
(HEAD already staged) release and the smokes both skip and the channel
flip proceeds against bytes that were smoked the night they were cut.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ci): point the smoke VMs at the shipped gvproxy explicitly

minvmd never resolves gvproxy from PATH — resolve_gvproxy_path() honours
only MINVMD_GVPROXY_BIN, the installer bin dir ($MINIMAL_BIN /
~/.local/bin), and /usr/lib/minimal/bin. Renaming the shipped binary
onto PATH therefore left both VM smokes switchless: gvproxy resolution
is best-effort, so minvmd boots without guest egress and the in-guest
pkgs clone during session mint dies with 'Could not resolve host' —
the exact 10/10 failure the Jul 16 session-e2e-soak hit for the same
reason. Set MINVMD_GVPROXY_BIN on both session-e2e steps, as the KVM
and macOS lanes do.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ci): stage gvproxy for the session-e2e soak

#760 deepened session-e2e.sh with the in-sandbox min add proof, whose
session mint clones the pkgs upstream inside the guest, and added
fetch-gvproxy + MINVMD_GVPROXY_BIN to the KVM and macOS lanes — but not
to nightly-tests' session-e2e-soak. With no gvproxy staged, minvmd
boots the VM switchless (resolution is best-effort and never errors),
the guest has no egress or DNS, and every one of the soak's 10
iterations dies with 'Could not resolve host: github.com' (first seen
on the Jul 16 scheduled run, the first after #760 merged). Stage the
pinned gvproxy and point MINVMD_GVPROXY_BIN at it, exactly as the KVM
lane does.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>