common --enable_bzlmod

test --test_output=errors --test_summary=terse

# ⚑⚑⚑ NO `--host_jvm_args=-Xmx…` RUNG HERE, AND ITS ABSENCE IS DELIBERATE. paperkit carries a

# documented climb (4G → 8G → 12G → 16G), each rung EVIDENCED by a coordinator OOM with

# `Build completed successfully` printed immediately above it. That ceiling tracks GRAPH SIZE, and

# paperkit's graph is ~150,000 actions across a mutation sweep. mtools has two distributions and

# ~120 checks. Copying a number derived from someone else's graph would be the stale-premise class

# this ecosystem keeps paying for — a figure restated away from the measurement that produced it.

# If the coordinator OOMs here, THAT is the measurement, and the rung starts one above it.

# ⚑⚑ NO `--stamp` / `workspace_status_command` YET, FOR THE SAME REASON. paperkit stamps because it

# has `toolchain`-tier checks whose verdicts are only valid against a pinned host toolchain

# (veraPDF, lualatex, soffice). mtools has exactly one host-coupled dependency — pandoc — and until

# the toolchain tier is wired the stamp would key a cache on a fingerprint nothing reads.

# ⟡mtools-toolchain-tier is where that lands.

# ⚑⚑⚑ THE REMOTE EXECUTOR IS A STRONGER SANDBOX THAN THE LOCAL ONE, AND THAT IS WHY IT IS HERE.

# BARE `linux-sandbox` shares this host's kernel, filesystem namespace and `$HOME` — which is how a

# test in this repository came to read `/home/mikemol/github/mtools/hooks/.venv/bin/ruff` from

# INSIDE a hermetic action, by calling `.resolve()` on a runfiles symlink and following it back

# out to the source tree. The sandbox permitted the escape and only a deliberate probe caught it.

#

# ⚑⚑ READ THE TENSE: THAT ESCAPE IS CLOSED, BY A FLAG ADDED FURTHER DOWN THIS SAME FILE.

# `--experimental_use_hermetic_linux_sandbox` is unconditional below, and RE-MEASURED under it the

# local sandbox no longer mounts `$HOME` at all: inside an action `Path.home()` answers

# `/home/mikemol` while that DIRECTORY DOES NOT EXIST, and the checkout is unreachable. A name

# outliving its referent — which is precisely why the present-tense sentence above kept reading as

# a live description of this configuration. It describes the configuration this file REPAIRS.

# ⚑ The KERNEL half is still true and unqualified; it is the `$HOME` half that is historical.

# //hooks:test_exec_env asserts the current state continuously, in both modes.

# An executor has no `~/github/mtools` and no venvs at all: an undeclared input is not a stale

# value there, it is a MISSING FILE, and every leak of that class fails on first run instead of

# waiting to be probed for.

#

# ⚑⚑ SO THIS IS NOT A SPEED FEATURE AND SHOULD NOT BE ADOPTED AS ONE. It is the instrument that

# PROVES the declarations are complete, where `bazel test //...` in a venv-less clone only

# supplies evidence that no further leak was found. Those are different claims.

#

# ⚑ THE ENDPOINT IS MEASURED, NOT COPIED FROM A PEER. `kubectl get endpoints -n cassian

# buildbuddy` routes :31985 to the buildbuddy-ENTERPRISE pod (Running, Ready); the original

# `buildbuddy` deployment beside it is ContainerStatusUnknown with no pod IP and is NOT in the

# endpoint list. A `buildbuddy-executor` pod is deployed alongside, which is what makes

# `--remote_executor` available rather than cache-only. Copying the port without checking which

# pod answers it would have been the stale-premise class this file already refuses twice above.

# ⚑⚑ MEASURED, ALL THREE, AGAINST THE FULL SUITE:

#

# --config=remote, cold 25 of 25 tests pass · 49 actions `remote` · 22s

# --config=remote, warm 25 of 25 tests pass · 150 action cache hits · 0 re-executed

# --config=bes streams to http://127.0.0.1:31080/invocation/<id>

#

# ⚑⚑⚑ AND THE COLD RUN IS THE HERMETICITY PROOF. An executor has no `~/github/mtools`, no venvs

# and no mise shims, so every one of those 49 actions ran with ONLY its declared inputs. Passing

# there is a different claim from passing in a venv-less clone, which shares this kernel and this

# `$HOME`: the clone shows no leak was FOUND, the executor shows none can be SUPPLIED.

#

# ⚑ THERE IS NO COLD-START PENALTY WORTH PLANNING AROUND, AND NO VENV IN THE GRAPH. `bazel aquery

# //...` reports 268 actions — 71 PyCompile, 48 TemplateExpand, 25 TestRunner, 25 SymlinkTree —

# and nothing that builds an interpreter or a venv. The Python toolchain and every wheel arrive

# through repository rules as FETCHED ARCHIVES, entering the CAS as inputs once rather than as

# build outputs. The `.venv` directories are host-side iteration only; no action reads them.

# ⚑⚑ A CONTENT-ADDRESSED DISK CACHE, BECAUSE THE EXECROOT PATH IS A HIDDEN INPUT TO EVERY ACTION

# KEY. Bazel derives its output base from a hash of the WORKSPACE PATH, so the same tree at two

# paths is two caches that share nothing. MEASURED ON THIS HOST: 11 output bases totalling 6.4G.

# A peer measured the sharper version of the same defect — two bases at 1.8G and 1.9G for ONE repo

# at two paths, and a worktree build running 129,577 actions against 4,631 hits while an identical

# tree sat fully built beside it.

#

# ⚑ IT IS UNCONDITIONAL, NOT A CONFIG, BECAUSE THE FRAGMENTATION IS UNCONDITIONAL. Every

# clean-clone probe this session created another base; that is how eleven accumulate. Namespaced

# per repository, matching both peers that carry one — a shared directory would be one cache

# keyed across unrelated action graphs.

# ⚑⚑⚑ THE HERMETIC SANDBOX IS ON BY DEFAULT, AND IT CANNOT BE A `--config`. Bazel's ordinary

# linux-sandbox SYMLINKS first-party packages back to the working tree, so an action can read the

# live checkout. MEASURED, one probe, both arms:

#

# without these flags live_venv=True source_tree=True

# with these flags live_venv=False source_tree=False

#

# `live_venv` is `/home/mikemol/github/mtools/hooks/.venv/bin/ruff` — a developer venv that no

# clone contains, reachable from inside a "hermetic" action. Ⓟ closed that escape by DISCIPLINE

# (an AST witness refusing `__file__).resolve()`); this closes it by CONSTRUCTION, and the two are

# not substitutes: the witness catches the pattern it knows, the sandbox removes the reachability.

#

# ⚑⚑ AND THESE ARE `build`, NOT `build:hermetic`, BECAUSE THE FLAGS ARE NOT IN THE ACTION KEY.

# MEASURED: a target run WITHOUT them caches a green verdict, and the same target run WITH them

# reports `Executed 0 out of 1 test: 1 test passes` — the verdict crosses the boundary unexecuted.

# So an opt-in hermeticity flag is not hermeticity; it is a regime a cache hit silently bypasses.

# A flag that changes what an action can READ but not what it is KEYED on cannot be left to the

# caller.

#

# ⚑ THE PREDICTED BREAKAGE WAS MEASURED AND DOES NOT OCCUR HERE. A peer measured this flag as

# broken for its workload, and the discriminator is whether a check needs writable scratch at a

# HOST path outside the execroot. Eight of these 25 modules touch temp paths — 90 references — but

# through pytest's `tmp_path`, which lives INSIDE the sandbox. 25 of 25 pass. Adopting on the

# peer's verdict rather than on this measurement would have been the stale-premise class; so would

# declining on it.

build --experimental_use_hermetic_linux_sandbox

build --sandbox_add_mount_pair=/bin

build --sandbox_add_mount_pair=/usr

build --sandbox_add_mount_pair=/lib

build --sandbox_add_mount_pair=/lib64

build --sandbox_add_mount_pair=/etc

# ⚑⚑⚑ THE OUTPUT ROOT AND THE DISK CACHE LIVE ON THE ZRAM AT /var/tmp, NOT ON md0 — operator

# ruling 2026-09-26, standing for every Bazel repo, relayed by luthen-observability and confirmed by

# the operator in this session. MEASURED BY LUTHEN before the change: this repository's two

# servers, bazel(mtools) and the gate's bazel(mtools-staged), were the top leaf writers on md0, at

# ~91 write ops/s and ~54 read ops/s from one session scope over 5 minutes, on a RAID6 already 94%

# busy. Both output bases and `~/.cache/bazel-disk/mtools` sat under /home, which is md0.

# ⚑ A LITERAL PATH, because .bazelrc does not expand `$USER`; it names this host's one user.

# ⚑ The disk cache MOVES rather than disappearing: the paragraph below still holds, and a cold

# output base still rebuilds from it, only now without touching the array.

startup --output_user_root=/var/tmp/bazel-mikemol

build --disk_cache=/var/tmp/bazel-disk/mtools

# ⚑⚑ BUILD WITHOUT THE BYTES IS THE DEFAULT ("declare what you need", same ruling). Outputs stay in

# the remote cache unless something asks for them. A target whose files a host process reads (the

# `.venv` the pycheck hook runs) must ask; that is a declaration, not a reason to download everything.

build --remote_download_minimal

# ⚑ THE OUTPUTS A HOST PROCESS READS, DECLARED: the pre-commit gate runs the ratchet CLI from

# `bazel-bin/ratchet/ratchet_cli.runfiles` (and refuses if that tree is absent), and EVERY

# distribution's `//<dist>:.venv` is a host artifact by design (the pycheck hook runs from hooks',

# and hooks/tests/test_venv_artifact.py runs each one's interpreter and suite). ⚑ MEASURED: a first

# cut declared only hooks' venv, and the preflight refused 45 cases across all nine distributions,

# each "has no built venv interpreter at bazel-bin/<dist>/.venv/bin/python3". Everything else stays

# remote.

build --remote_download_regex=.*/ratchet/ratchet_cli.*|.*/\.venv/.*

# ⚑⚑⚑ AND THE DISK CACHE IS WHAT MAKES A PURGE CHEAP, WHICH IS WORTH KNOWING BEFORE ONE IS ORDERED.

# An operator directive to reclaim disk had this repository stop its server and delete THREE output

# bases — 4.7G: the live one plus two the gate creates, because `.githooks/pre-commit` materialises

# the staged index with `git checkout-index` and runs the suite at a transient path that gets its

# own base. MEASURED IMMEDIATELY AFTER:

#

# bazel test //hooks:ruff -> 5.8s, "2 disk cache hit", 1 test passes

#

# A cold output base rebuilt in under six seconds because THIS line survived it. Purging output

# bases and keeping the disk cache are not in tension; the second is what makes the first free.

#

# ⚑⚑ ONE TRANSIENT FAILURE FOLLOWS A PURGE AND IS NOT A DEFECT. The first full run after refetching

# the Python toolchain failed three targets with `input dependency

# .../encodings/__pycache__/__init__.cpython-313.pyc was modified during execution` — the

# interpreter byte-compiles its freshly-fetched stdlib while parallel actions read the same tree.

# The immediately following run: 36 of 36, rc=0, no change to any file. ⚑ RE-RUN BEFORE CONCLUDING;

# a first-run race after a purge reads exactly like a hermeticity defect and is not one.

# ⚑⚑⚑ THE REMOTE CACHE AND BES ARE UNCONDITIONAL, BECAUSE AN OPT-IN CONFIGURATION IS NOT THE

# CONFIGURATION. These sat behind `--config=cache` and `--config=bes` while `.githooks/pre-commit`

# invoked a bare `bazel test //...` — so EVERY commit and every default build used neither.

# MEASURED before the change: a default `bazel test //ratchet:mypy` reported `2 linux-sandbox`

# and zero remote actions.

#

# ⚑⚑ THAT IS RULE 2'S CLASS AGAIN, ONE LAYER OUT. This file already argues that the hermetic

# sandbox flags must be `build` lines because a `--config` is a regime a cache hit silently

# bypasses. The same reasoning applies to participation itself: a shared cache nobody's default

# build reaches is a shared cache in name, and a BES stream nobody emits is an observability

# claim with no observations behind it.

#

# ⚑ AND CROSS-REPO SHARING IS THE PAYOFF THAT WAS BEING FORFEITED. An action is identified by what

# it declares, so two repositories declaring the same action have computed the same thing. Sitting

# behind a flag meant this repository contributed nothing to that store and drew nothing from it.

# ⚑⚑⚑ SERVICES BY NAME, NOT NODEPORTS — operator ruling 2026-09-20, applied here 2026-09-21. The

# four addresses below were `127.0.0.1:31985` / `:31080`, loopback NodePorts measured on 09-19

# (the §40 note above). The operator retired every loopback NodePort on the cluster the next day

# and nothing listens on them again; this file kept the written values and the gate reported a

# REFUSED sink for a BuildBuddy that was up. ⚑ A written address is read as authoritative by every

# later reader and goes stale exactly that way, so the rows below are COPIED FROM THE DIRECTORY at

# the moment of editing, and the directory — not this file — is the authority:

#

# /home/mikemol/github/luthen-observability/.venv/bin/python \

# /home/mikemol/github/luthen-observability/checks/endpoints_query.py grpc-bes --side host

# … endpoints_query.py http-web --side host # bes_results_url, append /invocation/

#

# Each answer carries an `as_of`; re-run it before trusting these lines. The names resolve on this

# workstation because a systemd dns-delegate forwards the cluster domain (luthen's disclosure, and

# `getent hosts buildbuddy-bes.buildbuddy.svc.cluster.local` → 10.43.197.218 measured here). The

# namespace is `buildbuddy`, not `cassian` as §40 says — that note is now historical and is left

# so the correction is visible. `bep_probe.py` asks the same directory at probe time.

build --remote_cache=grpc://buildbuddy-bes.buildbuddy.svc.cluster.local:1985

build --remote_cache_async=true

build --bes_backend=grpc://buildbuddy-bes.buildbuddy.svc.cluster.local:1985

build --bes_results_url=http://buildbuddy-web.buildbuddy.svc.cluster.local:8080/invocation/

build:remote --remote_executor=grpc://buildbuddy-bes.buildbuddy.svc.cluster.local:1985

build:remote --remote_instance_name=mtools-gate

# ⚑ THE EXEC PLATFORM IS DECLARED RATHER THAN INFERRED. Twenty-five targets passed remotely

# without it, which means bazel inferred one correctly — and a correct inference is still an

# undeclared assumption. //rbe records what this repository believes about the executor, and why

# its constraints are non-empty where both peers' are bare.

build:remote --extra_execution_platforms=//rbe:mtools_gate_platform

# ⚑⚑⚑ NO `--remote_local_fallback`, AND ITS ABSENCE IS A DECISION RATHER THAN AN OVERSIGHT — which

# is the difference this file already insists on twice. Two operator rulings recorded next door,

# both against it, and one repository still carries it as the outlier:

#

# · "don't do that; it evades the scheduler and consumes resources against the very same machine

# the scheduler is protecting". THE EXECUTOR IS COLOCATED ON THIS BOX, so a fallback action does

# not run somewhere else instead — it runs on the IDENTICAL CORES, outside the executor's

# accounting, at precisely the moment the queue was full BECAUSE those cores were busy. A

# control that is present, plausible, and inert exactly under load is the shape of a hook that

# fails open when its linter is absent.

#

# · a fallback "would silently re-open the escape whenever the executor is down" — and the escape

# is the whole reason this repository adopted remote execution. Degrading to it on failure

# discards the property being bought.

#

# ⚑⚑⚑ THE $HOME CLAUSE THIS LEG ONCE CARRIED IS WITHDRAWN AS STALE — TRUE OF BARE

# `linux-sandbox`, FALSE UNDER THE HERMETIC FLAG THIS FILE SETS UNCONDITIONALLY.

# It read: "a local sandbox shares this kernel and this $HOME; that is how a test here read

# the author's own venv from inside a hermetic action." MEASURED, by a probe reporting through

# the FAILURE channel — a print is captured on a pass, so the first attempt at this very

# measurement returned nothing and read as agreement:

#

# field local sandbox --config=remote

# hostname cassian buildbuddy-executor-654cd76d7f-lzb2n

# uid 1000 0

# $HOME /home/mikemol /root

# $HOME EXISTS FALSE True (the executor's own)

# checkout reachable FALSE False

#

# The local sandbox does NOT mount $HOME. `Path.home()` answers `/home/mikemol` because the

# VARIABLE is set while the DIRECTORY is absent — a name outliving its referent, which is

# exactly why the sentence read as true. The author's tree is unreachable in BOTH modes.

# ⚑⚑ THE CONCLUSION SURVIVES ON THE FIRST LEG ALONE (colocated executor, identical cores,

# outside the accounting) — a two-legged argument losing one leg, the shape commit 29240c9

# recorded one file over. Kept visible because any measurement citing $HOME-sharing as the

# reason a run was weakly hermetic cited something that did not happen.

# ⚑ THE KERNEL HALF IS UNTOUCHED and still true: a local sandbox shares this kernel.

# Asserted continuously by //hooks:test_exec_env, which is run in both modes.

#

# ⚑ SO `--config=remote` IS FAIL-CLOSED: an action routed to the executor runs THERE or fails

# loudly. The accepted cost is that it hard-depends on the executor being up, which is the honest

# trade — a gate that quietly weakens when its instrument is unavailable is the class this

# repository refuses everywhere else.

# ⚑ BES IS SEPARATE FROM BOTH, and linux-sources records its own absence of it as OWED. The same

# service exposes a `grpc-bes` port; a build invoked with `--config=bes` reports its result stream

# where every repo's runs can be compared, rather than only in this terminal.

# ⚑ `--config=bes` IS RETAINED AS A NO-OP ALIAS so an existing invocation naming it does not

# fail; the flags it once set are unconditional above.

build:bes --announce_rc

# ⚑⚑⚑ `--remote_instance_name` DOES NOT ISOLATE, AND ON THIS DEPLOYMENT IT DOES NOT PARTITION

# EITHER. A peer derived that it is a CAS addressing parameter rather than part of the action key,

# and warned that identical actions under different names are the same action with different

# storage. MEASURED HERE, and it is weaker still — after `bazel clean`, with the disk cache off:

#

# --config=remote 2 remote (re-executed)

# --config=remote --remote_instance_name=OTHER 1 REMOTE CACHE HIT

#

# A result computed under one instance name was SERVED under another. So `mtools-gate` is a label

# on the same shared store rather than a namespace.

#

# ⚑⚑⚑ AND A PEER RAN THE SAME PROBE WITH A CONTROL ARM AND GOT THE OPPOSITE RESULT. `paperkit`

# reports: warm run executes and uploads; `--remote_instance_name=WRONG` MISSES and re-executes;

# the unmodified config HITS — `bazel clean` between each arm, with the third arm existing so a

# miss cannot be confused with a system that was not caching at all. **Two probes, same stated

# controls, opposite verdicts.** Recorded rather than resolved.

#

# ⚑⚑ THEIR HYPOTHESIS FOR THE DISCREPANCY DOES NOT FIT THIS PROBE, AND SAYING SO IS NOT THE SAME AS

# BEING RIGHT. They suggest a hit without `bazel clean` between arms may be the LOCAL action cache,

# since a local hit and a remote hit are indistinguishable at the "that was fast" layer. The arms

# above did clean and did disable the disk cache, and the token read was the literal

# `REMOTE CACHE HIT` — the same discriminator they name. So the offered explanation does not cover

# this measurement, which leaves the disagreement unexplained rather than settled in either

# direction.

#

# ⚑⚑⚑ AND IT CANNOT BE RE-RUN TODAY: `ss -ltn sport = :31985` returns no listener. The endpoint

# both probes measured is down — the buildbuddy Service was deleted after the node hit disk

# pressure. **A negative about a live shared artifact carries its window, and this one's window has

# closed.** Whoever re-opens it should run BOTH arms in ONE session against a restored endpoint,

# because the two results were produced hours apart against a store that has since been evicted.

#

# ⚑ NONE OF WHICH CHANGES ANY DECISION HERE, and that is the point worth keeping. Both parties

# withdrew the FRAMING and kept the measurement, and the framing was the part that mattered:

# partitioning was never a property worth wanting, so a flag that does or does not deliver it

# changes nothing about adoption. The reconciliation is owed to the record, not to the config.

#

# ⚑⚑⚑ AND CROSS-REPO CACHE SHARING IS INTENTIONAL (operator) — this is the payoff, not a leak. An

# action is identified by WHAT IT DECLARES, so two repositories that declare the same action have

# computed the same thing and there is no reason for either to compute it twice. Content

# addressing does not know what a repository is. An earlier revision of this comment wrote the

# sharing up as a hazard to be tolerated, which had it backwards: partitioning would be discarding

# the property the CAS exists to provide.

#

# ⚑⚑⚑ AND THE CACHE CANNOT BE IMPROPER IF THE CACHE KEYS ARE PROPER (operator). That is the whole

# of it. A key is proper when the declared domain IS the real one — Π-typed — and a hit on a

# proper key is correct BY CONSTRUCTION, across repositories, machines and years alike. There is

# no residual caching hazard to hedge against, and two earlier revisions of this comment kept

# hedging: first calling the sharing a hazard, then conceding it was intentional while still

# warning about verdicts that "travel". A wrong verdict is not a cache that misbehaved. It is an

# IMPROPER KEY — the cache faithfully served a function nobody meant.

#

# ⚑⚑ SO Π-TYPING IS WHERE ALL OF THE OBLIGATION SITS, and it is owed to repositories this one

# never names. Every defect this session called a caching problem was an under-declared domain:

# a test reaching the author's venv, a witness reaching a host binary, `mypy(f)` ranging over a

# closure it did not declare. Fix the key and the cache needs no supervision at all.

#

# ⚑⚑ THE EXEC PLATFORM *DOES* PARTICIPATE, through toolchain resolution rather than as a string:

# it selects WHICH toolchain, and the resolved toolchain's files are inputs. So a remote verdict

# and a local one are not trivially interchangeable when they resolve different toolchains — which

# is also why a platform declaring NO constraints resolves nothing and fails outright.

#

# ⚑⚑⚑ AND POISONING NEEDS BOTH PARTIES' KEYS TO BE IMPROPER, IN THE SAME WAY (operator). A

# collision requires an identical key on both sides. If a producer under-declares — reads a host

# venv it never named — the key it computes describes that NARROWER action. A consumer receives

# that entry only by computing the same narrow key, which means it made the SAME omission. A

# properly-declared consumer computes a different key and never sees the bad entry.

#

# ⚑⚑ SO A CORRECT DECLARATION IS SELF-PROTECTING, and that is why a shared CAS is safe between

# repositories that do not audit each other. Being honest about this repository's domains does not

# only stop it EXPORTING a wrong verdict; it makes it unable to IMPORT one, whatever any peer does.

#

# ⚑ WHICH LOCATES THE REAL RISK PRECISELY: not under-declaration in general, but the omissions two

# parties make IDENTICALLY — the conventional ones. Everyone forgetting to declare pandoc, everyone

# reading $HOME, everyone resolving __file__ back out of the sandbox. Idiosyncratic sloppiness is

# harmless here; a SHARED HABIT is the hazard, which is an argument for writing the measured

# declaration rules down where peers can read them rather than each repository discovering them.