Skip to content

feat(adoption): instrument the distribution surface, and fix three installer bugs - #216

Open
xerj-org wants to merge 1 commit into
mainfrom
feat/adoption-instrumentation
Open

feat(adoption): instrument the distribution surface, and fix three installer bugs#216
xerj-org wants to merge 1 commit into
mainfrom
feat/adoption-instrumentation

Conversation

@xerj-org

@xerj-org xerj-org commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Why

The project cannot measure its own adoption. curl -fsSL https://xerj.org/get | sh is the documented install path, /get is served as text/plain, and curl executes no JavaScript — so no page beacon has ever been able to observe an install, and no counter has ever existed. Cloudflare RUM additionally stopped recording on 2026-07-22, so the site analytics zero is false. The one uninflated number available (145 release-asset downloads all-time, ≤~28 completed installer runs) is a running total GitHub keeps no history for, so it can be read once but never trended. The repo-level totals — 1,239 stars, 291 forks — are inflated roughly 13× by a synthetic cohort and must not be quoted anywhere.

Privacy posture, up front

No telemetry was added to the xerj binary, and none is planned. Everything here instruments the distribution surface — our own web server and the public GitHub release API. A request to a web server we operate is already observable to us; nothing new is collected from a user's machine, nothing runs on it, and nothing is sent back after an install.

The install counter records: timestamp, request.cf.country, User-Agent (capped at 200 chars), a coarse OS/arch guess parsed from that UA, which script, and a ?v= param if sent.

It records no IP address in any form, no hash of one, no cookie, no Set-Cookie, no localStorage, no stable identifier. Two installs by the same person are indistinguishable from two installs by two people, by construction. Referer, Accept-Language, and every request.cf field except country (city, postcode, ASN, lat/long — all present at that point) are discarded.

This is a deliberate divergence from functions/lead.js, which does store ip. A lead is a person who typed their email into a form; an installer request is not, and there is no consent event on the /get path to hang IP retention on. There are assertions in the test suite that fail if an IP, cookie, Referer or request.cf geo field ever reaches the stored record, and a source-level check that the function never reads cf-connecting-ip / x-forwarded-for / x-real-ip / set-cookie.

A first-run ping from the binary is the only thing that could separate installed from used. For an Apache-2.0 tool that engineers run on their own hardware it would cost more credibility than the data is worth, so that gap stays open and is documented as blind, not estimated.

What this adds

1. functions/get.js — the install counter. A Pages Function on /get and /get.ps1. It reads the script body from the existing static file via env.ASSETS, so landing/get stays the single source of truth and the served bytes are the bytes people audit. It classifies each request as installer / browser / bot / unknown from the UA, because without that crawler traffic would silently inflate the count — and an inflated number is worse than none.

Fail-open throughout. The body is resolved and the Response built before any write; the R2 put goes through context.waitUntil so it never delays a byte; missing binding, malformed binding and thrown writes are all swallowed. Cache-Control moves from public, max-age=300 to no-store for the counted response — a cached response never reaches the origin, so a cache hit is an uncounted install.

landing/_routes.json pins Function invocation to /get, /get.ps1 and /lead, claiming the route explicitly rather than relying on asset-vs-function precedence. ⚠️ A new Function added under functions/ will not be routed until its path is added to that include list.

2. .github/workflows/release-metrics.yml — the download series. Appends per-asset download_count to metrics/release-downloads.jsonl daily. It also captures the 14-day traffic windows, which GitHub expires day by day — a day not snapshotted is gone permanently. The push uses GITHUB_TOKEN, which by design triggers no workflow runs, so the daily commit does not fire CI or the 8-target cross-compile in release.yml; [skip ci] is carried in the message as belt-and-braces if a PAT is ever substituted.

3. Three installer bugs fixed — details below.

4. scripts/adoption-snapshot.sh — prints the funnel on demand in three sections: quotable, contaminated (with the reason for each), and blind.

The three installer bugs

(a) landing/get:61 made linux-gnu unreachable. The case arm set unknown-linux-gnu, then an unconditional override replaced it with unknown-linux-musl — so the installer could never request a gnu asset while the code read as though it could. All 17 linux-gnu downloads to date came from something else.

Decision: musl stays the default, because it is the right default — static, any distro including Alpine, and no glibc-version floor (a gnu binary linked against newer glibc dies at exec on an older host, which is an install failure we cannot see). The usual musl objection, its slow allocator, does not apply: xerj links jemalloc as the global allocator on every non-MSVC target (engine/crates/xerj-server/Cargo.toml:78). The case arm now says musl, the comment explains why, and XERJ_LIBC=gnu opts into the glibc build. The comment and the behaviour now agree.

(b) landing/get:101 was fail-OPEN on verification. It printed warning: sha256sum/shasum not found — skipping checksum verification and installed anyway — three lines below a message promising to refuse an unverified binary. It now tries sha256sum, shasum, openssl, sha256, busybox and cksum, validates the downloaded digest is 64 hex chars, and dies if no tool is found. XERJ_INSECURE_SKIP_CHECKSUM=1 remains as an explicit, loudly-warned user choice for a machine with no hasher at all.

(c) The download order is UNCHANGED, on purpose. The .sha256 is still fetched before the tool search, so every installer run downloads exactly one binary and one matching checksum — and that 1:1 pairing is the only usable install fingerprint in the release data. The order is now marked DO NOT REORDER with the reason, in the script and in metrics/README.md.

Verification — commands run, real output

command result
node .github/scripts/functions-get-test.mjs 64 assertions, all passing
bash .github/scripts/installer-lint.sh INSTALLER LINT PASSED
shellcheck -s sh -S warning landing/get clean (the severity CI uses)
sh -n / dash -n / busybox ash -n landing/get all clean
.github/scripts/snapshot-release-downloads.sh 88 binaries + 57 checksums = 145 — matches the audit exactly
scripts/adoption-snapshot.sh independently reproduces 95 pre-spike stars (peak day 2026-08-03, 674 stars = 54% of all)

Four real end-to-end installs of v1.0.0-rc.12 against live GitHub:

  • default → x86_64-unknown-linux-musl, sha256 verified, file reports static-pie linked
  • XERJ_LIBC=gnux86_64-unknown-linux-gnu, sha256 verified, file reports dynamically linked ← previously unreachable
  • XERJ_LIBC=boguserror: XERJ_LIBC must be 'gnu' or 'musl' (got 'bogus'), exit 1
  • PATH with no hasher → dies, 0 files installed; with XERJ_INSECURE_SKIP_CHECKSUM=1 it warns and proceeds

What is NOT verified

Cloudflare Pages Functions cannot be executed locally without wrangler, which is not available in this environment. The following are unverified and must be confirmed on the first deploy:

  • that Pages routes /get.ps1 to functions/get.ps1.js (filename-minus-.js routing)
  • that the _routes.json include list causes the Function to take /get ahead of the static asset
  • that env.ASSETS and the R2 binding resolve in production

Every one of those failing degrades to "the installer still works, we just do not count", which is the intended direction. The handler logic itself is covered by the Node test.

Deploy step required

The R2 bucket xerj-installs must be created once (Cloudflare dashboard → R2 → Create bucket). Until it exists the function serves the installer normally and logs one line instead of counting. Optionally set INSTALLS_TOKEN to enable the export:

curl -s "https://xerj.org/get?token=$INSTALLS_TOKEN&days=30" | jq .

Disclosure

My own end-to-end installer tests added roughly 4–6 asset downloads to v1.0.0-rc.12 while testing the very metric being instrumented. The seeded first line of metrics/release-downloads.jsonl includes them.

@cla-bot cla-bot Bot added the cla-signed label Aug 8, 2026
…staller bugs

The project was flying blind on adoption. The documented install path is
`curl -fsSL https://xerj.org/get | sh`; /get is served as text/plain and curl
executes no JavaScript, so no page beacon could ever observe it. There has
never been an install counter of any kind. Cloudflare RUM additionally stopped
recording on 2026-07-22, so site traffic reads a false zero. The only
uninflated number available — 145 release-asset downloads all-time, of which
at most ~28 are completed installer runs — is a running total that GitHub
keeps no history for, so it could be read once but never trended. Meanwhile
the repo-level totals (1,239 stars, 291 forks) are inflated ~13x by a
synthetic cohort and must not be quoted.

What this adds instruments the DISTRIBUTION surface only — the web server and
the release API. It adds nothing to the xerj binary.

  NO TELEMETRY WAS ADDED TO THE ENGINE, AND NONE IS PLANNED.

A first-run ping is the only thing that could separate "installed" from
"used", and for an Apache-2.0 tool that engineers run on their own hardware it
would cost more credibility than the data is worth. That gap stays open on
purpose and is documented as blind rather than estimated.

── 1. functions/get.js — the install counter ────────────────────────────────

A Pages Function on /get and /get.ps1. It reads the script body out of the
existing static file via env.ASSETS (so landing/get remains the single source
of truth and the served bytes are the bytes people audit), counts the request,
and returns the script.

Recorded: timestamp, request.cf.country, User-Agent (capped at 200 chars), a
coarse OS/arch guess parsed from that UA, which script, and a ?v= param if
sent. Also a `kind` class — installer / browser / bot / unknown — because
without it crawler traffic would silently inflate the number, and an inflated
number is worse than none.

NOT recorded: no IP address in any form, no hash of one, no cookie, no
localStorage, no stable identifier. Two installs by one person are
indistinguishable from two installs by two people, by construction. Referer,
Accept-Language, and every request.cf field except country (city, postcode,
ASN, lat/long) are available at that point and are all discarded.

This is a deliberate divergence from functions/lead.js, which does store `ip`:
a lead is a person who typed their email into a form, an installer request is
not, and there is no consent event on the /get path to hang IP retention on.

Fail-open throughout. The body is resolved and the Response built before any
write; the R2 put goes through context.waitUntil so it never delays a byte;
missing binding, malformed binding and thrown writes are all swallowed. An
installer that breaks because a counter failed would be far worse than no
counter. The Cache-Control changes from `public, max-age=300` to `no-store`
for the counted response, because a cached response never reaches the origin
and would undercount by an unknowable amount.

One record per request (no read-modify-write, so concurrent installs cannot
lose each other), date-prefixed, aggregated at read time by a token-guarded
export that ships its own caveat text.

landing/_routes.json pins Function invocation to /get, /get.ps1 and /lead, so
the route is claimed explicitly rather than depending on asset-vs-function
precedence — and every other request skips Functions entirely.

── 2. .github/workflows/release-metrics.yml — the download series ───────────

Daily snapshot of per-asset download_count into metrics/release-downloads.jsonl.
GitHub keeps no history for that counter; one committed line a day turns a
scalar into a trend at zero infrastructure cost. It also captures the 14-day
traffic windows, which GitHub expires day by day — a day not snapshotted is
gone permanently.

The commit is pushed with GITHUB_TOKEN, which by design triggers no workflow
runs, so the daily commit does NOT fire CI or the 8-target cross-compile in
release.yml. [skip ci] is carried in the message as belt-and-braces in case a
PAT is ever substituted.

── 3. three installer bugs ─────────────────────────────────────────────────

a) landing/get:61 unconditionally overrode the target with unknown-linux-musl
   after the case arm had set unknown-linux-gnu, so the installer could never
   request a linux-gnu asset — while the code read as though it could. All 17
   linux-gnu downloads to date came from something else.

   Decision: musl stays the default, because it is the right one — static, any
   distro including Alpine, no glibc-version floor (a gnu binary linked against
   newer glibc dies at exec on an older host, an install failure we cannot
   see), and musl's slow allocator is not in the path because xerj links
   jemalloc on every non-MSVC target. The case arm now says musl, the comment
   explains why, and XERJ_LIBC=gnu opts into the glibc build. Verified both
   ways against the live rc.12 release: default installs a static-pie musl
   binary, XERJ_LIBC=gnu installs a dynamically linked glibc one.

b) landing/get:101 was fail-OPEN on checksum verification — it printed
   "warning: sha256sum/shasum not found — skipping checksum verification" and
   installed anyway, three lines below a message promising to refuse an
   unverified binary. It now tries sha256sum, shasum, openssl, sha256, busybox
   and cksum, validates the downloaded digest is 64 hex chars, and dies if no
   tool is found. XERJ_INSECURE_SKIP_CHECKSUM=1 remains as an explicit,
   loudly-warned user choice for a box with no hasher at all.

c) The .sha256 is still fetched BEFORE the tool search, unchanged and on
   purpose. That order is what makes every installer run download exactly one
   binary and one matching checksum, and that 1:1 pairing is the only usable
   install fingerprint in the release data. The order is now marked DO NOT
   REORDER with the reason, in the script and in metrics/README.md.

── 4. scripts/adoption-snapshot.sh ─────────────────────────────────────────

Prints the funnel on demand in three sections: quotable, contaminated (with
the reason for each), and blind. It computes the defensible pre-spike star
count live rather than making anyone remember it — independently reproducing
the audit's figure of 95 stars before 2026-08-03, against 1,239 raw.

── verification ────────────────────────────────────────────────────────────

  node .github/scripts/functions-get-test.mjs      64 assertions, all passing
  bash .github/scripts/installer-lint.sh           PASS
  shellcheck -s sh -S warning landing/get          clean (CI severity)
  sh -n / dash -n / busybox ash -n landing/get     all clean
  real install of v1.0.0-rc.12, 4 paths            musl default, XERJ_LIBC=gnu,
                                                   XERJ_LIBC=bogus rejected, and
                                                   no-hasher fail-closed (nothing
                                                   installed) + opt-out proceeds
  .github/scripts/snapshot-release-downloads.sh    88 binaries + 57 checksums
                                                   = 145, matching the audit

Cloudflare Pages Functions cannot be executed locally without wrangler, which
is not available here: the routing (/get.ps1 from functions/get.ps1.js, the
_routes.json include list, and env.ASSETS/R2 binding resolution) is unverified
and must be confirmed on the first deploy. The handler logic itself is covered
by the Node test above. The R2 bucket `xerj-installs` must be created once in
the dashboard; until it exists the counter is a no-op and installs are
unaffected.
@xerj-org
xerj-org force-pushed the feat/adoption-instrumentation branch from 1b3cef9 to 8184495 Compare August 8, 2026 06:17
@xerj-org xerj-org closed this Aug 8, 2026
@xerj-org xerj-org reopened this Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant