From afefc1ebbf5a7d770bf175167fd45291c3943f08 Mon Sep 17 00:00:00 2001 From: hmmhmmhm Date: Mon, 13 Jul 2026 20:19:00 +0900 Subject: [PATCH 01/78] docs: design security operations and R2 hardening --- ...ecurity-operations-data-delivery-design.md | 340 ++++++++++++++++++ 1 file changed, 340 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-13-security-operations-data-delivery-design.md diff --git a/docs/superpowers/specs/2026-07-13-security-operations-data-delivery-design.md b/docs/superpowers/specs/2026-07-13-security-operations-data-delivery-design.md new file mode 100644 index 00000000..7ce09317 --- /dev/null +++ b/docs/superpowers/specs/2026-07-13-security-operations-data-delivery-design.md @@ -0,0 +1,340 @@ +# Security, Operations, and Data Delivery Hardening Design + +**Date:** 2026-07-13 + +## 1. Objective + +Complete the next hardening cycle for `ground.codes` without weakening the +behavior validated by PR #66. The work must: + +- remove known high-severity production dependency advisories; +- fix invalid Worker-isolate start time and uptime metrics; +- enable repository security controls and protect `main`; +- reduce recurring production-smoke checkout and runner cost; +- make production deployments reproducible and serialization-safe; +- add enforceable test-coverage gates for critical maintained code; +- move the 8.35 GiB region dataset out of the current Git tree and into + versioned Cloudflare R2 objects without rewriting Git history; and +- finish with green CI, deployments, and post-deploy production smoke on the + merged `main` commit. + +This is a staged program, not one indivisible change. Each stage must be +independently reversible and must leave production healthy. + +## 2. Current-State Evidence + +The design responds to observed state on merged commit +`827847ddf42f5432f104b40eaf3ec9791d5e0581`: + +- `pnpm audit --prod` reports 4 high, 14 moderate, and 3 low advisories. + Direct runtime dependencies include Elysia 1.2.25, `@elysiajs/cors` 1.2.0, + and next-intl 4.0.2. +- `https://api.ground.codes/metrics` reports + `startedAt: "1970-01-01T00:00:00.000Z"` and an uptime of roughly 56 years. +- The repository has no branch protection or repository rulesets. Dependabot + security updates, secret scanning, and push protection are disabled. +- `MOSHI_WEBHOOK_TOKEN` is absent from GitHub Actions secrets, so smoke + failures fall back to issue comments instead of push notification. +- the scheduled production smoke checks out the full repository. A recent run + spent 58 seconds checking out the repository and 49 seconds executing the + smoke checks. +- `packages/geoint/region-dist` contains 901 files totaling 7.56 GiB and + `packages/geoint/region-db` contains 1,130 files totaling 0.79 GiB. +- Web and Grok Spiral deployments have no concurrency group. Wrangler and Bun + are installed from moving `latest` references. +- The repository has 58 test files but no code-coverage command or threshold. + +## 3. Delivery Strategy + +Use four ordered delivery stages: + +1. **Security and repository governance** +2. **Metrics, smoke, and observability** +3. **CI, deployment reproducibility, and coverage** +4. **R2 data publication and Git-tree cutover** + +The stages may live on one long-running feature branch, but commits and pull +request descriptions must preserve these boundaries. Security and the metrics +bug are completed before the longer data migration. + +## 4. Security and Repository Governance + +### 4.1 Dependency remediation + +- Replace moving runtime dependency specifications such as `"latest"` with + reviewed explicit versions. +- Upgrade Elysia to at least 1.4.27 and use compatible current releases of the + Elysia CORS, static, and Swagger plugins. +- Upgrade next-intl to at least 4.9.2. +- Upgrade Cesium or apply reviewed lockfile overrides so its protobufjs and + DOMPurify dependency graph has no known high-severity advisory. +- Patch or override picomatch and PostCSS to fixed release lines. +- Keep the frozen lockfile authoritative in every CI and deploy workflow. +- Add `pnpm audit --prod --audit-level high` to CI. Completion requires zero + critical and zero high advisories. Moderate advisories must either be fixed + or documented with applicability and a follow-up issue. + +Dependency upgrades are separated into compatible groups and tested after each +group. Major framework upgrades such as Next 16, React 19.2, TypeScript 7, or +ESLint 10 are not required unless needed to remove a security advisory. + +### 4.2 Automated security maintenance + +- Add `.github/dependabot.yml` for weekly grouped npm and GitHub Actions + updates. +- Enable Dependabot security updates. +- Enable secret scanning, non-provider pattern scanning, validity checks, and + push protection where GitHub exposes them for the public repository. +- Pin GitHub Actions to full commit SHAs with an adjacent release comment. +- Store `MOSHI_WEBHOOK_TOKEN` as an Actions secret through stdin; never place + the value in a file, command output, workflow, commit, or pull request. + +### 4.3 Main-branch rules + +After the feature PR has a stable required check name, create an active +repository ruleset for `main` that: + +- requires changes to arrive through a pull request; +- requires the `verify` CI status check; +- requires branches to be up to date before merge; +- blocks force pushes and branch deletion; +- does not require another human approval, because this is currently a solo + repository; and +- permits administrator bypass only for an emergency recovery. + +Enable automatic deletion of merged head branches. Capture the resulting +ruleset ID and settings in the final verification report. + +## 5. Metrics and Observability + +### 5.1 Worker start-time fix + +Cloudflare may evaluate wall-clock time at module initialization as the Unix +epoch. The metrics module therefore stores no eager `new Date()` value. + +- Keep `startedAt` unset until the first incoming request reaches `onRequest`. +- Initialize it once from request-time wall-clock state. +- Serialize a valid ISO timestamp and calculate non-negative uptime from the + same initialized value. +- Keep the current Worker-isolate scope explicit. Empty route/path counters + remain valid because the metrics request can reach a fresh isolate. + +### 5.2 Metrics contract validation + +Production smoke and unit tests validate: + +- `startedAt` is a valid ISO timestamp later than 2020-01-01; +- it is not materially in the future; +- `uptimeSeconds` agrees with `startedAt` within a small clock/rounding + tolerance; +- top-level totals and every present route, status, and path counter are finite + non-negative numbers; and +- the service name, scope, and deployed commit remain observable. + +### 5.3 Structured operational signals + +- Emit one structured JSON record for API request completion containing route, + method, status, duration, runtime commit, and service name. Do not include + coordinates, search strings, codes, IP addresses, or credentials. +- Document initial service objectives: + - readiness and web-root monthly availability target: 99.9%; + - representative encode/search smoke latency target: under 2 seconds; + - post-deploy full-smoke success required before an incident is closed. +- Keep Cloudflare observability enabled and document the dashboard queries and + rollback/runbook links used during an incident. + +## 6. Production-Smoke Design + +Split smoke execution into two profiles while retaining one shared check +implementation: + +- **quick:** readiness, web root/robots/sitemap, metrics, representative Earth + encode/search/decode, one non-Latin language, and Moon/Mars encode; +- **full:** every current language and all operational/error-path checks. + +Rules: + +- post-Web deploy and manual runs execute the full profile; +- the 30-minute schedule executes quick checks; +- a daily schedule executes full checks; +- full remains the default when the profile is omitted; +- the workflow sparse-checks out only `scripts/production-smoke*.mjs` and uses + shallow history; +- workflow concurrency cancels an older scheduled smoke when a newer scheduled + smoke starts, but never cancels a post-deploy full check; and +- notification failure does not hide the original smoke failure. + +The workflow-contract tests assert triggers, profiles, sparse checkout, +concurrency, notification behavior, and the complete 180-language full +coverage. + +## 7. CI, Deployment, and Coverage + +### 7.1 Reproducible workflows + +- Use `pnpm install --frozen-lockfile` everywhere. +- Add Wrangler as an exact root development dependency and invoke it through + pnpm instead of installing a global moving version. +- Pin the Bun runtime to one verified version rather than `latest`. +- Add production concurrency groups to Web and Grok Spiral deployments. + Superseded queued or in-progress deploys are cancelled so an older commit + cannot finish after a newer commit. +- Preserve the API deployment's serialized, non-cancelling database import and + Worker deployment behavior. +- Add `environment: production` to deployment jobs so secrets and deployment + history are scoped consistently. No manual approval is required by default. + +### 7.2 Coverage gates + +Coverage applies to maintained executable code, not generated language/data +tables, declarations, Next build output, or third-party assets. + +- Add coverage commands for `packages/ground-codes`, `apps/api-ground-codes`, + and `apps/web` critical libraries/hooks. +- Enforce at least 80% line coverage and 80% function coverage for each scoped + target. Branch coverage starts at the measured baseline and may only rise. +- Add focused tests where the initial baseline is below the required line or + function threshold; do not lower the target to make CI pass. +- Include metrics initialization, R2 manifest verification, smoke profile + selection, dependency audit policy, and deployment workflow contracts in the + tested surface. + +## 8. R2 Region-Data Architecture + +### 8.1 Object model + +Create the write-private bucket `ground-codes-region-data`, expose only its +immutable dataset objects through a public read endpoint, and use this release +prefix: + +```text +releases//manifest.json +releases//objects/.json.gz +``` + +The version is a content-derived identifier. Each manifest entry contains: + +- repository-relative logical path; +- uncompressed byte size; +- compressed byte size; +- SHA-256 of uncompressed content; +- R2 object key; and +- dataset group (`region-dist` or `region-db`). + +Identical content is stored once by hash. A small committed manifest pointer +declares the active dataset version. Tiny deterministic test fixtures stay in +Git. + +### 8.2 Access and credentials + +- Publication uses dedicated least-privilege R2 S3 credentials stored as + `R2_REGION_DATA_ACCESS_KEY_ID` and `R2_REGION_DATA_SECRET_ACCESS_KEY`, plus + `CLOUDFLARE_ACCOUNT_ID`. +- CI and public pull requests must not receive write credentials. +- Dataset objects are publicly readable because the same data is already + public in Git. A repository variable `REGION_DATA_BASE_URL` supplies the + read-only R2 custom-domain or R2.dev base URL. +- The publisher writes immutable hash keys first and the release manifest last. + It never overwrites a published release. + +### 8.3 Synchronizer + +Add a maintained Node synchronizer that: + +- downloads only missing or hash-mismatched logical files; +- supports `region-dist`, `region-db`, and explicit-path scopes; +- limits concurrent requests and retries transient failures with backoff; +- decompresses into a temporary file, verifies uncompressed size and SHA-256, + then atomically renames into place; +- removes neither unrelated files nor user changes by default; +- has a `--prune` mode used only by CI for exact materialization; and +- fails closed on manifest, network, decompression, or hash errors. + +No consumer is allowed to silently fall back to incomplete data. + +### 8.4 Two-phase cutover + +**Phase A: publish and shadow-verify** + +1. Generate the manifest for the current Git-tracked data. +2. Upload immutable compressed objects and manifest to R2. +3. Materialize a clean temporary directory from R2 and compare every path, + size, and hash to Git. +4. Modify CI/deploy workflows to materialize their required scopes from R2. +5. Keep Git data present for one successful PR CI, merged-main CI, API deploy, + Web deploy, and post-deploy full smoke. + +**Phase B: remove from the current tree** + +1. Delete `packages/geoint/region-dist` and `packages/geoint/region-db` from the + current Git tree, retaining fixtures and manifests. +2. Update generators, audits, local setup, CI, and deploy workflows to call the + synchronizer explicitly. +3. Verify a fresh sparse clone can materialize data, run full tests, build, and + deploy. +4. Merge only after the R2-only path passes every authoritative check. + +Git history is not rewritten. Existing commits remain a recovery source. + +### 8.5 Future data publication + +Data changes run through a manual or protected workflow that: + +1. materializes the active R2 release; +2. runs the existing generator and quality audits; +3. publishes a new immutable release; +4. opens or updates a pull request containing only the manifest pointer, + documentation, and any source/generator changes; and +5. leaves the previous release available for rollback. + +## 9. Failure Handling and Rollback + +- Dependency upgrades are reverted as a group if focused tests or production + smoke regress. +- Metrics initialization never throws; an uninitialized state is initialized + at request time and tested directly. +- Quick-smoke failure opens or updates one incident issue and sends the Moshi + notification when configured. +- Deployment workflows expose the commit SHA and keep Cloudflare's immutable + deployment history. Rollback redeploys the last known-good SHA. +- R2 publication is append-only. The active manifest pointer changes only in + Git, so rollback is a normal pull request restoring the prior version. +- R2 synchronization failure stops CI/deployment before tests or mutation of + production infrastructure. +- The Git-data deletion is never merged until R2-only verification passes from + a fresh checkout. + +## 10. Verification and Completion Criteria + +The program is complete only when all of the following evidence exists for the +same merged `main` commit: + +1. `pnpm audit --prod --audit-level high` exits successfully with zero high or + critical advisories. +2. Format, source-size, lint, type, operational-script, data-quality, unit, + coverage, build, and browser-smoke CI gates pass from a full authoritative + environment. +3. Production metrics show a realistic ISO `startedAt`, consistent uptime, + valid counters, and the merged runtime commit. +4. Quick and full smoke profiles both pass, and a scheduled run demonstrates + the sparse checkout path. +5. API, Web, and Grok Spiral production deployments succeed. +6. `main` ruleset, Dependabot security updates, secret scanning, push + protection, merged-branch deletion, and required Actions secrets are + verified through GitHub's APIs without exposing secret values. +7. R2 manifest publication and a clean R2 materialization produce every + expected file with matching SHA-256. +8. `region-dist` and `region-db` are absent from the current Git tree except for + explicitly declared fixtures/manifests, and a fresh checkout passes the + full R2-only CI/deploy path. +9. The post-deploy full production smoke passes and any issue opened by a + regression is resolved with links to the successful runs. + +## 11. Out of Scope + +- Rewriting existing Git history or force-pushing `main`. +- Migrating away from Cloudflare Pages/Workers or Supabase/PostGIS. +- Unrelated Next, React, TypeScript, ESLint, or UI feature rewrites. +- Requiring a second human reviewer in a single-maintainer repository. +- Making private data public; only the already-public region datasets are + eligible for the public read endpoint. From 437fa5b1d411044c40918199d33651a358b5fb56 Mon Sep 17 00:00:00 2001 From: hmmhmmhm Date: Mon, 13 Jul 2026 22:10:13 +0900 Subject: [PATCH 02/78] docs: plan security operations and R2 delivery --- .../plans/2026-07-13-ci-deploy-coverage.md | 238 ++++++++++++++ .../2026-07-13-metrics-smoke-observability.md | 224 +++++++++++++ .../2026-07-13-r2-region-data-delivery.md | 299 ++++++++++++++++++ .../plans/2026-07-13-security-governance.md | 220 +++++++++++++ 4 files changed, 981 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-13-ci-deploy-coverage.md create mode 100644 docs/superpowers/plans/2026-07-13-metrics-smoke-observability.md create mode 100644 docs/superpowers/plans/2026-07-13-r2-region-data-delivery.md create mode 100644 docs/superpowers/plans/2026-07-13-security-governance.md diff --git a/docs/superpowers/plans/2026-07-13-ci-deploy-coverage.md b/docs/superpowers/plans/2026-07-13-ci-deploy-coverage.md new file mode 100644 index 00000000..869ec40c --- /dev/null +++ b/docs/superpowers/plans/2026-07-13-ci-deploy-coverage.md @@ -0,0 +1,238 @@ +# CI, Deployment, and Coverage Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make every automated install and production deploy reproducible, prevent stale Web/Grok deploys from winning races, and enforce measured coverage floors for critical maintained code. + +**Architecture:** Tool versions live in the workspace lockfile and package manifests; workflows invoke local binaries only. Coverage producers emit LCOV into one root directory, and a pure policy checker aggregates only declared maintained source files per target. Line/function floors are fixed at 80%; branch floors are captured from the first green baseline and may only rise. + +**Tech Stack:** pnpm/Turborepo, Bun 1.3.1, Node.js, c8 11, LCOV, GitHub Actions, Wrangler 4.110.0, Cloudflare Pages/Workers. + +**Execution Order:** Run after the security and metrics/smoke plans and finish it before R2 Phase A. The R2 plan explicitly extends the established operations coverage target with its new modules. + +--- + +### Task 1: Pin workflow runtimes and the local Cloudflare CLI + +**Files:** + +- Modify: `package.json` +- Modify: `pnpm-lock.yaml` +- Modify: `.github/workflows/ci.yml` +- Modify: `.github/workflows/deploy-api.yml` +- Modify: `.github/workflows/deploy-web.yml` +- Modify: `.github/workflows/deploy-grok-spiral.yml` +- Modify: `.github/workflows/visual-qa.yml` +- Modify: `scripts/qa-workflows.test.mjs` + +- [ ] Extend workflow tests to require `bun-version: "1.3.1"`, frozen installs in every workflow that installs packages, no `pnpm install -g`, and deployment commands that invoke `pnpm exec wrangler` or a package script backed by it. +- [ ] Run `node --test scripts/qa-workflows.test.mjs`; expect RED for Bun `latest`, unfrozen Web/Grok installs, and global Wrangler installation. +- [ ] Add exact root development dependencies: + + ```sh + pnpm add -Dw --save-exact wrangler@4.110.0 c8@11.0.0 + ``` + +- [ ] Change every Bun setup to `bun-version: "1.3.1"` and every workflow install to `pnpm install --frozen-lockfile`. +- [ ] Remove global Wrangler steps. Change app deploy scripts to run `pnpm exec wrangler pages deploy`, and API deployment to run `pnpm exec wrangler deploy --config apps/api-ground-codes/wrangler.toml ...` from the repository root. +- [ ] Run `pnpm install --frozen-lockfile`, `pnpm exec wrangler --version`, and the workflow tests; expect Wrangler `4.110.0` and PASS. +- [ ] Commit as `chore(ci): pin bun and wrangler toolchain`. + +### Task 2: Serialize production deployment outcomes correctly + +**Files:** + +- Modify: `.github/workflows/deploy-api.yml` +- Modify: `.github/workflows/deploy-web.yml` +- Modify: `.github/workflows/deploy-grok-spiral.yml` +- Modify: `scripts/qa-workflows.test.mjs` + +- [ ] Add contract assertions for named production environments and exact concurrency semantics: Web/Grok cancel superseded work, API remains serialized without cancellation. +- [ ] Run workflow tests; expect RED because Web/Grok lack concurrency and jobs lack environments. +- [ ] Add these workflow-level blocks: + + ```yaml + # deploy-web.yml + concurrency: + group: deploy-web-production + cancel-in-progress: true + + # deploy-grok-spiral.yml + concurrency: + group: deploy-grok-spiral-production + cancel-in-progress: true + ``` + +- [ ] Preserve API's `deploy-api-production` group with `cancel-in-progress: false` so a database import/deploy is never interrupted. +- [ ] Add `environment: production` to all three deployment jobs. Do not add a reviewer/approval gate to the environment. +- [ ] Run workflow tests; expect PASS. +- [ ] Commit as `ci(deploy): enforce production concurrency`. + +### Task 3: Build an LCOV policy checker test-first + +**Files:** + +- Create: `scripts/coverage-policy.mjs` +- Create: `scripts/coverage-policy.test.mjs` +- Create: `scripts/check-coverage.mjs` +- Modify: `package.json` + +- [ ] Write temporary LCOV fixture tests for line, function, and branch aggregation; missing declared files; duplicate records; excluded generated files; exactly-at-threshold PASS; and below-threshold RED. +- [ ] Use a temporary fixture policy with four targets so tests prove failures are reported per target rather than hidden by a repository-wide average. +- [ ] Run `node --test scripts/coverage-policy.test.mjs`; expect RED because the parser/checker does not exist. +- [ ] Implement `parseLcov`, `collectTargetCoverage`, and `evaluateCoveragePolicy`. Normalize source paths relative to repository root and require every explicitly included file/glob to be represented. +- [ ] Define and test the policy schema without creating the real repository policy yet: + + ```json + { + "schemaVersion": 1, + "targets": { + "ground-codes": { + "lcov": "coverage/ground-codes/lcov.info", + "include": ["packages/ground-codes/src/**/*.ts"], + "exclude": ["**/*.d.ts"], + "minimum": { "line": 0.8, "function": 0.8, "branch": 0.731 } + } + } + } + ``` + + The `0.731` branch value is fixture data used only by the unit test. Task 7 creates the real policy from measured reports. + +- [ ] Make `check-coverage.mjs` print one compact line per target and exit non-zero for a missing report/file or failed metric. +- [ ] Add `coverage:check` as `node scripts/check-coverage.mjs`. +- [ ] Run the policy tests; expect PASS. +- [ ] Commit as `test(coverage): define lcov policy enforcement`. + +### Task 4: Produce coverage for the Ground Codes core + +**Files:** + +- Modify: `packages/ground-codes/package.json` +- Modify: `package.json` +- Modify: `packages/ground-codes/test/region-dataset-resolver.test.ts` +- Modify: `packages/ground-codes/test/region-store.test.ts` +- Modify: `packages/ground-codes/test/region-fallback.test.ts` +- Modify: `packages/ground-codes/test/spiral-fixture.test.ts` + +- [ ] Add a `test:coverage` command that uses the exact root c8 binary, source maps, `--all`, `--include src/**/*.ts`, `--exclude src/**/*.d.ts`, and LCOV output at `coverage/ground-codes`. +- [ ] Add the root `coverage:ground-codes` command to invoke that package script without Turbo caching. +- [ ] Run `pnpm coverage:ground-codes`; expect `coverage/ground-codes/lcov.info` and a readable text summary. +- [ ] Add focused tests for any untested resolver/cache/error branches until maintained core line and function coverage each reach at least 80%. Do not remove files from the target to make the result pass. +- [ ] Run the core unit and standalone package tests after coverage; expect PASS. +- [ ] Commit as `test(ground-codes): add maintained-source coverage`. + +### Task 5: Produce coverage for API and critical Web code + +**Files:** + +- Modify: `apps/api-ground-codes/package.json` +- Modify: `apps/web/package.json` +- Modify: `package.json` +- Modify: `apps/api-ground-codes/src/app.test.ts` +- Modify: `apps/api-ground-codes/src/app-part-2.test.ts` +- Modify: `apps/api-ground-codes/src/app-part-3.test.ts` +- Modify: `apps/api-ground-codes/src/app-part-4.test.ts` +- Modify: `apps/api-ground-codes/src/metrics.test.ts` +- Modify: `apps/web/lib/code/ground-codes.test.ts` +- Modify: `apps/web/lib/code/share-url.test.ts` +- Modify: `apps/web/lib/i18n/ground-code-language.test.ts` +- Modify: `apps/web/lib/map/celestial-bodies.test.ts` +- Modify: `apps/web/lib/map/google-maps-availability.test.ts` +- Modify: `apps/web/hooks/use-disable-zoom.test.ts` + +- [ ] Add Bun LCOV commands: + + ```json + "test:coverage": "bun test --coverage --coverage-reporter=lcov --coverage-dir=../../coverage/api src/*.test.ts" + ``` + + for the API, and the corresponding Web command targeting `./lib ./app ./components ./hooks` with output `../../coverage/web`. + +- [ ] Declare all maintained API `src/**/*.ts` files except `*.test.ts`, declarations, and generated OpenAPI/build output as the API target. +- [ ] Declare these critical Web sources as the minimum Web target, expanding rather than shrinking the list when another critical pure library/hook is identified: + + ```text + apps/web/lib/code/ground-codes.ts + apps/web/lib/code/share-url.ts + apps/web/lib/i18n/ground-code-language.ts + apps/web/lib/map/celestial-bodies.ts + apps/web/lib/map/google-maps-availability.ts + apps/web/hooks/use-disable-zoom.ts + ``` + +- [ ] Run both coverage producers; expect LCOV reports. Add focused tests for metrics initialization/logging, validation errors, Web URL/code behavior, locale resolution, body selection, map availability, and zoom cleanup until line and function coverage are at least 80% for each target. +- [ ] Ensure the policy excludes test files, declarations, `.next`, `.vercel`, generated language/data tables, and third-party assets—never ordinary maintained runtime modules. +- [ ] Run API/Web unit and type tests; expect PASS. +- [ ] Commit as `test(apps): add api and web coverage`. + +### Task 6: Cover critical operational policy code + +**Files:** + +- Modify: `package.json` +- Modify: `scripts/production-audit-policy.test.mjs` +- Modify: `scripts/production-smoke.test.mjs` +- Modify: `scripts/qa-workflows.test.mjs` +- Modify: `scripts/github-governance.test.mjs` + +- [ ] Add `coverage:operations` using c8/Node test with LCOV output at `coverage/operations`. Include exactly the security-audit policy, smoke helpers/profile selection, workflow/governance policy, and coverage policy modules; exclude CLI entry points whose only behavior is process wiring. +- [ ] Run the operations coverage producer; expect every declared module in LCOV and all existing contract tests PASS. +- [ ] Add focused error/edge cases until line and function coverage for this target are each at least 80%. This makes metrics, smoke profiles, audit policy, and workflow contracts part of the authoritative covered surface; the R2 plan later adds manifest/synchronizer/publisher/verification modules to the same target. +- [ ] Commit as `test(operations): cover automation policy modules`. + +### Task 7: Record and lock the branch baselines + +**Files:** + +- Modify: `scripts/check-coverage.mjs` +- Create: `scripts/coverage-policy.json` +- Create: `docs/quality/coverage.md` + +- [ ] Add a read-only `--report-baseline` mode that prints exact line/function/branch ratios from the four LCOV reports but never writes policy. +- [ ] Run all four coverage commands followed by `node scripts/check-coverage.mjs --report-baseline`; expect finite branch ratios for every target. +- [ ] Set each target's committed branch minimum to its measured ratio rounded down by less than 0.001, while keeping line/function exactly `0.8`. If a producer emits no branch records, add/replace instrumentation rather than committing a zero branch floor. +- [ ] Document the measured values, date, Bun/c8 versions, included source boundary, and the rule that thresholds may only increase in `docs/quality/coverage.md`. +- [ ] Run `pnpm coverage:check`; expect all four targets PASS. +- [ ] Commit as `test(coverage): lock measured branch baselines`. + +### Task 8: Add the authoritative coverage command and CI gate + +**Files:** + +- Create: `scripts/run-coverage.mjs` +- Create: `scripts/run-coverage.test.mjs` +- Modify: `package.json` +- Modify: `.github/workflows/ci.yml` +- Modify: `turbo.json` +- Modify: `scripts/qa-workflows.test.mjs` + +- [ ] Test that the orchestrator executes the four producers in a fixed order, stops on producer failure, removes stale target reports first, and runs the policy checker last. +- [ ] Implement root `coverage` as `node scripts/run-coverage.mjs`; do not reuse stale LCOV or allow Turbo cache to stand in for execution. +- [ ] Add `coverage/**` to appropriate ignored build outputs while keeping reports out of Git. +- [ ] Add a named `Enforce coverage policy` CI step after unit tests and before build. Assert it in workflow tests. +- [ ] Run `pnpm coverage`, `pnpm scripts:test`, and `node --test scripts/qa-workflows.test.mjs`; expect PASS. +- [ ] Commit as `ci: enforce coverage policy`. + +### Task 9: Verify reproducible deploy commands without publishing + +**Files:** + +- Modify: `docs/operations/incident-runbook.md` + +- [ ] Run `pnpm install --frozen-lockfile` twice; expect no manifest or lockfile changes. +- [ ] Run `pnpm --filter web pages:build`, `pnpm --filter grok-spiral pages:build`, and `pnpm --filter api-ground-codes build`; expect PASS using only lockfile-resolved tools. +- [ ] Run `pnpm exec wrangler deploy --config apps/api-ground-codes/wrangler.toml --dry-run --outdir /tmp/ground-codes-worker-dry-run`; expect a Worker bundle without production mutation. +- [ ] Document the local exact-version verification and deployment rollback commands. +- [ ] Commit as `docs(deploy): record reproducible release commands`. + +### Task 10: Complete the CI/deploy/coverage stage gate + +**Files:** + +- Modify: this plan + +- [ ] Run `pnpm format:check`, `pnpm code:size-check`, `pnpm security:audit`, `pnpm scripts:test`, `pnpm lint`, `pnpm check-types`, `pnpm coverage`, and `pnpm build`; expect PASS. +- [ ] Confirm `git grep -n 'pnpm install -g\|bun-version: latest\|pnpm install$' .github/workflows` returns no matches. +- [ ] Confirm workflow tests prove Web/Grok cancellation, API non-cancellation, production environments, frozen installs, pinned actions, and local Wrangler use. +- [ ] Mark all checkboxes and commit as `docs: complete ci deployment coverage plan`. diff --git a/docs/superpowers/plans/2026-07-13-metrics-smoke-observability.md b/docs/superpowers/plans/2026-07-13-metrics-smoke-observability.md new file mode 100644 index 00000000..96d0966b --- /dev/null +++ b/docs/superpowers/plans/2026-07-13-metrics-smoke-observability.md @@ -0,0 +1,224 @@ +# Metrics, Smoke, and Observability Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Correct Worker-isolate start time/uptime, emit privacy-safe structured request completion logs, split production smoke into cost-efficient quick/full profiles, and document measurable operating objectives. + +**Architecture:** Metrics state is created per Elysia app and initialized only inside request handling through an injected clock. Smoke checks remain in one implementation registry and profiles select subsets by stable check IDs. The workflow captures smoke outcome, attempts notifications independently, and re-emits the original failure after all notification steps. + +**Tech Stack:** TypeScript, Elysia hooks, Bun tests, Node.js test runner/fetch, GitHub Actions, Cloudflare Workers observability. + +**Execution Order:** Run after `2026-07-13-security-governance.md` and before the coverage plan so the corrected metrics and smoke-profile modules enter the first coverage baseline. + +--- + +### Task 1: Reproduce the epoch-start bug with a deterministic metrics clock + +**Files:** + +- Modify: `apps/api-ground-codes/src/endpoints/metrics.ts` +- Create: `apps/api-ground-codes/src/metrics.test.ts` +- Modify: `apps/api-ground-codes/src/app.ts` +- Modify: `apps/api-ground-codes/src/app.test.ts` + +- [ ] Add a `metrics.test.ts` contract that constructs an app with a clock whose module-creation wall time is `0`, advances it to `2026-07-13T00:00:00.000Z` before the first request, and expects `startedAt` to equal the request-time value rather than 1970. +- [ ] Add assertions for a second request 5.4 seconds later: `startedAt` is unchanged, uptime rounds consistently, counters stay finite, and `/metrics` itself is not counted. +- [ ] Run `pnpm --filter api-ground-codes test`; expect RED because metrics currently initialize eagerly and the app cannot inject a clock. +- [ ] Introduce these testable interfaces without retaining global eager wall-clock state: + + ```ts + export interface MetricsClock { + nowMs(): number; + monotonicMs(): number; + } + + export interface MetricsOptions { + clock?: MetricsClock; + writeLog?: (record: RequestCompletionLog) => void; + } + + export const createMetricsEndpoint = (options: MetricsOptions = {}) => { + let startedAtMs: number | undefined; + // onRequest: startedAtMs ??= clock.nowMs() + }; + ``` + +- [ ] Extend `AppOptions` with `metrics?: MetricsOptions`, replace the singleton `.use(metricsEndpoint)` with `.use(createMetricsEndpoint(options.metrics))`, and ensure each test app owns isolated counters. +- [ ] Serialize `startedAt` from the stored millisecond value and calculate `uptimeSeconds` from that same value with `Math.max(0, ...)`. Include `runtimeCommit` in the metrics response. +- [ ] Run API tests; expect PASS and no epoch timestamps. +- [ ] Commit as `fix(api): initialize metrics clock on first request`. + +### Task 2: Emit a privacy-safe structured request completion record + +**Files:** + +- Modify: `apps/api-ground-codes/src/endpoints/metrics.ts` +- Modify: `apps/api-ground-codes/src/metrics.test.ts` +- Modify: `apps/api-ground-codes/src/worker.ts` + +- [ ] Add failing tests that send encode/search requests containing sentinel coordinates, query text, codes, IP headers, and authorization values. Parse captured log output and assert none of the sentinels occur. +- [ ] Assert exactly one completion record per request, including error responses, with this schema: + + ```ts + interface RequestCompletionLog { + event: "api.request.completed"; + service: "api-ground-codes"; + route: string; + method: string; + status: string; + durationMs: number; + runtimeCommit: string; + } + ``` + +- [ ] Run the metrics tests; expect RED because no completion log is emitted. +- [ ] Reuse `recordRequest` to produce the record from `new URL(request.url).pathname`, `request.method`, normalized status, duration, and runtime metadata. Do not serialize URL queries, bodies, headers, region names, coordinates, search terms, or generated codes. +- [ ] Default `writeLog` to `record => console.log(JSON.stringify(record))`; keep it injectable so tests capture records without patching the console. +- [ ] Guard against duplicate records when the Elysia error hook runs by deleting the request's start marker after completion. +- [ ] Run `pnpm --filter api-ground-codes test` and inspect a local `/readyz` request log; expect one valid JSON line and no request content. +- [ ] Commit as `feat(api): emit structured completion logs`. + +### Task 3: Strengthen the production metrics validator + +**Files:** + +- Modify: `scripts/production-smoke-helpers.mjs` +- Modify: `scripts/production-smoke.test.mjs` + +- [ ] Replace the empty valid fixture with one containing `service`, `scope`, a valid post-2020 `startedAt`, `uptimeSeconds`, `runtimeCommit`, and counters. Add failing fixtures for invalid ISO, pre-2020 date, future date, inconsistent uptime, missing service, missing runtime commit, and non-finite nested values. +- [ ] Run `node --test scripts/production-smoke.test.mjs`; expect RED for the missing timestamp/commit validation. +- [ ] Change the validator signature to accept a deterministic comparison time: + + ```js + export const validateMetricsSnapshot = ( + metrics, + { nowMs = Date.now(), uptimeToleranceSeconds = 3 } = {}, + ) => errors; + ``` + +- [ ] Require `startedAt >= 2020-01-01T00:00:00.000Z`, `startedAt <= now + 3s`, `Math.abs(expectedUptime - uptimeSeconds) <= 3`, service `api-ground-codes`, scope `worker-isolate`, and a 40-character runtime commit. Keep empty route maps valid. +- [ ] Run the smoke tests; expect PASS. +- [ ] Commit as `test(smoke): validate metrics clock and runtime metadata`. + +### Task 4: Define one smoke registry with quick and full profiles + +**Files:** + +- Create: `scripts/production-smoke-profiles.mjs` +- Create: `scripts/production-smoke-quick.mjs` +- Modify: `scripts/production-smoke.mjs` +- Modify: `scripts/production-smoke-core.mjs` +- Modify: `scripts/production-smoke-expanded.mjs` +- Modify: `scripts/production-smoke-additional.mjs` +- Modify: `scripts/production-smoke-additional-latin.mjs` +- Modify: `scripts/production-smoke-operations.mjs` +- Modify: `scripts/production-smoke.test.mjs` +- Modify: `scripts/language-support-completeness.test.mjs` +- Modify: `package.json` + +- [ ] Add tests for `resolveSmokeProfile(undefined) === "full"`, explicit `quick`/`full`, rejection of unknown profiles, stable check IDs, and the exact runner selection. Assert the full registry covers every API language exactly once while quick contains at least one non-Latin language. +- [ ] Run focused tests; expect RED because profiles do not exist and all modules currently run unconditionally. +- [ ] Extract the shared fast checks into `runQuickSmokeChecks`: API readiness, Web root, robots, sitemap, metrics, representative English Earth encode/search/decode, Korean encode, Moon encode, and Mars encode. Each check keeps the same HTTP implementation when selected from full. +- [ ] Leave all remaining 180-language cases in the existing split modules and move only operational/error checks not already in quick to `runFullOperationsSmokeChecks`. +- [ ] Implement the registry: + + ```js + export const smokeProfiles = { + quick: [runQuickSmokeChecks], + full: [ + runQuickSmokeChecks, + runCoreLanguageSmokeChecks, + runExpandedSmokeChecks, + runAdditionalSmokeChecks, + runAdditionalLatinSmokeChecks, + runFullOperationsSmokeChecks, + ], + }; + ``` + +- [ ] Read `GROUND_CODES_SMOKE_PROFILE`, default to full, print the chosen profile in console and step summary, and preserve forced-failure testing. +- [ ] Add root commands: + + ```json + "production:smoke": "node scripts/production-smoke.mjs", + "production:smoke:quick": "GROUND_CODES_SMOKE_PROFILE=quick node scripts/production-smoke.mjs", + "production:smoke:full": "GROUND_CODES_SMOKE_PROFILE=full node scripts/production-smoke.mjs" + ``` + +- [ ] Run `node --test scripts/production-smoke.test.mjs scripts/language-support-completeness.test.mjs`; expect PASS and exact 180-language full coverage. +- [ ] Run quick and full against production; expect both PASS and quick to execute materially fewer checks. +- [ ] Commit as `feat(smoke): add quick and full profiles`. + +### Task 5: Make smoke failure notification independent from test outcome + +**Files:** + +- Create: `scripts/production-smoke-notify.mjs` +- Create: `scripts/production-smoke-notify.test.mjs` +- Modify: `.github/workflows/production-smoke.yml` +- Modify: `scripts/qa-workflows.test.mjs` + +- [ ] Write notifier tests with an injected `fetch` for success, missing token, HTTP failure, and exception. Assert the payload contains only title, run URL/message, and token from the environment—not smoke request data. +- [ ] Add workflow-contract tests that require the smoke step to use `continue-on-error: true`, notification steps to inspect `steps.smoke.outcome`, and a final step to exit 1 after notifications when smoke failed. +- [ ] Run both test files; expect RED. +- [ ] Implement the notifier using `process.env.MOSHI_WEBHOOK_TOKEN` and JSON-encoded `fetch` body, returning a non-zero exit on delivery failure without printing the token. +- [ ] Restructure the workflow steps in this order: run smoke and capture outcome; attempt Moshi with `continue-on-error: true`; create/update the fallback issue when the token is missing or delivery fails; finally fail the job if the captured smoke outcome was failure. +- [ ] Run notifier/workflow tests; expect PASS. +- [ ] Commit as `fix(smoke): preserve failures across notification attempts`. + +### Task 6: Reduce scheduled smoke checkout and runner work + +**Files:** + +- Modify: `.github/workflows/production-smoke.yml` +- Modify: `scripts/qa-workflows.test.mjs` + +- [ ] Add failing workflow assertions for shallow history, non-cone sparse checkout of `scripts/production-smoke*.mjs`, two schedules, profile mapping, and event-aware concurrency. +- [ ] Configure schedules and manual input: + + ```yaml + workflow_dispatch: + inputs: + profile: + type: choice + options: [full, quick] + default: full + schedule: + - cron: "*/30 * * * *" # quick + - cron: "17 3 * * *" # daily full + ``` + +- [ ] Set `GROUND_CODES_SMOKE_PROFILE` so the daily cron, Web `workflow_run`, and default manual invocation use full; the 30-minute cron uses quick; an explicit manual selection wins. +- [ ] Configure checkout with `fetch-depth: 1`, `sparse-checkout-cone-mode: false`, and only `scripts/production-smoke*.mjs`. +- [ ] Use a shared `scheduled` concurrency group with `cancel-in-progress: true` only for schedule events. Give post-deploy and manual runs unique groups so they are never cancelled by a newer scheduled run. +- [ ] Run `node --test scripts/qa-workflows.test.mjs`; expect PASS. +- [ ] Trigger one manual quick and one manual full run, and inspect checkout timing/artifact summary. +- [ ] Commit as `ci(smoke): use profiles and sparse checkout`. + +### Task 7: Document SLOs, queries, and rollback operations + +**Files:** + +- Create: `docs/operations/service-objectives.md` +- Create: `docs/operations/incident-runbook.md` +- Modify: `README.md` +- Modify: `apps/api-ground-codes/src/app.test.ts` + +- [ ] Add a documentation contract test that asserts the service-objectives document states 99.9% monthly readiness and Web-root availability, representative encode/search under 2 seconds, and full post-deploy smoke before incident closure. +- [ ] Document Cloudflare observability filters for `event=api.request.completed`, status families, route duration, runtime commit, and deployment rollback to the last known-good SHA. Link the smoke workflow, Worker/Pages deployment histories, metrics endpoint, and incident issue workflow. +- [ ] State explicitly that logs must not include coordinates, search strings, ground codes, IP addresses, headers, or credentials. +- [ ] Link the operational documents from the root README. +- [ ] Run API/document tests and Prettier; expect PASS. +- [ ] Commit as `docs(operations): define service objectives and runbook`. + +### Task 8: Complete the metrics and smoke stage gate + +**Files:** + +- Modify: this plan + +- [ ] Run `pnpm --filter api-ground-codes test`, `pnpm scripts:test`, `pnpm check-types`, and `pnpm build`; expect PASS. +- [ ] Run `GROUND_CODES_SMOKE_PROFILE=quick pnpm production:smoke` and `GROUND_CODES_SMOKE_PROFILE=full pnpm production:smoke`; expect both PASS. +- [ ] Fetch production `/metrics` after deployment and verify a post-2020 ISO `startedAt`, consistent uptime within 3 seconds, valid counters, and the deployed 40-character commit. +- [ ] Inspect Cloudflare logs for a representative request and verify the completion JSON schema contains no request payload or identifiers. +- [ ] Mark all completed checkboxes and commit as `docs: complete metrics and smoke plan`. diff --git a/docs/superpowers/plans/2026-07-13-r2-region-data-delivery.md b/docs/superpowers/plans/2026-07-13-r2-region-data-delivery.md new file mode 100644 index 00000000..33f7cf7b --- /dev/null +++ b/docs/superpowers/plans/2026-07-13-r2-region-data-delivery.md @@ -0,0 +1,299 @@ +# R2 Region Data Delivery Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Publish the 8.35 GiB region dataset as immutable, content-addressed Cloudflare R2 releases; prove complete hash-equivalent materialization; then remove `region-dist` and `region-db` from the current Git tree without rewriting history. + +**Architecture:** A deterministic manifest maps repository-relative logical files to gzip-compressed SHA-256 object keys. A read-only synchronizer verifies the committed release pointer and every downloaded uncompressed object before atomic placement. Publication is append-only and writes the manifest last. The migration has an enforced shadow phase with Git data retained, followed by a separate R2-only cutover after merged CI/deploy evidence exists. + +**Tech Stack:** Node.js ESM and built-ins, gzip/SHA-256, `@aws-sdk/client-s3` 3.1085.0, Cloudflare R2, pnpm, GitHub Actions, PostGIS import tooling. + +**Execution Order:** Run after the security, metrics/smoke, and CI/deploy/coverage plans. Phase A and Phase B are separate merge/deploy gates and must not be collapsed. + +--- + +### Task 1: Specify and validate the manifest model + +**Files:** + +- Create: `scripts/region-data/manifest.mjs` +- Create: `scripts/region-data/manifest.test.mjs` +- Create: `scripts/region-data/fixtures/region-dist/sample.json` +- Create: `scripts/region-data/fixtures/region-db/sample.index` +- Create: `scripts/region-data/fixtures/region-db/sample/000001.log` +- Modify: `scripts/code-size-policy.test.mjs` +- Modify: `package.json` + +- [ ] Write tests for stable logical-path ordering, binary-safe SHA-256, uncompressed/compressed sizes, group assignment, duplicate-content object-key reuse, content-derived release version, canonical JSON serialization, and rejection of paths outside `region-dist`/`region-db`. +- [ ] Define this versioned document shape in the tests: + + ```js + { + schemaVersion: 1, + version: "sha256-<64 lowercase hex characters>", + entries: [ + { + path: "packages/geoint/region-dist/sample.json", + group: "region-dist", + size: 123, + compressedSize: 91, + sha256: "<64 lowercase hex characters>", + objectKey: "releases//objects/.json.gz" + } + ] + } + ``` + +- [ ] Run `node --test scripts/region-data/manifest.test.mjs`; expect RED because the module is absent. +- [ ] Implement deterministic recursive enumeration of regular files only. Reject symlinks, sockets, traversal, absolute paths, duplicate logical paths, unsupported schema versions, and malformed hashes. +- [ ] Compute `version` from canonical entry metadata before release-prefixing object keys so the algorithm has no circular dependency. Do not include timestamps, host paths, usernames, or compression timestamps in the version input. +- [ ] Keep every new maintained module below 450 lines and add the new directory to the source-size boundary test. +- [ ] Expand root `scripts:test` to `node --test scripts/*.test.mjs scripts/region-data/*.test.mjs` so nested region-data contracts are mandatory in CI. +- [ ] Run manifest and source-size tests; expect PASS. +- [ ] Commit as `test(region-data): define immutable manifest contract`. + +### Task 2: Generate deterministic compressed release artifacts + +**Files:** + +- Create: `scripts/region-data/generate-release.mjs` +- Create: `scripts/region-data/generate-release.test.mjs` +- Create: `scripts/generate-region-data-release.mjs` +- Modify: `package.json` +- Modify: `.gitignore` + +- [ ] Write tests that generate the same release twice and compare manifest bytes, object bytes, version, and deduplication. Change one source byte and assert only its hash object and release version change. +- [ ] Run the generator tests; expect RED. +- [ ] Gzip every regular file with deterministic metadata and write each unique object once under `.region-data-staging/releases//objects/.json.gz`. The `.json.gz` suffix is retained for the approved object contract even when a `region-db` logical file contains binary LevelDB data. +- [ ] Write `.region-data-staging/releases//manifest.json` only after all objects exist, and calculate a separate SHA-256 over the final manifest bytes. +- [ ] Make the CLI accept only explicit arguments: + + ```sh + node scripts/generate-region-data-release.mjs \ + --source packages/geoint \ + --staging .region-data-staging \ + --pointer-out packages/geoint/region-data-release.json + ``` + +- [ ] The pointer output must be small and canonical: + + ```js + { + schemaVersion: 1, + version: manifest.version, + manifestSha256, + } + ``` + +- [ ] Add `.region-data-staging/` to `.gitignore` and root script `region-data:generate` for the exact CLI above. +- [ ] Run tests; expect PASS. +- [ ] Commit as `feat(region-data): generate content-addressed releases`. + +### Task 3: Build a fail-closed synchronizer + +**Files:** + +- Create: `scripts/region-data/sync.mjs` +- Create: `scripts/region-data/sync.test.mjs` +- Create: `scripts/sync-region-data.mjs` +- Modify: `package.json` + +- [ ] Test against a local HTTP server for: valid full download, already-valid skip, corrupt-local replacement, manifest hash mismatch, object hash mismatch, truncated gzip, 404, retryable 429/5xx, bounded concurrency, group scope, explicit-path scope, atomic temp rename, default no-prune, and explicit prune. +- [ ] Add traversal and malicious-manifest tests proving no write can escape the requested root. +- [ ] Run `node --test scripts/region-data/sync.test.mjs`; expect RED. +- [ ] Implement pointer validation first, then download `releases//manifest.json` from `REGION_DATA_BASE_URL`, verify `manifestSha256`, validate every entry, and select entries by `--groups` or repeated `--path` flags. +- [ ] For each selected entry, skip an existing matching file; otherwise stream the gzip object to a sibling temporary file, decompress while hashing/counting uncompressed bytes, verify size/SHA-256, fsync/close, then atomically rename. Remove temporary files on all failures. +- [ ] Use default concurrency 4, three attempts, and bounded exponential backoff. Retry network errors, 429, and 5xx only. Fail immediately for other 4xx, schema, hash, size, path, or decompression errors. +- [ ] Default behavior must never delete files. `--prune` removes only files under the selected managed group roots that are absent from the verified manifest; reject prune with an explicit-path-only scope. +- [ ] Add root scripts: + + ```json + "region-data:sync": "node scripts/sync-region-data.mjs --groups region-dist,region-db", + "region-data:sync:ci": "node scripts/sync-region-data.mjs --groups region-dist,region-db --prune" + ``` + +- [ ] Run synchronizer tests and an offline fixture sync; expect PASS. +- [ ] Commit as `feat(region-data): add verified r2 synchronizer`. + +### Task 4: Implement immutable R2 publication + +**Files:** + +- Create: `scripts/region-data/publish.mjs` +- Create: `scripts/region-data/publish.test.mjs` +- Create: `scripts/publish-region-data.mjs` +- Modify: `package.json` +- Modify: `pnpm-lock.yaml` + +- [ ] Add exact development dependency `@aws-sdk/client-s3@3.1085.0`. +- [ ] Write publisher tests with a fake S3 client for missing-object upload, existing-object skip, metadata mismatch failure, bounded multipart/concurrent upload behavior, retry, manifest-last ordering, existing-manifest immutability, and no credential logging. +- [ ] Run publisher tests; expect RED. +- [ ] Configure the S3 client from `CLOUDFLARE_ACCOUNT_ID`, `R2_REGION_DATA_ACCESS_KEY_ID`, and `R2_REGION_DATA_SECRET_ACCESS_KEY`, using endpoint `https://.r2.cloudflarestorage.com`, region `auto`, and bucket `ground-codes-region-data`. Fail before network access if any credential is absent. +- [ ] HEAD each unique hash object; upload only absent objects with immutable cache metadata. Treat a conflicting size/metadata response as fatal rather than overwriting. +- [ ] Upload `releases//manifest.json` only after all objects succeed. If the manifest already exists, download/compare exact bytes and succeed only on equality. +- [ ] Print only counts, bytes, version, and object keys; never print S3 request headers, secret values, or the environment. +- [ ] Add root `region-data:publish` command targeting `.region-data-staging`. +- [ ] Run publisher tests; expect PASS. +- [ ] Commit as `feat(region-data): publish immutable r2 releases`. + +### Task 5: Add whole-release verification independent of sync + +**Files:** + +- Create: `scripts/region-data/verify.mjs` +- Create: `scripts/region-data/verify.test.mjs` +- Create: `scripts/verify-region-data.mjs` +- Modify: `package.json` +- Modify: `scripts/coverage-policy.json` +- Modify: `docs/quality/coverage.md` + +- [ ] Write tests that compare source and materialized trees by path, group, byte size, and SHA-256; cover missing, extra, changed, and permission/read failures. +- [ ] Implement `region-data:verify` to support both source-vs-materialized shadow verification and manifest-vs-materialized verification. Extra managed files fail in exact mode. +- [ ] Emit a compact result with version, entry count, group counts, bytes, and mismatches; never dump dataset content. +- [ ] Extend `coverage:operations` and its policy include list with `manifest.mjs`, `generate-release.mjs`, `sync.mjs`, `publish.mjs`, and `verify.mjs`. Keep line/function floors at 80%, measure the expanded target's branch ratio, and raise or preserve—not lower—the committed branch floor. Record the expanded boundary and result in `docs/quality/coverage.md`. +- [ ] Run all region-data tests; expect PASS. +- [ ] Commit as `test(region-data): verify complete release materialization`. + +### Task 6: Provision the R2 read/write boundary and GitHub settings + +**Files:** + +- Create: `docs/operations/region-data-delivery.md` + +- [ ] Create the bucket with `pnpm exec wrangler r2 bucket create ground-codes-region-data` and verify it exists with `pnpm exec wrangler r2 bucket list`. +- [ ] Configure the public read custom domain `region-data.ground.codes` for this bucket while retaining private writes. Verify HTTPS GET is available only for published object paths; bucket listing is not exposed. +- [ ] Create a least-privilege R2 S3 token scoped only to object read/write for `ground-codes-region-data`. +- [ ] Store `R2_REGION_DATA_ACCESS_KEY_ID`, `R2_REGION_DATA_SECRET_ACCESS_KEY`, and `CLOUDFLARE_ACCOUNT_ID` as GitHub Actions secrets through stdin sessions; store `REGION_DATA_BASE_URL=https://region-data.ground.codes` as a GitHub Actions variable. Never pass secret values in command arguments or save them in files. +- [ ] Document bucket, domain, credential scope, rotation steps, variable/secret names, and rollback behavior. Record secret presence only. +- [ ] Verify with GitHub/Cloudflare APIs that the names and endpoint exist without returning credential values. +- [ ] Commit as `docs(region-data): document r2 delivery operations`. + +### Task 7: Publish the current Git-backed release and shadow-verify it + +**Files:** + +- Create: `packages/geoint/region-data-release.json` +- Modify: `docs/operations/region-data-delivery.md` + +- [ ] Materialize the current `main` data in an isolated full worktree and verify the source inventory is 901 `region-dist` files plus 1,130 `region-db` files before publication. Stop if the authoritative source differs unexpectedly. +- [ ] Run `pnpm region-data:generate`; expect one content-derived version, 2,031 logical entries, deduplicated gzip objects, and a small pointer file. +- [ ] Run `pnpm region-data:publish`; expect all missing objects first and the manifest last. Re-run it and expect an idempotent zero-object upload. +- [ ] Sync that version into an empty temporary directory with public read credentials only: + + ```sh + REGION_DATA_BASE_URL=https://region-data.ground.codes \ + node scripts/sync-region-data.mjs \ + --root /tmp/ground-codes-region-shadow \ + --groups region-dist,region-db --prune + ``` + +- [ ] Run source-vs-shadow verification; expect 2,031 matching logical paths and zero missing/extra/hash/size mismatches. +- [ ] Delete the temporary materialization, repeat once from an empty directory, and verify again to rule out accidental local fallback. +- [ ] Commit the real pointer and evidence as `feat(region-data): publish initial r2 release`. + +### Task 8: Use R2 in CI/deploy while Git data remains available (Phase A) + +**Files:** + +- Modify: `.github/workflows/ci.yml` +- Modify: `.github/workflows/deploy-api.yml` +- Modify: `.github/workflows/visual-qa.yml` +- Modify: `apps/api-ground-codes/scripts/list-changed-region-datasets.mjs` +- Create: `apps/api-ground-codes/scripts/list-changed-region-datasets.test.mjs` +- Modify: `scripts/qa-workflows.test.mjs` +- Modify: `README.md` +- Modify: `packages/geoint/README.md` + +- [ ] Add workflow tests requiring a verified sync before any data audit/test/import and requiring `REGION_DATA_BASE_URL` from repository variables. Public PR CI must not reference R2 write secrets. +- [ ] Update CI to run `node scripts/sync-region-data.mjs --groups region-dist,region-db --prune` before tests. During Phase A this validates/replaces the checked-out copies but does not yet reduce checkout size. +- [ ] Update API deploy to sync `region-dist` before changed-dataset detection, API tests, PostGIS import, or build. Keep database deployment concurrency non-cancelling. +- [ ] Change changed-dataset detection to compare old/new release manifests by logical `region-dist` entry hash when the pointer changes. Preserve `__all_missing__` for manual imports and return empty when the release pointer is unchanged. +- [ ] Add focused tests for added, removed, and changed dataset hashes and for unavailable prior manifests. Fail closed rather than silently importing nothing. +- [ ] Add local setup instructions: clone, frozen install, `REGION_DATA_BASE_URL=... pnpm region-data:sync`, tests/build. State that generators require an explicitly materialized working tree. +- [ ] Run workflow, detector, sync, full test, coverage, and build commands locally; expect PASS. +- [ ] Commit as `ci(region-data): shadow materialize from r2`. + +### Task 9: Prove Phase A on merged main before any deletion + +**Files:** + +- Modify: `docs/operations/region-data-delivery.md` + +- [ ] Open the Phase A pull request and require green `verify`, including R2 sync, all data/language audits, unit tests, coverage, build, and browser smoke. +- [ ] Merge Phase A with Git data still tracked. Record the merged commit and successful merged-main CI URL. +- [ ] Confirm API, Web, and Grok Spiral deploy workflows succeed for that commit where applicable; run/record the full post-deploy production smoke. +- [ ] Run a clean public-URL shadow sync after merge and verify every manifest entry/hash again. +- [ ] Record the successful run URLs and hashes. Do not begin Task 10 if any Phase A check is missing or failed. +- [ ] Commit the evidence update on a fresh Phase B branch as `docs(region-data): record shadow rollout evidence`. + +### Task 10: Remove managed data from the current Git tree (Phase B) + +**Files:** + +- Delete: `packages/geoint/region-dist/**` +- Delete: `packages/geoint/region-db/**` +- Modify: `.gitignore` +- Modify: `packages/geoint/package.json` +- Modify: `packages/geoint/src/index.ts` +- Modify: `packages/geoint/README.md` +- Modify: `README.md` +- Modify: `.github/workflows/deploy-api.yml` +- Modify: `scripts/language-support-completeness.test.mjs` +- Modify: all generators/audits found by `rg -l 'packages/geoint/region-(dist|db)' scripts packages apps .github` + +- [ ] Add a failing repository contract that asserts neither managed directory is tracked by `git ls-files`, both are ignored materialization targets, and the pointer/fixtures remain tracked. +- [ ] Run the contract test; expect RED while Git data remains. +- [ ] Add `/packages/geoint/region-dist/` and `/packages/geoint/region-db/` to `.gitignore`; remove those paths from the geoint package `files` array and include `region-data-release.json` plus sync/setup documentation. +- [ ] Allow `@ground-codes/geoint.load` to receive an explicit `dataDir` (defaulting to the materialized package path or `GROUND_CODES_REGION_DB_DIR`) and throw a clear sync instruction when required data is absent. Never download implicitly at runtime. +- [ ] Update generators, audits, PostGIS tooling, runtime-pin package assembly, and workflows to call the synchronizer explicitly before reading data. Keep their logical paths unchanged after materialization to minimize consumer changes. +- [ ] Replace API deploy path triggers for deleted directories with the release pointer, manifest/sync scripts, generator/source inputs, and publication workflow. +- [ ] Remove the managed directories from the Git index in one dedicated commit without history rewriting. Do not delete `region-dataset` sources, tiny test fixtures, or the active pointer. +- [ ] Run the repository contract after removal; expect PASS. +- [ ] Commit as `chore(region-data): remove generated datasets from git tree`. + +### Task 11: Verify a fresh R2-only checkout + +**Files:** + +- Create: `scripts/region-data/fresh-checkout.mjs` +- Create: `scripts/region-data/fresh-checkout.test.mjs` +- Modify: `scripts/qa-workflows.test.mjs` +- Modify: `docs/operations/region-data-delivery.md` + +- [ ] Unit-test command construction and cleanup with a tiny local fixture repository. Implement the opt-in verification script to create a clean temporary checkout of the Phase B commit with no shared untracked data, assert both managed directories are initially absent, sync from the public URL, and verify the release. Do not run the 8.35 GiB network verification from the ordinary `scripts:test` glob. +- [ ] In that clean checkout run frozen install, format check, source-size check, security audit, operational scripts, data/language audits, lint, type checks, all unit/coverage tests, build, and browser smoke. +- [ ] Run API Worker dry-run packaging and confirm the sync happens before PostGIS detection/import. Do not mutate production in this step. +- [ ] Delete managed directories again and prove a second sync reconstructs the same file count and hashes. +- [ ] Record command output summaries and version/hash in the operations document. +- [ ] Commit as `test(region-data): prove fresh r2-only checkout`. + +### Task 12: Add the protected future publication workflow + +**Files:** + +- Create: `.github/workflows/publish-region-data.yml` +- Modify: `scripts/qa-workflows.test.mjs` +- Modify: `docs/operations/region-data-delivery.md` + +- [ ] Add workflow tests requiring manual dispatch, minimal `contents: write`/`pull-requests: write`, `environment: region-data-publisher`, pinned actions, frozen install, active-release sync, all data quality audits, immutable publication, pointer-only commit validation, and pull request creation. +- [ ] Implement the workflow on `main`: sync the active release; run approved generators/source transformations already committed to main; run data/language tests; generate and publish a new immutable release; verify a clean materialization; commit only `packages/geoint/region-data-release.json` and documentation; push `data-release/`; open a PR with `gh pr create`. +- [ ] Reject a release branch if `git diff --cached --name-only` includes materialized `region-dist`/`region-db`, credentials, staging files, or unrelated source changes. +- [ ] Give the publisher environment access to R2 write secrets; keep normal CI and pull requests read-only/public. +- [ ] Test a no-content-change dispatch and require it to exit successfully without publishing or opening a PR. Test a fixture content change in a non-production bucket before the first production use. +- [ ] Run workflow-contract tests; expect PASS. +- [ ] Commit as `ci(region-data): add protected publication workflow`. + +### Task 13: Merge Phase B and complete the R2 stage gate + +**Files:** + +- Modify: this plan +- Modify: `docs/operations/region-data-delivery.md` + +- [ ] Open the Phase B pull request and verify authoritative CI starts from a checkout where managed directories are absent, materializes from R2, and passes every format/security/data/test/coverage/build/browser gate. +- [ ] Merge only after green CI. Verify the same merged main commit deploys API, Web, and Grok Spiral successfully where path filters apply. +- [ ] Run and record full post-deploy production smoke. Verify `/metrics` reports the merged commit and realistic uptime. +- [ ] Use GitHub APIs to confirm the release pointer is tracked, managed directories are absent from the current tree, rules/security settings remain enabled, and required checks are green. +- [ ] Perform one final clean R2 materialization and manifest-vs-filesystem hash verification; expect 2,031 entries unless a documented Phase A source change intentionally changed the inventory. +- [ ] Verify the previous R2 release remains readable and document rollback by restoring its pointer in a normal PR. +- [ ] Mark all checkboxes and commit as `docs: complete r2 region data delivery plan`. diff --git a/docs/superpowers/plans/2026-07-13-security-governance.md b/docs/superpowers/plans/2026-07-13-security-governance.md new file mode 100644 index 00000000..4ab175d4 --- /dev/null +++ b/docs/superpowers/plans/2026-07-13-security-governance.md @@ -0,0 +1,220 @@ +# Security and Repository Governance Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Remove every high/critical production dependency advisory, eliminate moving runtime pins, make dependency and workflow security enforceable in CI, and configure GitHub repository security controls without exposing credentials. + +**Architecture:** Security policy is expressed as small Node scripts with fixture-driven tests and workflow-contract assertions. Runtime upgrades are committed in compatible groups so each can be reverted independently. Repository settings are applied by an idempotent `gh api` wrapper only after the stable `verify` check exists on the pull request. + +**Tech Stack:** pnpm 9, Node.js ESM, Bun/Elysia, Next.js, GitHub Actions, Dependabot, GitHub REST API, `gh` CLI. + +**Execution Order:** Run this plan first. The metrics, CI/coverage, and R2 plans rely on its pinned actions, audit command, and stable `verify` check. + +--- + +### Task 1: Turn the production audit requirement into an executable policy + +**Files:** + +- Create: `scripts/production-audit-policy.mjs` +- Create: `scripts/production-audit-policy.test.mjs` +- Create: `scripts/check-production-audit.mjs` +- Modify: `package.json` + +- [ ] Write fixture-driven tests that pass when `metadata.vulnerabilities.high` and `critical` are zero, fail when either is non-zero, and preserve moderate/low counts for the report. Cover both a single pnpm JSON document and newline-delimited diagnostic output before the final JSON document. +- [ ] Run `node --test scripts/production-audit-policy.test.mjs` and verify RED because the policy module does not exist. +- [ ] Implement a pure parser and evaluator with this public shape: + + ```js + export const evaluateProductionAudit = (rawAuditJson) => ({ + ok: high === 0 && critical === 0, + counts: { critical, high, moderate, low }, + }); + ``` + +- [ ] Implement `check-production-audit.mjs` so it runs `pnpm audit --prod --json`, prints only severity counts and advisory package names, and exits non-zero only for high/critical findings or an unreadable audit result. Never print dependency environment values. +- [ ] Add exact root scripts: + + ```json + "security:audit": "node scripts/check-production-audit.mjs", + "security:audit:raw": "pnpm audit --prod --audit-level high" + ``` + +- [ ] Run `node --test scripts/production-audit-policy.test.mjs`; expect all policy tests PASS. +- [ ] Run `pnpm security:audit`; expect RED against the current lockfile with `critical=0 high=4`. +- [ ] Commit the test-first policy as `test(security): enforce production audit severity`. + +### Task 2: Upgrade and pin the Elysia runtime group + +**Files:** + +- Modify: `apps/api-ground-codes/package.json` +- Modify: `scripts/api-runtime-pins.test.mjs` +- Modify: `scripts/check-api-runtime-pins.mjs` +- Modify: `scripts/update-api-runtime-pins.mjs` +- Modify: `pnpm-lock.yaml` + +- [ ] Change the existing runtime-pin tests to require exact semver values and to reject `latest`, `*`, Git branch references, and unbounded ranges for `elysia` and `bun-types`. Add plugin compatibility assertions. +- [ ] Run `node --test scripts/api-runtime-pins.test.mjs`; expect RED because the current package still uses `latest`. +- [ ] Install this reviewed compatible group with exact specifications: + + ```sh + pnpm --filter api-ground-codes add --save-exact \ + elysia@1.4.29 @elysiajs/cors@1.4.2 \ + @elysiajs/static@1.4.10 @elysiajs/swagger@1.3.1 + pnpm --filter api-ground-codes add --save-dev --save-exact bun-types@1.3.1 + ``` + +- [ ] Update the runtime-pin checker/updater so generated deployment package manifests retain those exact versions rather than restoring `latest`. +- [ ] Run `pnpm runtime:check-pins`, `pnpm --filter api-ground-codes check-types`, and `pnpm --filter api-ground-codes test`; expect PASS with the existing API contract unchanged. +- [ ] Run `pnpm --filter api-ground-codes build`; expect the workspace runtime build to complete. +- [ ] Commit as `chore(api): upgrade and pin elysia runtime`. + +### Task 3: Upgrade vulnerable Web dependencies and lock transitive fixes + +**Files:** + +- Modify: `apps/web/package.json` +- Modify: `package.json` +- Modify: `pnpm-lock.yaml` +- Create: `docs/security/dependency-audit.md` + +- [ ] Add a contract test to `scripts/production-audit-policy.test.mjs` that rejects a lockfile finding whose severity is high even when it is transitive. +- [ ] Upgrade Web runtime dependencies exactly: + + ```sh + pnpm --filter web add --save-exact next-intl@4.13.2 cesium@1.143.0 + ``` + +- [ ] Add root pnpm overrides for the reviewed fixed transitive floors, using exact values: + + ```json + "pnpm": { + "overrides": { + "dompurify@3.4.2": "3.4.12", + "picomatch@2.3.1": "2.3.2", + "postcss@8.4.31": "8.4.49", + "postcss@8.5.3": "8.5.19", + "protobufjs@8.2.0": "8.7.1" + } + } + ``` + +- [ ] Run `pnpm install --frozen-lockfile`; expect no lockfile drift after the upgrade command has written the lockfile. +- [ ] Run `pnpm --filter web test`, `pnpm --filter web check-types`, `pnpm --filter web build`, and `pnpm --filter web test:e2e:smoke`; expect PASS. +- [ ] Run `pnpm security:audit`; expect `critical=0 high=0`. If moderate findings remain, document each package, affected path, runtime applicability, mitigating control, and a concrete GitHub issue URL in `docs/security/dependency-audit.md`; do not waive a high/critical finding. +- [ ] Commit as `chore(web): remediate production dependency advisories`. + +### Task 4: Add Dependabot and pin every GitHub Action + +**Files:** + +- Create: `.github/dependabot.yml` +- Modify: `.github/workflows/ci.yml` +- Modify: `.github/workflows/deploy-api.yml` +- Modify: `.github/workflows/deploy-web.yml` +- Modify: `.github/workflows/deploy-grok-spiral.yml` +- Modify: `.github/workflows/production-smoke.yml` +- Modify: `.github/workflows/visual-qa.yml` +- Modify: `scripts/qa-workflows.test.mjs` + +- [ ] Extend workflow-contract tests to reject any `uses:` value that is not a local action and does not end in a 40-character hexadecimal SHA plus a release comment. Assert weekly npm and GitHub Actions Dependabot groups with a pull-request limit. +- [ ] Run `node --test scripts/qa-workflows.test.mjs`; expect RED for the current mutable tags and absent Dependabot config. +- [ ] Create `.github/dependabot.yml` with `/` npm updates and GitHub Actions updates scheduled weekly on Monday in `Asia/Seoul`, grouped separately as `runtime-security`, `development-tooling`, and `github-actions`, with a limit of 10 open PRs. +- [ ] Replace current action tags with these resolved SHAs and adjacent comments: + + ```yaml + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6 + uses: actions/cache@caa296126883cff596d87d8935842f9db880ef25 # v5 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6 + uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2 + ``` + +- [ ] Run `node --test scripts/qa-workflows.test.mjs`; expect PASS. +- [ ] Commit as `chore(actions): pin automation and add dependabot`. + +### Task 5: Make GitHub governance configuration reproducible + +**Files:** + +- Create: `scripts/github-governance.mjs` +- Create: `scripts/github-governance.test.mjs` +- Create: `scripts/configure-github-governance.mjs` +- Modify: `package.json` +- Modify: `docs/security/dependency-audit.md` + +- [ ] Write tests for a pure ruleset builder and repository settings patch. Assert `refs/heads/main`, pull requests with zero required approvals, strict `verify` status, deletion/non-fast-forward protections, administrator-only bypass, security-analysis enablement, and merged-branch deletion. +- [ ] Run `node --test scripts/github-governance.test.mjs`; expect RED because the module is absent. +- [ ] Implement an idempotent `gh api` wrapper that discovers an existing ruleset by name `main-protection`, creates it when absent, and updates the same ruleset when present. Its rules payload must include: + + ```js + rules: [ + { type: "deletion" }, + { type: "non_fast_forward" }, + { + type: "pull_request", + parameters: { required_approving_review_count: 0 }, + }, + { + type: "required_status_checks", + parameters: { + strict_required_status_checks_policy: true, + required_status_checks: [{ context: "verify" }], + }, + }, + ]; + ``` + + Set `bypass_actors` to `[{ actor_id: 5, actor_type: "RepositoryRole", bypass_mode: "always" }]`, which is the repository administrator role; do not grant bypass to write/maintain roles or GitHub Apps. + +- [ ] Have the same command enable `delete_branch_on_merge`, Dependabot security updates, secret scanning, push protection, non-provider patterns, and validity checks where the public-repository API reports support. Treat an unsupported optional feature as a named warning; treat failure to enable core secret scanning/push protection as fatal. +- [ ] Add root scripts: + + ```json + "github:governance:apply": "node scripts/configure-github-governance.mjs --apply", + "github:governance:verify": "node scripts/configure-github-governance.mjs --verify" + ``` + +- [ ] Run the unit tests; expect PASS without making network calls. +- [ ] Commit as `feat(security): codify repository governance`. + +### Task 6: Wire the security gates into CI + +**Files:** + +- Modify: `.github/workflows/ci.yml` +- Modify: `scripts/qa-workflows.test.mjs` +- Modify: `docs/security/dependency-audit.md` + +- [ ] Add contract assertions that the `verify` job runs `pnpm install --frozen-lockfile`, `pnpm security:audit`, and all governance/pinning script tests before build/deploy-relevant tests. +- [ ] Run the workflow tests; expect RED until CI contains the audit gate. +- [ ] Add a named `Audit production dependencies` step that runs `pnpm security:audit` immediately after install and before expensive browser setup. +- [ ] Run `pnpm scripts:test`, `pnpm security:audit`, `pnpm lint`, and `pnpm check-types`; expect PASS and zero high/critical advisories. +- [ ] Commit as `ci: enforce production security policy`. + +### Task 7: Apply and verify external GitHub security settings + +**Files:** + +- Modify: `docs/security/dependency-audit.md` + +- [ ] Push the feature branch first and confirm the pull request exposes the stable status check name `verify`; do not activate the required check before it exists. +- [ ] Set `MOSHI_WEBHOOK_TOKEN` with `gh secret set MOSHI_WEBHOOK_TOKEN` and provide the value only through the command's stdin pipe/session. Do not use `--body`, a shell literal, a temporary file, command tracing, or log output. +- [ ] Run `pnpm github:governance:apply` against `hmmhmmhm/ground.codes` with normal output limited to setting names, ruleset ID, and enabled/disabled state. +- [ ] Run `pnpm github:governance:verify`; expect an active `main-protection` ruleset, required `verify`, strict up-to-date enforcement, no force push/deletion, security features enabled, Dependabot security updates enabled, and automatic merged-branch deletion enabled. +- [ ] Record the ruleset ID, verification timestamp, and GitHub API URLs in `docs/security/dependency-audit.md`; record only the presence of secret names, never values. +- [ ] Commit the evidence update as `docs(security): record repository controls`. + +### Task 8: Complete the security stage gate + +**Files:** + +- Modify: this plan + +- [ ] Run `pnpm install --frozen-lockfile` and verify no changes to `pnpm-lock.yaml`. +- [ ] Run `pnpm security:audit`, `pnpm scripts:test`, `pnpm --filter api-ground-codes test`, `pnpm --filter web test`, `pnpm check-types`, and `pnpm build`; expect every command PASS. +- [ ] Run `git grep -nE 'uses: [^ ]+@v[0-9]|\"latest\"|: latest$' -- .github apps packages package.json`; expect no mutable runtime/action pins, allowing ordinary prose uses of the English word “latest”. +- [ ] Mark all completed checkboxes in this plan and commit as `docs: complete security governance plan`. From 7174f8835d83bb7b8c4ddfeea152e5334bd7141c Mon Sep 17 00:00:00 2001 From: hmmhmmhm Date: Mon, 13 Jul 2026 22:17:14 +0900 Subject: [PATCH 03/78] test(security): enforce production audit severity --- package.json | 2 + scripts/check-production-audit.mjs | 29 +++++++++ scripts/production-audit-policy.mjs | 67 ++++++++++++++++++++ scripts/production-audit-policy.test.mjs | 81 ++++++++++++++++++++++++ 4 files changed, 179 insertions(+) create mode 100644 scripts/check-production-audit.mjs create mode 100644 scripts/production-audit-policy.mjs create mode 100644 scripts/production-audit-policy.test.mjs diff --git a/package.json b/package.json index 9c903e6f..b184d70b 100644 --- a/package.json +++ b/package.json @@ -8,6 +8,8 @@ "format": "prettier --write \"**/*.{ts,tsx,md}\"", "format:check": "prettier --check \"**/*.{ts,tsx,md}\"", "code:size-check": "node scripts/check-code-size.mjs", + "security:audit": "node scripts/check-production-audit.mjs", + "security:audit:raw": "pnpm audit --prod --audit-level high", "check-types": "turbo run check-types", "production:smoke": "node scripts/production-smoke.mjs", "data:audit-labels": "pnpm --filter ground-codes test:data-audit && node --test scripts/region-label-quality.test.mjs", diff --git a/scripts/check-production-audit.mjs b/scripts/check-production-audit.mjs new file mode 100644 index 00000000..f5ad2f08 --- /dev/null +++ b/scripts/check-production-audit.mjs @@ -0,0 +1,29 @@ +import { spawnSync } from "node:child_process"; + +import { + evaluateProductionAudit, + getProductionAuditPackageNames, +} from "./production-audit-policy.mjs"; + +const audit = spawnSync("pnpm", ["audit", "--prod", "--json"], { + encoding: "utf8", +}); + +try { + if (audit.error) throw audit.error; + + const result = evaluateProductionAudit(audit.stdout); + const packageNames = getProductionAuditPackageNames(audit.stdout); + + console.log( + `critical=${result.counts.critical} high=${result.counts.high} moderate=${result.counts.moderate} low=${result.counts.low}`, + ); + if (packageNames.length > 0) { + console.log(`advisory packages: ${packageNames.join(", ")}`); + } + + if (!result.ok) process.exitCode = 1; +} catch { + console.error("Production audit result is unreadable."); + process.exitCode = 1; +} diff --git a/scripts/production-audit-policy.mjs b/scripts/production-audit-policy.mjs new file mode 100644 index 00000000..120fcdc3 --- /dev/null +++ b/scripts/production-audit-policy.mjs @@ -0,0 +1,67 @@ +const AUDIT_SEVERITIES = ["critical", "high", "moderate", "low"]; + +const unreadableAudit = () => + new Error("Expected a readable pnpm audit JSON document."); + +export const parseProductionAudit = (rawAuditJson) => { + if (typeof rawAuditJson !== "string") throw unreadableAudit(); + + const trimmed = rawAuditJson.trim(); + const objectStarts = []; + for (let index = trimmed.length - 1; index >= 0; index -= 1) { + if (trimmed[index] === "{") objectStarts.push(index); + } + + for (const index of [0, ...objectStarts]) { + try { + const document = JSON.parse(trimmed.slice(index)); + if ( + document && + typeof document === "object" && + !Array.isArray(document) + ) { + return document; + } + } catch { + // Try the next object start so diagnostics may precede the final document. + } + } + + throw unreadableAudit(); +}; + +const getSeverityCounts = (auditDocument) => { + const vulnerabilities = auditDocument.metadata?.vulnerabilities; + const counts = {}; + + for (const severity of AUDIT_SEVERITIES) { + const count = vulnerabilities?.[severity]; + if (!Number.isSafeInteger(count) || count < 0) throw unreadableAudit(); + counts[severity] = count; + } + + return counts; +}; + +export const evaluateProductionAudit = (rawAuditJson) => { + const counts = getSeverityCounts(parseProductionAudit(rawAuditJson)); + + return { + ok: counts.high === 0 && counts.critical === 0, + counts, + }; +}; + +export const getProductionAuditPackageNames = (rawAuditJson) => { + const auditDocument = parseProductionAudit(rawAuditJson); + const advisoryNames = Object.values(auditDocument.advisories ?? {}).map( + (advisory) => advisory?.module_name, + ); + const vulnerabilityNames = Object.entries( + auditDocument.vulnerabilities ?? {}, + ).map(([name, vulnerability]) => vulnerability?.name ?? name); + + return [...new Set([...advisoryNames, ...vulnerabilityNames])] + .filter((name) => typeof name === "string" && name.length > 0) + .sort((left, right) => left.localeCompare(right)); +}; diff --git a/scripts/production-audit-policy.test.mjs b/scripts/production-audit-policy.test.mjs new file mode 100644 index 00000000..02b91eda --- /dev/null +++ b/scripts/production-audit-policy.test.mjs @@ -0,0 +1,81 @@ +import assert from "node:assert/strict"; +import { describe, test } from "node:test"; + +import { + evaluateProductionAudit, + getProductionAuditPackageNames, +} from "./production-audit-policy.mjs"; + +const auditDocument = ({ critical, high, moderate, low }) => ({ + advisories: { + 1001: { module_name: "example-high-package" }, + 1002: { module_name: "example-low-package" }, + }, + metadata: { + vulnerabilities: { critical, high, moderate, low }, + }, +}); + +const fixtures = [ + { + name: "accepts a single JSON document without high or critical findings", + raw: JSON.stringify( + auditDocument({ critical: 0, high: 0, moderate: 14, low: 3 }), + ), + expected: { + ok: true, + counts: { critical: 0, high: 0, moderate: 14, low: 3 }, + }, + }, + { + name: "rejects a high finding after newline-delimited diagnostics", + raw: [ + "WARN audit endpoint returned a diagnostic", + JSON.stringify( + auditDocument({ critical: 0, high: 4, moderate: 2, low: 1 }), + null, + 2, + ), + ].join("\n"), + expected: { + ok: false, + counts: { critical: 0, high: 4, moderate: 2, low: 1 }, + }, + }, + { + name: "rejects a critical finding", + raw: JSON.stringify( + auditDocument({ critical: 1, high: 0, moderate: 0, low: 0 }), + ), + expected: { + ok: false, + counts: { critical: 1, high: 0, moderate: 0, low: 0 }, + }, + }, +]; + +describe("production audit policy", () => { + for (const fixture of fixtures) { + test(fixture.name, () => { + assert.deepEqual(evaluateProductionAudit(fixture.raw), fixture.expected); + }); + } + + test("reports unique advisory package names without advisory details", () => { + const raw = JSON.stringify( + auditDocument({ critical: 0, high: 0, moderate: 1, low: 1 }), + ); + + assert.deepEqual(getProductionAuditPackageNames(raw), [ + "example-high-package", + "example-low-package", + ]); + }); + + test("rejects an unreadable audit result", () => { + assert.throws( + () => evaluateProductionAudit("audit failed without JSON"), + /readable pnpm audit JSON document/, + ); + }); +}); From 4eea80041fd4d75492a0321e4ed6ad928a478d50 Mon Sep 17 00:00:00 2001 From: hmmhmmhm Date: Mon, 13 Jul 2026 22:34:41 +0900 Subject: [PATCH 04/78] fix(security): harden production audit runner --- scripts/check-production-audit.mjs | 55 +++++++----- scripts/check-production-audit.test.mjs | 103 +++++++++++++++++++++++ scripts/production-audit-policy.mjs | 80 ++++++++++++------ scripts/production-audit-policy.test.mjs | 89 +++++++++++++++----- 4 files changed, 260 insertions(+), 67 deletions(-) create mode 100644 scripts/check-production-audit.test.mjs diff --git a/scripts/check-production-audit.mjs b/scripts/check-production-audit.mjs index f5ad2f08..2180c2a7 100644 --- a/scripts/check-production-audit.mjs +++ b/scripts/check-production-audit.mjs @@ -1,29 +1,42 @@ import { spawnSync } from "node:child_process"; +import { resolve } from "node:path"; +import { fileURLToPath } from "node:url"; -import { - evaluateProductionAudit, - getProductionAuditPackageNames, -} from "./production-audit-policy.mjs"; +import { analyzeProductionAudit } from "./production-audit-policy.mjs"; -const audit = spawnSync("pnpm", ["audit", "--prod", "--json"], { - encoding: "utf8", -}); +// Large workspaces can exceed Node's default buffer; keep a bounded 16 MiB cap. +export const AUDIT_MAX_BUFFER_BYTES = 16 * 1024 * 1024; -try { - if (audit.error) throw audit.error; +export const runProductionAudit = ({ + spawn = spawnSync, + writeError = console.error, + writeOutput = console.log, +} = {}) => { + try { + const audit = spawn("pnpm", ["audit", "--prod", "--json"], { + encoding: "utf8", + maxBuffer: AUDIT_MAX_BUFFER_BYTES, + }); + if (audit.error) throw audit.error; - const result = evaluateProductionAudit(audit.stdout); - const packageNames = getProductionAuditPackageNames(audit.stdout); + const result = analyzeProductionAudit(audit.stdout); - console.log( - `critical=${result.counts.critical} high=${result.counts.high} moderate=${result.counts.moderate} low=${result.counts.low}`, - ); - if (packageNames.length > 0) { - console.log(`advisory packages: ${packageNames.join(", ")}`); + writeOutput( + `critical=${result.counts.critical} high=${result.counts.high} moderate=${result.counts.moderate} low=${result.counts.low}`, + ); + if (result.packageNames.length > 0) { + writeOutput(`advisory packages: ${result.packageNames.join(", ")}`); + } + + return result.ok ? 0 : 1; + } catch { + writeError("Production audit result is unreadable."); + return 1; } +}; + +const isMain = + process.argv[1] && + resolve(process.argv[1]) === fileURLToPath(import.meta.url); - if (!result.ok) process.exitCode = 1; -} catch { - console.error("Production audit result is unreadable."); - process.exitCode = 1; -} +if (isMain) process.exitCode = runProductionAudit(); diff --git a/scripts/check-production-audit.test.mjs b/scripts/check-production-audit.test.mjs new file mode 100644 index 00000000..2b10f579 --- /dev/null +++ b/scripts/check-production-audit.test.mjs @@ -0,0 +1,103 @@ +import assert from "node:assert/strict"; +import { describe, test } from "node:test"; + +import { + AUDIT_MAX_BUFFER_BYTES, + runProductionAudit, +} from "./check-production-audit.mjs"; + +const readableAudit = JSON.stringify({ + advisories: { + 1001: { module_name: "moderate-package" }, + }, + metadata: { + vulnerabilities: { critical: 0, high: 0, moderate: 1, low: 0 }, + }, +}); + +const captureReport = () => { + const output = []; + const errors = []; + return { + errors, + output, + writeError: (message) => errors.push(message), + writeOutput: (message) => output.push(message), + }; +}; + +describe("production audit command runner", () => { + test("runs pnpm with an explicit bounded audit-report buffer", () => { + const report = captureReport(); + let invocation; + + const exitCode = runProductionAudit({ + ...report, + spawn: (command, args, options) => { + invocation = { args, command, options }; + return { status: 1, stderr: "not reported", stdout: readableAudit }; + }, + }); + + assert.equal(AUDIT_MAX_BUFFER_BYTES, 16 * 1024 * 1024); + assert.deepEqual(invocation, { + command: "pnpm", + args: ["audit", "--prod", "--json"], + options: { encoding: "utf8", maxBuffer: AUDIT_MAX_BUFFER_BYTES }, + }); + assert.equal(exitCode, 0); + assert.deepEqual(report.output, [ + "critical=0 high=0 moderate=1 low=0", + "advisory packages: moderate-package", + ]); + assert.deepEqual(report.errors, []); + }); + + test("reports ENOBUFS without exposing subprocess output or errors", () => { + const report = captureReport(); + const exitCode = runProductionAudit({ + ...report, + spawn: () => ({ + error: Object.assign(new Error("ENOBUFS DEPENDENCY_TOKEN=secret"), { + code: "ENOBUFS", + }), + stderr: "DEPENDENCY_TOKEN=secret", + stdout: "raw audit output", + }), + }); + + assert.equal(exitCode, 1); + assert.deepEqual(report.output, []); + assert.deepEqual(report.errors, ["Production audit result is unreadable."]); + }); + + test("reports unreadable stdout without exposing raw diagnostics", () => { + const report = captureReport(); + const exitCode = runProductionAudit({ + ...report, + spawn: () => ({ + status: 1, + stderr: "DEPENDENCY_TOKEN=secret", + stdout: "WARN DEPENDENCY_TOKEN=secret", + }), + }); + + assert.equal(exitCode, 1); + assert.deepEqual(report.output, []); + assert.deepEqual(report.errors, ["Production audit result is unreadable."]); + }); + + test("reports a thrown spawn failure without exposing its message", () => { + const report = captureReport(); + const exitCode = runProductionAudit({ + ...report, + spawn: () => { + throw new Error("spawn failed with DEPENDENCY_TOKEN=secret"); + }, + }); + + assert.equal(exitCode, 1); + assert.deepEqual(report.output, []); + assert.deepEqual(report.errors, ["Production audit result is unreadable."]); + }); +}); diff --git a/scripts/production-audit-policy.mjs b/scripts/production-audit-policy.mjs index 120fcdc3..628bbf9f 100644 --- a/scripts/production-audit-policy.mjs +++ b/scripts/production-audit-policy.mjs @@ -3,28 +3,47 @@ const AUDIT_SEVERITIES = ["critical", "high", "moderate", "low"]; const unreadableAudit = () => new Error("Expected a readable pnpm audit JSON document."); +const findFinalObjectStart = (text) => { + let depth = 0; + let insideString = false; + + for (let index = text.length - 1; index >= 0; index -= 1) { + const character = text[index]; + if (character === '"') { + let precedingBackslashes = 0; + for ( + let escapeIndex = index - 1; + escapeIndex >= 0 && text[escapeIndex] === "\\"; + escapeIndex -= 1 + ) { + precedingBackslashes += 1; + } + if (precedingBackslashes % 2 === 0) insideString = !insideString; + } else if (!insideString && character === "}") { + depth += 1; + } else if (!insideString && character === "{") { + depth -= 1; + if (depth === 0) return index; + } + } + + return -1; +}; + export const parseProductionAudit = (rawAuditJson) => { if (typeof rawAuditJson !== "string") throw unreadableAudit(); const trimmed = rawAuditJson.trim(); - const objectStarts = []; - for (let index = trimmed.length - 1; index >= 0; index -= 1) { - if (trimmed[index] === "{") objectStarts.push(index); - } + const finalDocumentStart = findFinalObjectStart(trimmed); + if (finalDocumentStart === -1) throw unreadableAudit(); - for (const index of [0, ...objectStarts]) { - try { - const document = JSON.parse(trimmed.slice(index)); - if ( - document && - typeof document === "object" && - !Array.isArray(document) - ) { - return document; - } - } catch { - // Try the next object start so diagnostics may precede the final document. + try { + const document = JSON.parse(trimmed.slice(finalDocumentStart)); + if (document && typeof document === "object" && !Array.isArray(document)) { + return document; } + } catch { + // Normalize parser details so raw audit diagnostics are never propagated. } throw unreadableAudit(); @@ -43,17 +62,7 @@ const getSeverityCounts = (auditDocument) => { return counts; }; -export const evaluateProductionAudit = (rawAuditJson) => { - const counts = getSeverityCounts(parseProductionAudit(rawAuditJson)); - - return { - ok: counts.high === 0 && counts.critical === 0, - counts, - }; -}; - -export const getProductionAuditPackageNames = (rawAuditJson) => { - const auditDocument = parseProductionAudit(rawAuditJson); +const getPackageNames = (auditDocument) => { const advisoryNames = Object.values(auditDocument.advisories ?? {}).map( (advisory) => advisory?.module_name, ); @@ -65,3 +74,20 @@ export const getProductionAuditPackageNames = (rawAuditJson) => { .filter((name) => typeof name === "string" && name.length > 0) .sort((left, right) => left.localeCompare(right)); }; + +export const analyzeProductionAudit = (rawAuditJson) => { + const auditDocument = parseProductionAudit(rawAuditJson); + const counts = getSeverityCounts(auditDocument); + + return { + ok: counts.high === 0 && counts.critical === 0, + counts, + packageNames: getPackageNames(auditDocument), + }; +}; + +export const evaluateProductionAudit = (rawAuditJson) => { + const { counts, ok } = analyzeProductionAudit(rawAuditJson); + + return { ok, counts }; +}; diff --git a/scripts/production-audit-policy.test.mjs b/scripts/production-audit-policy.test.mjs index 02b91eda..e2ed8c90 100644 --- a/scripts/production-audit-policy.test.mjs +++ b/scripts/production-audit-policy.test.mjs @@ -2,18 +2,23 @@ import assert from "node:assert/strict"; import { describe, test } from "node:test"; import { + analyzeProductionAudit, evaluateProductionAudit, - getProductionAuditPackageNames, } from "./production-audit-policy.mjs"; -const auditDocument = ({ critical, high, moderate, low }) => ({ - advisories: { - 1001: { module_name: "example-high-package" }, - 1002: { module_name: "example-low-package" }, - }, +const auditDocument = ({ + critical, + high, + moderate, + low, + advisories = {}, + vulnerabilities = {}, +}) => ({ + advisories, metadata: { vulnerabilities: { critical, high, moderate, low }, }, + vulnerabilities, }); const fixtures = [ @@ -30,7 +35,8 @@ const fixtures = [ { name: "rejects a high finding after newline-delimited diagnostics", raw: [ - "WARN audit endpoint returned a diagnostic", + "WARN audit endpoint returned { retry: true }", + JSON.stringify({ level: "warn", context: { attempt: 2 } }), JSON.stringify( auditDocument({ critical: 0, high: 4, moderate: 2, low: 1 }), null, @@ -61,21 +67,66 @@ describe("production audit policy", () => { }); } - test("reports unique advisory package names without advisory details", () => { + test("analyzes unique package names from advisories and vulnerabilities", () => { const raw = JSON.stringify( - auditDocument({ critical: 0, high: 0, moderate: 1, low: 1 }), + auditDocument({ + critical: 0, + high: 0, + moderate: 1, + low: 1, + advisories: { + 1001: { module_name: "duplicate-package" }, + 1002: { module_name: "advisory-package" }, + }, + vulnerabilities: { + duplicate: { name: "duplicate-package" }, + vulnerability: { name: "vulnerability-package" }, + }, + }), ); - assert.deepEqual(getProductionAuditPackageNames(raw), [ - "example-high-package", - "example-low-package", - ]); + assert.deepEqual(analyzeProductionAudit(raw), { + ok: true, + counts: { critical: 0, high: 0, moderate: 1, low: 1 }, + packageNames: [ + "advisory-package", + "duplicate-package", + "vulnerability-package", + ], + }); }); - test("rejects an unreadable audit result", () => { - assert.throws( - () => evaluateProductionAudit("audit failed without JSON"), - /readable pnpm audit JSON document/, - ); - }); + for (const fixture of [ + { + name: "rejects an unreadable audit result", + raw: "audit failed without JSON", + }, + { + name: "rejects a malformed severity count", + raw: JSON.stringify( + auditDocument({ critical: 0, high: "4", moderate: 0, low: 0 }), + ), + }, + { + name: "rejects a missing severity count", + raw: JSON.stringify({ + metadata: { + vulnerabilities: { critical: 0, high: 0, moderate: 0 }, + }, + }), + }, + { + name: "rejects a negative severity count", + raw: JSON.stringify( + auditDocument({ critical: 0, high: 0, moderate: 0, low: -1 }), + ), + }, + ]) { + test(fixture.name, () => { + assert.throws( + () => evaluateProductionAudit(fixture.raw), + /readable pnpm audit JSON document/, + ); + }); + } }); From c9b30372a76f54a689a5e2fd2ddd4a4139c4d3d5 Mon Sep 17 00:00:00 2001 From: hmmhmmhm Date: Mon, 13 Jul 2026 22:40:17 +0900 Subject: [PATCH 05/78] fix(security): reject incomplete audit processes --- scripts/check-production-audit.mjs | 9 +++++- scripts/check-production-audit.test.mjs | 43 +++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/scripts/check-production-audit.mjs b/scripts/check-production-audit.mjs index 2180c2a7..27943354 100644 --- a/scripts/check-production-audit.mjs +++ b/scripts/check-production-audit.mjs @@ -17,7 +17,14 @@ export const runProductionAudit = ({ encoding: "utf8", maxBuffer: AUDIT_MAX_BUFFER_BYTES, }); - if (audit.error) throw audit.error; + if ( + audit.error || + audit.signal != null || + !Number.isInteger(audit.status) || + (audit.status !== 0 && audit.status !== 1) + ) { + throw new Error("Audit process did not complete normally."); + } const result = analyzeProductionAudit(audit.stdout); diff --git a/scripts/check-production-audit.test.mjs b/scripts/check-production-audit.test.mjs index 2b10f579..929ecac6 100644 --- a/scripts/check-production-audit.test.mjs +++ b/scripts/check-production-audit.test.mjs @@ -100,4 +100,47 @@ describe("production audit command runner", () => { assert.deepEqual(report.output, []); assert.deepEqual(report.errors, ["Production audit result is unreadable."]); }); + + for (const fixture of [ + { + name: "rejects a signaled audit process", + processResult: { + signal: "SIGTERM", + status: null, + stderr: "DEPENDENCY_TOKEN=signaled-secret", + stdout: readableAudit, + }, + }, + { + name: "rejects an abnormal audit process status", + processResult: { + signal: null, + status: 2, + stderr: "DEPENDENCY_TOKEN=status-secret", + stdout: readableAudit, + }, + }, + { + name: "rejects a missing audit process status", + processResult: { + signal: null, + stderr: "DEPENDENCY_TOKEN=missing-status-secret", + stdout: readableAudit, + }, + }, + ]) { + test(fixture.name, () => { + const report = captureReport(); + const exitCode = runProductionAudit({ + ...report, + spawn: () => fixture.processResult, + }); + + assert.equal(exitCode, 1); + assert.deepEqual(report.output, []); + assert.deepEqual(report.errors, [ + "Production audit result is unreadable.", + ]); + }); + } }); From 82e30e91c20351e198b0f7c4649cfa3af77569df Mon Sep 17 00:00:00 2001 From: hmmhmmhm Date: Mon, 13 Jul 2026 22:51:24 +0900 Subject: [PATCH 06/78] chore(api): upgrade and pin elysia runtime --- apps/api-ground-codes/package.json | 10 +- apps/api-ground-codes/src/endpoints/docs.ts | 4 +- pnpm-lock.yaml | 189 +++++++++++++------- scripts/api-runtime-pins.mjs | 62 +++++-- scripts/api-runtime-pins.test.mjs | 141 ++++++++++++++- scripts/check-api-runtime-pins.mjs | 19 +- scripts/update-api-runtime-pins.mjs | 16 +- 7 files changed, 335 insertions(+), 106 deletions(-) diff --git a/apps/api-ground-codes/package.json b/apps/api-ground-codes/package.json index 35df4a85..9e672964 100644 --- a/apps/api-ground-codes/package.json +++ b/apps/api-ground-codes/package.json @@ -13,19 +13,19 @@ "test": "bun test src/*.test.ts" }, "dependencies": { - "@elysiajs/cors": "^1.2.0", - "@elysiajs/static": "^1.2.0", - "@elysiajs/swagger": "^1.2.2", + "@elysiajs/cors": "1.4.2", + "@elysiajs/static": "1.4.10", + "@elysiajs/swagger": "1.3.1", "@ground-codes/geoint": "workspace:*", "@repo/codebook": "workspace:*", "@sinclair/typebox": "^0.34.30", - "elysia": "latest", + "elysia": "1.4.29", "ground-codes": "workspace:*", "pg": "^8.21.0" }, "devDependencies": { "@types/pg": "^8.20.0", - "bun-types": "latest" + "bun-types": "1.3.1" }, "module": "src/index.js" } diff --git a/apps/api-ground-codes/src/endpoints/docs.ts b/apps/api-ground-codes/src/endpoints/docs.ts index 7b28b342..3bb7f353 100644 --- a/apps/api-ground-codes/src/endpoints/docs.ts +++ b/apps/api-ground-codes/src/endpoints/docs.ts @@ -1,4 +1,4 @@ -import Elysia from "elysia"; +import Elysia, { type Context } from "elysia"; const docsHtml = ` @@ -161,7 +161,7 @@ const docsHtml = ` `; -const serveDocs = ({ set }: { set: { headers: Record } }) => { +const serveDocs = ({ set }: { set: Context["set"] }) => { set.headers["cache-control"] = "public, max-age=300"; set.headers["content-type"] = "text/html; charset=utf-8"; return docsHtml; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index a95428e5..c73e8c02 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -27,14 +27,14 @@ importers: apps/api-ground-codes: dependencies: '@elysiajs/cors': - specifier: ^1.2.0 - version: 1.2.0(elysia@1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2)) + specifier: 1.4.2 + version: 1.4.2(elysia@1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2)) '@elysiajs/static': - specifier: ^1.2.0 - version: 1.2.0(elysia@1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2)) + specifier: 1.4.10 + version: 1.4.10(elysia@1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2)) '@elysiajs/swagger': - specifier: ^1.2.2 - version: 1.2.2(elysia@1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2)) + specifier: 1.3.1 + version: 1.3.1(elysia@1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2)) '@ground-codes/geoint': specifier: workspace:* version: link:../../packages/geoint @@ -45,8 +45,8 @@ importers: specifier: ^0.34.30 version: 0.34.49 elysia: - specifier: latest - version: 1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2) + specifier: 1.4.29 + version: 1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2) ground-codes: specifier: workspace:* version: link:../../packages/ground-codes @@ -58,8 +58,8 @@ importers: specifier: ^8.20.0 version: 8.20.0 bun-types: - specifier: latest - version: 1.2.5 + specifier: 1.3.1 + version: 1.3.1(@types/react@19.0.10) apps/grok-spiral: dependencies: @@ -457,6 +457,9 @@ packages: resolution: {integrity: sha512-aA63XwOkcl4xxQa3HjPMqOP6LiK0ZDv3mUPYEFXkpHbaFjtGggE1A61FjFzJnB+p7/oy2gA8E+rcBNl/zC1tMg==} engines: {node: '>=6.9.0'} + '@borewit/text-codec@0.2.2': + resolution: {integrity: sha512-DDaRehssg1aNrH4+2hnj1B7vnUGEjU6OIlyRdkMd0aUdIUvKXrJfXsy8LVtXAy7DRvYVluWbMspsRhz2lcW0mQ==} + '@cesium/engine@25.0.0': resolution: {integrity: sha512-vJZfgAAB7WTvUPsI8dFV+wKleweCZgpJj1ckcTIERnC+NWvJ9bjlAx0xPl0We57NoQKcdMDy+6UiFjOvPB/yGg==} engines: {node: '>=22.0.0'} @@ -613,20 +616,20 @@ packages: resolution: {integrity: sha512-IchNf6dN4tHoMFIn/7OE8LWZ19Y6q/67Bmf6vnGREv8RSbBVb9LPJxEcnwrcwX6ixSvaiGoomAUvu4YSxXrVgw==} engines: {node: '>=12'} - '@elysiajs/cors@1.2.0': - resolution: {integrity: sha512-qsJwDAg6WfdQRMfj6uSMcDPSpXvm/zQFeAX1uuJXhIgazH8itSfcDxcH9pMuXVRX1yQNi2pPwNQLJmAcw5mzvw==} + '@elysiajs/cors@1.4.2': + resolution: {integrity: sha512-FTCcbH35brTLigF1W7BYySRZomgI/dBEMK9BgK9RP9Nez7zmpGh4koL/Yr1BFv8nYz7CfhRvcM8d/c+XnwMaVQ==} peerDependencies: - elysia: '>= 1.2.0' + elysia: '>= 1.4.0' - '@elysiajs/static@1.2.0': - resolution: {integrity: sha512-oLpAi8c+maPpA0XhhK3BELaIjIG+nXg/K9p8cFfW4q5ayRD59a3MOMOOGgpiXZkHJzLPWcouhhyyLAYtaANW4g==} + '@elysiajs/static@1.4.10': + resolution: {integrity: sha512-BN/3PZVWjqkL/6TM7RH69Pae89CfU0OqfEYPeBp2hB/GR5Euds39Wy+0ug9Vbvu++GG4tRRANf2QjaLnriJveg==} peerDependencies: - elysia: '>= 1.2.0' + elysia: '>= 1.4.0' - '@elysiajs/swagger@1.2.2': - resolution: {integrity: sha512-DG0PbX/wzQNQ6kIpFFPCvmkkWTIbNWDS7lVLv3Puy6ONklF14B4NnbDfpYjX1hdSYKeCqKBBOuenh6jKm8tbYA==} + '@elysiajs/swagger@1.3.1': + resolution: {integrity: sha512-LcbLHa0zE6FJKWPWKsIC/f+62wbDv3aXydqcNPVPyqNcaUgwvCajIi+5kHEU6GO3oXUCpzKaMsb3gsjt8sLzFQ==} peerDependencies: - elysia: '>= 1.2.0' + elysia: '>= 1.3.0' '@emnapi/runtime@1.10.0': resolution: {integrity: sha512-ewvYlk86xUoGI0zQRNq/mC+16R1QeDlKQy21Ki3oSYXNgLb45GV1P6A0M+/s6nyCuNDqe5VpaY84BzXGwVbwFA==} @@ -2374,6 +2377,13 @@ packages: '@tailwindcss/postcss@4.0.12': resolution: {integrity: sha512-r59Sdr8djCW4dL3kvc4aWU8PHdUAVM3O3te2nbYzXsWwKLlHPCuUoZAc9FafXb/YyNDZOMI7sTbKTKFmwOrMjw==} + '@tokenizer/inflate@0.4.1': + resolution: {integrity: sha512-2mAv+8pkG6GIZiF1kNg1jAjh27IDxEPKwdGul3snfztFerfPGI1LjDezZp3i7BElXompqEtPmoPx6c2wgtWsOA==} + engines: {node: '>=18'} + + '@tokenizer/token@0.3.0': + resolution: {integrity: sha512-OvjF+z51L3ov0OyAU0duzsYuvO01PH7x4t6DJx+guahgTnBHkhJdG7soQeTSFLWN3efnHyibZ4Z8l2EuWwJN3A==} + '@turbo/darwin-64@2.9.14': resolution: {integrity: sha512-t7QiPflaEyBE4oayeZtSmu4mEfjgIrcNlNNl1z1dmIVPqEdtA7+CfTf8d7KXsOGPh6aNgWjKxyvQg9uGfDQF+A==} cpu: [x64] @@ -2461,9 +2471,6 @@ packages: '@types/trusted-types@2.0.7': resolution: {integrity: sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw==} - '@types/ws@8.5.14': - resolution: {integrity: sha512-bd/YFLW+URhBzMXurx7lWByOu+xzU9+kb3RboOteXYDfW+tr+JZa99OyNmPINEGB/ahzKrEuc8rcv4gnpJmxTw==} - '@types/yoga-layout@1.9.2': resolution: {integrity: sha512-S9q47ByT2pPvD65IvrWp7qppVMpk9WGMbVq9wbWZOHg6tnXSD4vyhao6nOSBwwfDdV2p3Kx9evA9vI+XWTfDvw==} @@ -2716,8 +2723,10 @@ packages: buffer@6.0.3: resolution: {integrity: sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==} - bun-types@1.2.5: - resolution: {integrity: sha512-3oO6LVGGRRKI4kHINx5PIdIgnLRb7l/SprhzqXapmoYkFl5m4j6EvALvbDVuuBFaamB46Ap6HCUxIXNLCGy+tg==} + bun-types@1.3.1: + resolution: {integrity: sha512-NMrcy7smratanWJ2mMXdpatalovtxVggkj11bScuWuiOoXTiKIu2eVS1/7qbyI/4yHedtsn175n4Sm4JcdHLXw==} + peerDependencies: + '@types/react': ^19 bundle-require@5.1.0: resolution: {integrity: sha512-3WrrOuZiyaaZPWiEt4G3+IffISVC9HYlWueJEBWED4ZH4aIAC2PnkdnuRrR94M+w6yGWn4AglWtJtBI8YqvgoA==} @@ -2808,10 +2817,6 @@ packages: client-only@0.0.1: resolution: {integrity: sha512-IV3Ou0jSMzZrd3pZ48nLkT9DA7Ag1pnPzaiQhpW7c3RbcqqzvzzVu+L8gfqMp/8IM2MQtSiqaCxrrcfu8I8rMA==} - clone@2.1.2: - resolution: {integrity: sha512-3Pe/CF1Nn94hyhIYpjtiLhdCoEoz0DqQ+988E9gmeEdQZlojxnOb74wctFyuwWQHzqyf9X7C7MG8juUpqBJT8w==} - engines: {node: '>=0.8'} - clsx@2.1.1: resolution: {integrity: sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA==} engines: {node: '>=6'} @@ -2866,8 +2871,8 @@ packages: resolution: {integrity: sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==} engines: {node: '>= 0.6'} - cookie@1.0.2: - resolution: {integrity: sha512-9Kr/j4O16ISv8zBBhJoi4bXOYNTkFLOqSL3UDB0njXxCXNezjeyVrJyGOWtgfs/q2km1gwBcfH8q1yEGoMYunA==} + cookie@1.1.1: + resolution: {integrity: sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==} engines: {node: '>=18'} cross-spawn@7.0.6: @@ -2994,14 +2999,17 @@ packages: eastasianwidth@0.2.0: resolution: {integrity: sha512-I88TYZWc9XiYHRQ4/3c5rjjfgkjhLyW2luGIheGERbNQ6OY7yTybanSpDXZa8y7VUP9YmDcYa+eyq4ca7iLqWA==} - elysia@1.2.25: - resolution: {integrity: sha512-WsdQpORJvb4uszzeqYT0lg97knw1iBW1NTzJ1Jm57tiHg+DfAotlWXYbjmvQ039ssV0fYELDHinLLoUazZkEHg==} + elysia@1.4.29: + resolution: {integrity: sha512-GwMRGGwSdjfPt+w3LA0fqTuYJtS8uVRJicvoar98/HrO5qdFKDc9CwjIb6Kja+v39lkY+58hr2JvdR9jQzlUuA==} peerDependencies: - '@sinclair/typebox': '>= 0.34.0' + '@sinclair/typebox': '>= 0.34.0 < 1' + '@types/bun': '>= 1.2.0' + exact-mirror: '>= 0.0.9' + file-type: '>= 20.0.0' openapi-types: '>= 12.0.0' typescript: '>= 5.0.0' peerDependenciesMeta: - openapi-types: + '@types/bun': optional: true typescript: optional: true @@ -3380,6 +3388,14 @@ packages: resolution: {integrity: sha512-T1C0XCUimhxVQzW4zFipdx0SficT651NnkR0ZSH3yQwh+mFMdLfgjABVi4YtMTtaL4s168593DaoaRLMqryavA==} engines: {node: '>=18.0.0'} + exact-mirror@1.2.2: + resolution: {integrity: sha512-jUlRkZOPN3rCAy4kmZeD2HCnoXwJ6KlWNkEBJ3sEJ2rbLSU+tN60hY7XzPZkImRQOJczllvOBYTYEJTYScPh5Q==} + peerDependencies: + typebox: '>= 1.1.0' + peerDependenciesMeta: + typebox: + optional: true + execa@8.0.1: resolution: {integrity: sha512-VyhnebXciFV2DESc+p6B+y0LjSm0krU4OgJN44qFAhBY0TJ+1V61tYD2+wHusZ6F9n5K+vl8k0sTy7PEfV4qpg==} engines: {node: '>=16.17'} @@ -3398,6 +3414,9 @@ packages: resolution: {integrity: sha512-hMQ4CX1p1izmuLYyZqLMO/qGNw10wSv9QDCPfzXfyFrOaCSSoRfqE1Kf1s5an66J5JZC62NewG+mK49jOCtQew==} engines: {node: '>=4'} + fast-decode-uri-component@1.0.1: + resolution: {integrity: sha512-WKgKWg5eUxvRZGwW8FvfbaH7AXSh2cL+3j5fMGzUMCxWBJ3dV3a7Wz8y2f/uQ0e3B6WmodD3oS54jTQ9HVTIIg==} + fast-deep-equal@3.1.3: resolution: {integrity: sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==} @@ -3439,6 +3458,10 @@ packages: resolution: {integrity: sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ==} engines: {node: '>=16.0.0'} + file-type@22.0.1: + resolution: {integrity: sha512-ww5Mhre0EE+jmBvOXTmXAbEMuZE7uX4a3+oRCQFNj8w++g3ev913N6tXQz0XTXbueQ5TWQfm6BdaViEHHn8bhA==} + engines: {node: '>=22'} + fill-range@7.1.1: resolution: {integrity: sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==} engines: {node: '>=8'} @@ -4020,8 +4043,8 @@ packages: resolution: {integrity: sha512-eefp6IduNPT6fVdwPp+1NgD0PML1NU5P6j1Mj5nz1nidX8/sWY7119WL8vTAHgqfsY74TzW0w1XPgdYEKkGZ5A==} engines: {node: '>=10'} - memoirist@0.3.0: - resolution: {integrity: sha512-wR+4chMgVPq+T6OOsk40u9Wlpw1Pjx66NMNiYxCQQ4EUJ7jDs3D9kTCeKdBOkvAiqXlHLVJlvYL01PvIJ1MPNg==} + memoirist@0.4.0: + resolution: {integrity: sha512-zxTgA0mSYELa66DimuNQDvyLq36AwDlTuVRbnQtB+VuTcKWm5Qc4z3WkSpgsFWHNhexqkIooqpv4hdcqrX5Nmg==} merge-stream@2.0.0: resolution: {integrity: sha512-abv/qOcuPfk3URPfDzmZU1LKmuw8kT+0nIHvKrKgFrwifol/doWcdA4ZqsWQ8ENrFKkd67Mfpo/LovbIUsbt3w==} @@ -4163,10 +4186,6 @@ packages: node-addon-api@7.1.1: resolution: {integrity: sha512-5m3bsyrjFWE1xf7nz7YXdN4udnVtXK6/Yfgn5qnahL6bCkf2yKt4k3nuTKAtT4r3IG8JNR2ncsIMdZuAzJjHQQ==} - node-cache@5.1.2: - resolution: {integrity: sha512-t1QzWwnk4sjLWaQAS8CHgOJ+RAfmHpxFWmc36IWTiWHQfs0w5JDMBS1b1ZxQteo0vVVuWJvIUKHDkkeK7vIGCg==} - engines: {node: '>= 8.0.0'} - node-exports-info@1.6.0: resolution: {integrity: sha512-pyFS63ptit/P5WqUkt+UUfe+4oevH+bFeIiPPdfb0pFeYEu/1ELnJu5l+5EcTKYL5M7zaAa7S8ddywgXypqKCw==} engines: {node: '>= 0.4'} @@ -4796,6 +4815,10 @@ packages: resolution: {integrity: sha512-6fPc+R4ihwqP6N/aIv2f1gMH8lOVtWQHoqC4yK6oSDVVocumAsfCqjkXnqiYMhmMwS/mEHLp7Vehlt3ql6lEig==} engines: {node: '>=8'} + strtok3@10.3.5: + resolution: {integrity: sha512-ki4hZQfh5rX0QDLLkOCj+h+CVNkqmp/CMf8v8kZpkNVK6jGQooMytqzLZYUVYIZcFZ6yDB70EfD8POcFXiF5oA==} + engines: {node: '>=18'} + styled-jsx@5.1.6: resolution: {integrity: sha512-qSVyDTeMotdvQYoHWLNGwRFJHC+i+ZvdBRYosOFgC+Wg1vx4frN2/RG/NA7SYqqvKNLf39P2LSRA2pu6n0XYZA==} engines: {node: '>= 12.0.0'} @@ -4891,6 +4914,10 @@ packages: resolution: {integrity: sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==} engines: {node: '>=8.0'} + token-types@6.1.2: + resolution: {integrity: sha512-dRXchy+C0IgK8WPC6xvCHFRIWYUbqqdEIKPaKo/AcTUNzwLTK6AH7RjdLWsEZcAN/TBdtfUw3PYEgPr5VPr6ww==} + engines: {node: '>=14.16'} + topojson-client@3.1.0: resolution: {integrity: sha512-605uxS6bcYxGXw9qi62XyrV6Q3xwbndjachmNxu8HWTtVPxZfEJN9fd/SZS1Q54Sn2y0TMyMxFj/cJINqGHrKw==} hasBin: true @@ -4981,6 +5008,10 @@ packages: ufo@1.6.4: resolution: {integrity: sha512-JFNbkD1Svwe0KvGi8GOeLcP4kAWQ609twvCdcHxq1oSL8svv39ZuSvajcD8B+5D0eL4+s1Is2D/O6KN3qcTeRA==} + uint8array-extras@1.5.0: + resolution: {integrity: sha512-rvKSBiC5zqCCiDZ9kAOszZcDvdAHwwIKJG33Ykj43OKcWsnmcBRL09YTU4nOeHZ8Y2a7l1MgTd08SBe9A8Qj6A==} + engines: {node: '>=18'} + unbox-primitive@1.1.0: resolution: {integrity: sha512-nWJ91DjeOkej/TA8pXQ3myruKpKEYgqvpw9lz4OPHj/NWFNluYrjbz9j01CJ8yKQd2g4jFoOkINCTW2I5LEEyw==} engines: {node: '>= 0.4'} @@ -5185,6 +5216,8 @@ snapshots: dependencies: regenerator-runtime: 0.14.1 + '@borewit/text-codec@0.2.2': {} + '@cesium/engine@25.0.0': dependencies: '@cesium/wasm-splats': 0.1.0-alpha.2 @@ -5422,20 +5455,19 @@ snapshots: dependencies: '@jridgewell/trace-mapping': 0.3.9 - '@elysiajs/cors@1.2.0(elysia@1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2))': + '@elysiajs/cors@1.4.2(elysia@1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2))': dependencies: - elysia: 1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2) + elysia: 1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2) - '@elysiajs/static@1.2.0(elysia@1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2))': + '@elysiajs/static@1.4.10(elysia@1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2))': dependencies: - elysia: 1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2) - node-cache: 5.1.2 + elysia: 1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2) - '@elysiajs/swagger@1.2.2(elysia@1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2))': + '@elysiajs/swagger@1.3.1(elysia@1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2))': dependencies: '@scalar/themes': 0.9.75 '@scalar/types': 0.0.12 - elysia: 1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2) + elysia: 1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2) openapi-types: 12.1.3 pathe: 1.1.2 @@ -6694,6 +6726,15 @@ snapshots: postcss: 8.5.3 tailwindcss: 4.0.12 + '@tokenizer/inflate@0.4.1': + dependencies: + debug: 4.4.3 + token-types: 6.1.2 + transitivePeerDependencies: + - supports-color + + '@tokenizer/token@0.3.0': {} + '@turbo/darwin-64@2.9.14': optional: true @@ -6768,10 +6809,6 @@ snapshots: '@types/trusted-types@2.0.7': optional: true - '@types/ws@8.5.14': - dependencies: - '@types/node': 22.13.9 - '@types/yoga-layout@1.9.2': {} '@typescript-eslint/eslint-plugin@8.26.0(@typescript-eslint/parser@8.26.0(eslint@9.21.0(jiti@2.4.2))(typescript@5.8.2))(eslint@9.21.0(jiti@2.4.2))(typescript@5.8.2)': @@ -7080,10 +7117,10 @@ snapshots: base64-js: 1.5.1 ieee754: 1.2.1 - bun-types@1.2.5: + bun-types@1.3.1(@types/react@19.0.10): dependencies: '@types/node': 22.13.9 - '@types/ws': 8.5.14 + '@types/react': 19.0.10 bundle-require@5.1.0(esbuild@0.27.7): dependencies: @@ -7180,8 +7217,6 @@ snapshots: client-only@0.0.1: {} - clone@2.1.2: {} - clsx@2.1.1: {} color-convert@2.0.1: @@ -7222,7 +7257,7 @@ snapshots: cookie@0.7.2: {} - cookie@1.0.2: {} + cookie@1.1.1: {} cross-spawn@7.0.6: dependencies: @@ -7325,13 +7360,16 @@ snapshots: eastasianwidth@0.2.0: {} - elysia@1.2.25(@sinclair/typebox@0.34.49)(openapi-types@12.1.3)(typescript@5.8.2): + elysia@1.4.29(@sinclair/typebox@0.34.49)(exact-mirror@1.2.2)(file-type@22.0.1)(openapi-types@12.1.3)(typescript@5.8.2): dependencies: '@sinclair/typebox': 0.34.49 - cookie: 1.0.2 - memoirist: 0.3.0 - optionalDependencies: + cookie: 1.1.1 + exact-mirror: 1.2.2 + fast-decode-uri-component: 1.0.1 + file-type: 22.0.1 + memoirist: 0.4.0 openapi-types: 12.1.3 + optionalDependencies: typescript: 5.8.2 emoji-regex@10.4.0: {} @@ -7975,6 +8013,8 @@ snapshots: eventsource-parser@3.0.0: {} + exact-mirror@1.2.2: {} + execa@8.0.1: dependencies: cross-spawn: 7.0.6 @@ -7999,6 +8039,8 @@ snapshots: iconv-lite: 0.4.24 tmp: 0.0.33 + fast-decode-uri-component@1.0.1: {} + fast-deep-equal@3.1.3: {} fast-glob@3.3.1: @@ -8037,6 +8079,15 @@ snapshots: dependencies: flat-cache: 4.0.1 + file-type@22.0.1: + dependencies: + '@tokenizer/inflate': 0.4.1 + strtok3: 10.3.5 + token-types: 6.1.2 + uint8array-extras: 1.5.0 + transitivePeerDependencies: + - supports-color + fill-range@7.1.1: dependencies: to-regex-range: 5.0.1 @@ -8612,7 +8663,7 @@ snapshots: maybe-combine-errors@1.0.0: {} - memoirist@0.3.0: {} + memoirist@0.4.0: {} merge-stream@2.0.0: {} @@ -8757,10 +8808,6 @@ snapshots: node-addon-api@7.1.1: {} - node-cache@5.1.2: - dependencies: - clone: 2.1.2 - node-exports-info@1.6.0: dependencies: array.prototype.flatmap: 1.3.3 @@ -9483,6 +9530,10 @@ snapshots: strip-json-comments@3.1.1: {} + strtok3@10.3.5: + dependencies: + '@tokenizer/token': 0.3.0 + styled-jsx@5.1.6(react@19.0.0): dependencies: client-only: 0.0.1 @@ -9566,6 +9617,12 @@ snapshots: dependencies: is-number: 7.0.0 + token-types@6.1.2: + dependencies: + '@borewit/text-codec': 0.2.2 + '@tokenizer/token': 0.3.0 + ieee754: 1.2.1 + topojson-client@3.1.0: dependencies: commander: 2.20.3 @@ -9683,6 +9740,8 @@ snapshots: ufo@1.6.4: {} + uint8array-extras@1.5.0: {} + unbox-primitive@1.1.0: dependencies: call-bound: 1.0.4 diff --git a/scripts/api-runtime-pins.mjs b/scripts/api-runtime-pins.mjs index e9a00cd3..8140630b 100644 --- a/scripts/api-runtime-pins.mjs +++ b/scripts/api-runtime-pins.mjs @@ -2,6 +2,18 @@ export const RUNTIME_REPOSITORY = "git+https://github.com/hmmhmmhm/ground.codes.git"; export const WORKSPACE_DEPENDENCY = "workspace:*"; +export const API_RUNTIME_VERSION_PINS = { + dependencies: { + elysia: "1.4.29", + "@elysiajs/cors": "1.4.2", + "@elysiajs/static": "1.4.10", + "@elysiajs/swagger": "1.3.1", + }, + devDependencies: { + "bun-types": "1.3.1", + }, +}; + export const RUNTIME_PACKAGES = [ { name: "ground-codes", @@ -32,20 +44,32 @@ export const usesWorkspaceRuntime = (packageJson) => ({ name }) => packageJson.dependencies?.[name] === WORKSPACE_DEPENDENCY, ); +export const getExactRuntimePinFailures = (packageJson) => + Object.entries(API_RUNTIME_VERSION_PINS).flatMap(([dependencyGroup, pins]) => + Object.entries(pins).flatMap(([name, expected]) => { + const actual = packageJson[dependencyGroup]?.[name]; + return actual === expected + ? [] + : [`${name} expected exact ${expected}, found ${actual ?? "missing"}`]; + }), + ); + export const getRuntimePinFailures = (packageJson, tag) => - usesWorkspaceRuntime(packageJson) - ? [] - : RUNTIME_PACKAGES.flatMap((runtimePackage) => { - const expected = buildPinnedDependency(tag, runtimePackage); - const actual = packageJson.dependencies?.[runtimePackage.name]; - return actual === expected - ? [] - : [ - `${runtimePackage.name} expected ${expected} or ${WORKSPACE_DEPENDENCY}, found ${ - actual ?? "missing" - }`, - ]; - }); + getExactRuntimePinFailures(packageJson).concat( + usesWorkspaceRuntime(packageJson) + ? [] + : RUNTIME_PACKAGES.flatMap((runtimePackage) => { + const expected = buildPinnedDependency(tag, runtimePackage); + const actual = packageJson.dependencies?.[runtimePackage.name]; + return actual === expected + ? [] + : [ + `${runtimePackage.name} expected ${expected} or ${WORKSPACE_DEPENDENCY}, found ${ + actual ?? "missing" + }`, + ]; + }), + ); export const getStrictRuntimePinFailures = (packageJson, tag) => RUNTIME_PACKAGES.flatMap((runtimePackage) => { @@ -62,6 +86,7 @@ export const getStrictRuntimePinFailures = (packageJson, tag) => export const updateRuntimePins = (packageJson, tag) => { packageJson.dependencies ??= {}; + packageJson.devDependencies ??= {}; let changed = false; for (const runtimePackage of RUNTIME_PACKAGES) { @@ -72,5 +97,16 @@ export const updateRuntimePins = (packageJson, tag) => { } } + for (const [dependencyGroup, pins] of Object.entries( + API_RUNTIME_VERSION_PINS, + )) { + for (const [name, nextValue] of Object.entries(pins)) { + if (packageJson[dependencyGroup][name] !== nextValue) { + packageJson[dependencyGroup][name] = nextValue; + changed = true; + } + } + } + return changed; }; diff --git a/scripts/api-runtime-pins.test.mjs b/scripts/api-runtime-pins.test.mjs index 072271ff..f57a4eb6 100644 --- a/scripts/api-runtime-pins.test.mjs +++ b/scripts/api-runtime-pins.test.mjs @@ -1,4 +1,5 @@ import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; import { describe, test } from "node:test"; import { @@ -10,7 +11,114 @@ import { usesWorkspaceRuntime, } from "./api-runtime-pins.mjs"; +const EXACT_RUNTIME_PINS = { + dependencies: { + elysia: "1.4.29", + "@elysiajs/cors": "1.4.2", + "@elysiajs/static": "1.4.10", + "@elysiajs/swagger": "1.3.1", + }, + devDependencies: { + "bun-types": "1.3.1", + }, +}; + +const readJson = (url) => JSON.parse(readFileSync(url, "utf8")); + +const apiPackageJsonUrl = new URL( + "../apps/api-ground-codes/package.json", + import.meta.url, +); + +const buildPackageJson = () => ({ + dependencies: { + "ground-codes": "workspace:*", + "@ground-codes/geoint": "workspace:*", + "@repo/codebook": "workspace:*", + ...EXACT_RUNTIME_PINS.dependencies, + }, + devDependencies: { ...EXACT_RUNTIME_PINS.devDependencies }, +}); + +const compareExactVersions = (left, right) => { + const leftParts = left.split(".").map(Number); + const rightParts = right.split(".").map(Number); + + for (let index = 0; index < 3; index += 1) { + if (leftParts[index] !== rightParts[index]) { + return leftParts[index] - rightParts[index]; + } + } + + return 0; +}; + describe("API runtime package pins", () => { + test("pins the reviewed Elysia runtime group to exact versions", () => { + const packageJson = readJson(apiPackageJsonUrl); + + assert.equal(packageJson.dependencies.elysia, "1.4.29"); + assert.equal(packageJson.devDependencies["bun-types"], "1.3.1"); + assert.equal(packageJson.dependencies["@elysiajs/cors"], "1.4.2"); + assert.equal(packageJson.dependencies["@elysiajs/static"], "1.4.10"); + assert.equal(packageJson.dependencies["@elysiajs/swagger"], "1.3.1"); + }); + + test("rejects moving and unbounded Elysia and Bun type pins", () => { + for (const dependencyName of ["elysia", "bun-types"]) { + for (const invalidPin of [ + "latest", + "*", + "^1.4.0", + ">=1.4.0", + "github:elysiajs/elysia#main", + ]) { + const packageJson = buildPackageJson(); + const dependencyGroup = + dependencyName === "bun-types" + ? packageJson.devDependencies + : packageJson.dependencies; + dependencyGroup[dependencyName] = invalidPin; + + assert.deepEqual( + getRuntimePinFailures(packageJson, "railway-api-runtime-20260519"), + [ + `${dependencyName} expected exact ${ + dependencyName === "bun-types" ? "1.3.1" : "1.4.29" + }, found ${invalidPin}`, + ], + ); + } + } + }); + + test("uses plugin releases whose declared peers include Elysia 1.4.29", () => { + const pluginPeers = { + "@elysiajs/cors": ">= 1.4.0", + "@elysiajs/static": ">= 1.4.0", + "@elysiajs/swagger": ">= 1.3.0", + }; + + for (const [pluginName, expectedPeerRange] of Object.entries(pluginPeers)) { + const pluginPackageJson = readJson( + new URL( + `../apps/api-ground-codes/node_modules/${pluginName}/package.json`, + import.meta.url, + ), + ); + const expectedPluginVersion = EXACT_RUNTIME_PINS.dependencies[pluginName]; + const peerRange = pluginPackageJson.peerDependencies?.elysia; + const minimumPeerVersion = peerRange?.match( + /^>=\s+(\d+\.\d+\.\d+)$/, + )?.[1]; + + assert.equal(pluginPackageJson.version, expectedPluginVersion); + assert.equal(peerRange, expectedPeerRange); + assert.ok(minimumPeerVersion); + assert.ok(compareExactVersions("1.4.29", minimumPeerVersion) >= 0); + } + }); + test("builds git dependency pins for all runtime packages", () => { assert.deepEqual(RUNTIME_PACKAGES, [ { @@ -54,13 +162,7 @@ describe("API runtime package pins", () => { }); test("accepts workspace runtime dependencies for monorepo deployments", () => { - const packageJson = { - dependencies: { - "ground-codes": "workspace:*", - "@ground-codes/geoint": "workspace:*", - "@repo/codebook": "workspace:*", - }, - }; + const packageJson = buildPackageJson(); assert.equal(usesWorkspaceRuntime(packageJson), true); assert.deepEqual( @@ -76,13 +178,34 @@ describe("API runtime package pins", () => { "@ground-codes/geoint": "workspace:*", "@repo/codebook": "workspace:*", elysia: "latest", + "@elysiajs/cors": "^1.2.0", + "@elysiajs/static": "^1.2.0", + "@elysiajs/swagger": "^1.2.2", + }, + devDependencies: { + "bun-types": "latest", }, }; - const updated = updateRuntimePins(packageJson, "railway-api-runtime-20260519"); + const updated = updateRuntimePins( + packageJson, + "railway-api-runtime-20260519", + ); assert.equal(updated, true); - assert.equal(packageJson.dependencies.elysia, "latest"); + assert.deepEqual(packageJson.dependencies, { + "ground-codes": + "git+https://github.com/hmmhmmhm/ground.codes.git#railway-api-runtime-20260519&path:packages/ground-codes", + "@ground-codes/geoint": + "git+https://github.com/hmmhmmhm/ground.codes.git#railway-api-runtime-20260519&path:packages/geoint", + "@repo/codebook": + "git+https://github.com/hmmhmmhm/ground.codes.git#railway-api-runtime-20260519&path:packages/codebook", + ...EXACT_RUNTIME_PINS.dependencies, + }); + assert.deepEqual( + packageJson.devDependencies, + EXACT_RUNTIME_PINS.devDependencies, + ); assert.deepEqual( getRuntimePinFailures(packageJson, "railway-api-runtime-20260519"), [], diff --git a/scripts/check-api-runtime-pins.mjs b/scripts/check-api-runtime-pins.mjs index 828c2e33..85a07f8b 100644 --- a/scripts/check-api-runtime-pins.mjs +++ b/scripts/check-api-runtime-pins.mjs @@ -13,13 +13,14 @@ const packageJson = JSON.parse( ); const runtimeTag = process.env.API_RUNTIME_TAG ?? getRuntimeTag(packageJson); +const workspaceRuntime = + usesWorkspaceRuntime(packageJson) && !process.env.API_RUNTIME_TAG; -const failures = - usesWorkspaceRuntime(packageJson) && !process.env.API_RUNTIME_TAG - ? [] - : runtimeTag - ? getRuntimePinFailures(packageJson, runtimeTag) - : ["unable to infer API runtime tag from package pins"]; +const failures = workspaceRuntime + ? getRuntimePinFailures(packageJson, "workspace") + : runtimeTag + ? getRuntimePinFailures(packageJson, runtimeTag) + : ["unable to infer API runtime tag from package pins"]; if (failures.length > 0) { console.error("API runtime package pins are not aligned:"); @@ -28,7 +29,7 @@ if (failures.length > 0) { } console.log( - usesWorkspaceRuntime(packageJson) && !process.env.API_RUNTIME_TAG - ? "API runtime packages use workspace dependencies." - : `API runtime package pins are aligned on ${runtimeTag}.`, + workspaceRuntime + ? "API runtime packages use workspace dependencies with exact Elysia pins." + : `API runtime package pins and exact Elysia pins are aligned on ${runtimeTag}.`, ); diff --git a/scripts/update-api-runtime-pins.mjs b/scripts/update-api-runtime-pins.mjs index be25767e..a5a8d715 100644 --- a/scripts/update-api-runtime-pins.mjs +++ b/scripts/update-api-runtime-pins.mjs @@ -1,6 +1,9 @@ import { readFileSync, writeFileSync } from "node:fs"; -import { updateRuntimePins } from "./api-runtime-pins.mjs"; +import { + getRuntimePinFailures, + updateRuntimePins, +} from "./api-runtime-pins.mjs"; const tag = process.argv[2]?.trim(); @@ -15,11 +18,18 @@ const packageJsonUrl = new URL( ); const packageJson = JSON.parse(readFileSync(packageJsonUrl, "utf8")); const changed = updateRuntimePins(packageJson, tag); +const failures = getRuntimePinFailures(packageJson, tag); + +if (failures.length > 0) { + console.error("Refusing to write incomplete API runtime pins:"); + for (const failure of failures) console.error(`- ${failure}`); + process.exit(1); +} writeFileSync(packageJsonUrl, `${JSON.stringify(packageJson, null, 2)}\n`); console.log( changed - ? `API runtime package pins updated to ${tag}.` - : `API runtime package pins already use ${tag}.`, + ? `API runtime and exact Elysia package pins updated to ${tag}.` + : `API runtime and exact Elysia package pins already use ${tag}.`, ); From 7b1a1717b0127557b628c7ddb75fdbe6e2f24ced Mon Sep 17 00:00:00 2001 From: hmmhmmhm Date: Mon, 13 Jul 2026 23:18:14 +0900 Subject: [PATCH 07/78] chore(web): remediate production dependency advisories --- apps/web/package.json | 4 +- docs/security/dependency-audit.md | 84 +++++ package.json | 9 + pnpm-lock.yaml | 376 ++++++++++++++++------- scripts/production-audit-policy.test.mjs | 49 +++ 5 files changed, 404 insertions(+), 118 deletions(-) create mode 100644 docs/security/dependency-audit.md diff --git a/apps/web/package.json b/apps/web/package.json index e3ee2c44..8be8aece 100644 --- a/apps/web/package.json +++ b/apps/web/package.json @@ -24,9 +24,9 @@ "@react-google-maps/api": "^2.20.8", "@repo/ui": "workspace:*", "@types/google.maps": "^3.64.1", - "cesium": "^1.141.0", + "cesium": "1.143.0", "next": "15.5.18", - "next-intl": "^4.0.2", + "next-intl": "4.13.2", "react": "^19.0.0", "react-dom": "^19.0.0", "react-icons": "^5.6.0" diff --git a/docs/security/dependency-audit.md b/docs/security/dependency-audit.md new file mode 100644 index 00000000..42ef434c --- /dev/null +++ b/docs/security/dependency-audit.md @@ -0,0 +1,84 @@ +# Production Dependency Audit + +Audit date: 2026-07-13 + +## Policy + +Production dependency changes must leave `pnpm security:audit` with zero +critical and zero high-severity findings. High and critical findings are never +waived, including findings introduced only through transitive dependencies. A +remaining moderate finding must have a recorded dependency path, runtime +applicability analysis, mitigating control, and concrete follow-up issue. + +## Remediation + +The Web runtime dependencies are pinned exactly: + +| Package | Previous manifest range | Audited version | +| ----------- | ----------------------- | --------------- | +| `cesium` | `^1.141.0` | `1.143.0` | +| `next-intl` | `^4.0.2` | `4.13.2` | + +The root lock policy uses selector-scoped overrides so only the vulnerable +transitive resolutions are replaced: + +| Vulnerable resolution | Locked replacement | Rationale | +| --------------------- | ------------------ | ------------------------------------------------------------------------------------------------------ | +| `dompurify@3.4.2` | `3.4.12` | Replaces the vulnerable Cesium engine resolution without changing other DOMPurify lines. | +| `picomatch@2.3.1` | `2.3.2` | Replaces the vulnerable matcher resolution used below workspace UI tooling. | +| `postcss@8.4.31` | `8.4.49` | Replaces the audited vulnerable Next.js PostCSS resolution while retaining Next.js 15.5 compatibility. | +| `postcss@8.5.3` | `8.5.19` | Moves the Tailwind PostCSS resolution to an audited release. | +| `protobufjs@8.2.0` | `8.7.1` | Replaces the vulnerable Cesium engine/runtime serialization resolution. | + +`pnpm-lock.yaml` records the exact direct versions and override results. The +production-audit policy contract also verifies that none of the five vulnerable +transitive resolution keys can return to the lock graph. + +## Audit Result + +| State | Critical | High | Moderate | Low | Packages reported | +| ------------------ | -------: | ---: | -------: | --: | -------------------------------------------------------------- | +| Before remediation | 0 | 2 | 12 | 3 | `dompurify`, `next-intl`, `picomatch`, `postcss`, `protobufjs` | +| After remediation | 0 | 0 | 1 | 0 | `postcss` | + +The after-remediation gate passes because critical and high are both zero. The +remaining moderate is documented below; it is not a high/critical waiver. + +### Remaining moderate: PostCSS + +- Package and advisory: `postcss@8.4.49`, + [GHSA-qx2v-qp2m-jg93](https://github.com/advisories/GHSA-qx2v-qp2m-jg93) / + CVE-2026-41305, CVSS 6.1. Versions below `8.5.10` are affected by unescaped + `` output when attacker-controlled CSS is parsed, stringified, and + embedded in an HTML `