Skip to content

ci: stop exporting the docker-smoke layer cache on pull requests - #506

Merged
cdeust merged 1 commit into
mainfrom
fix/docker-smoke-cache-export
Sep 8, 2026
Merged

cdeust merged 1 commit into
mainfrom
fix/docker-smoke-cache-export

Conversation

@cdeust

@cdeust cdeust commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Problem

Docker Smoke (bare-container DB-less contract) went red on PR #492 — cancelled at its 13-minute bound before the smoke-test body ran. Same signature as the first attempt on #503.

Reading the job log (run 34225008423, job 102057139452) rather than guessing: the image built successfully (#20 exporting to docker image format … DONE 70.8s, image loaded). What then consumed the remaining ~12 minutes was the buildx cache export:

#22 exporting to GitHub Actions Cache
#22 preparing build cache for export 46.3s done
#22 sending cache export
#22 writing layer sha256:eadf4b3c… 38.8s done
#22 writing layer sha256:fc852874…
##[error]The operation was canceled.       ← 12:31:31, t+827s

So the PR was marked red by a step that asserts nothing about the code under review.

Measurement

Same job, same workflow, two contexts:

Run Context Build step Whole job
34217523633 job 102032799571 push to main, warm cache 225 s 242 s ✅
34225008423 job 102057139452 PR #492, cold mode=max export build done at 70.8 s >827 s, cancelled ❌

The existing # source: on timeout-minutes: 13 cited "run 33951959734 (2026-09-05), max 369s". Today's 827 s observation invalidates 369 s as an upper bound.

Root cause

cache-to: type=gha,mode=max ran on every event. GitHub Actions scopes a cache entry to the ref that created it:`` the default branch's entries are readable from every branch, but an entry a PR run writes is readable only by that same PR's later runs. This job runs once per head commit — so the PR-written entry is exported and then never read. Pure cost on the critical path.

mode=max also exports every intermediate build stage, including the ~421 MB layer visible in the log above.

Fix

Skip the export on pull_request only:

cache-from: type=gha
cache-to: ${{ github.event_name != 'pull_request' && 'type=gha,mode=max' || '' }}
  • cache-from stays unconditional — PRs keep reading main's warm cache, so the torch CPU-wheel layer is still not re-downloaded. The stated intent of the existing comment block is preserved.
  • Pushes to main still populate the cache, in mode=max, at the measured 242 s steady state.
  • The two sibling Docker Build jobs already use mode=min; this job was the odd one out.

The condition is deliberately written != 'pull_request' && <value> || '' rather than == 'pull_request' && '' || <value>: GitHub expressions return the operand rather than a boolean and the empty string is falsy, so the latter form falls through to <value> on every event and disables nothing. That pitfall is called out in a comment beside the line.

timeout-minutes: 13 is kept and re-sourced to the warm-path steady state it actually bounds, recording explicitly that the cold export is removed rather than budgeted for.

What is NOT changed

The BLOCKING contract this job asserts is untouched — it still builds the bare repo and runs tools/list with zero env vars and zero external services. No continue-on-error, no skip, no test disabled.

Verification

  • actionlint v1.7.12 (the version pinned in ci.yml, checksum-verified release binary) — clean on ci.yml
  • python scripts/check_ci_gate_complete.py — OK (no job names change)
  • python scripts/check_craftsmanship.py --base origin/main — OK
  • python scripts/check_doc_claims.py, check_version_surfaces.py — OK
  • ruff check ., ruff format --check . — clean
  • YAML parses

The real proof is this PR's own Docker Smoke run: it exercises the pull_request branch of the new condition.

🤖 Generated with Claude Code

https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1


Generated by Claude Code

The Docker Smoke job wrote its buildx layer cache with
`cache-to: type=gha,mode=max` on every event. On run 34225008423
(PR #492) the image build finished at 70.8s and the job was still in
`exporting to GitHub Actions Cache` when the 13-minute bound fired at
827s. The smoke-test body never ran, so the PR went red on a step that
asserts nothing about the code under review.

GitHub Actions scopes a cache entry to the ref that created it: the
default branch's entries are readable from every branch, but an entry a
PR run writes is readable only by that same PR's later runs. This job
runs once per head commit, so a PR-written entry is exported and then
never read — pure cost on the critical path.

Skip the export on `pull_request`. `cache-from` stays unconditional, so
PRs keep reading main's warm cache and the ~200MB torch CPU-wheel layer
is still not re-downloaded; on run 34217523633 (push to main, warm
cache) the whole job took 242s with the export included.

The `timeout-minutes: 13` comment cited "run 33951959734, max 369s" as
its basis; today's 827s observation invalidates that as an upper bound.
It is re-sourced to the warm-path steady state it actually bounds, and
records explicitly that the cold `mode=max` export is removed rather
than budgeted for.

The two sibling Docker Build jobs already used `mode=min`; this job was
the odd one out.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1
@cdeust
cdeust marked this pull request as ready for review September 8, 2026 12:50
@cdeust
cdeust merged commit 54dfb16 into main Sep 8, 2026
28 checks passed
cdeust pushed a commit that referenced this pull request Sep 8, 2026
Brings in #506 (docker-smoke cache export skipped on pull_request), the
fix for the Docker Smoke timeout that was the only red check on this
branch's last run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1
pull Bot pushed a commit to asleekgeek/Cortex that referenced this pull request Sep 8, 2026
Thirteen merged PRs (cdeust#475-cdeust#487, cdeust#506) and a full SCI-grounded energy
harness under benchmarks/energy/ had no mention anywhere in the README.
A reader had no way to know the programme existed, which also meant no
way to hold it to its own evidence rule.

The section states what the harness measures and, at equal length, what
it does not: its own README opens by saying the fixtures do not measure
device energy and do not establish an energy improvement, the carbon
factors are mandatory operator inputs with no defaults, and the boundary
is the sensor's CPU+GPU+ANE estimate excluding memory, storage, screen
and power-supply losses.

No energy figure is published, and the section says why: no energy
results are committed to this repository, because a number measured on
one operator's machine, region and duty cycle is not a property of the
software. The demand-reduction paragraph is explicitly labelled a design
rationale rather than a measurement, since no end-to-end token-savings
or CO2 figure has been measured.

The one quantitative claim is the already-committed hook boot constant
(~0.05s vs ~0.6s registry import, measured 2026-07-28), cited to its
call sites in mcp_server/hooks/auto_recall.py.

check_doc_claims.py exits 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01StMBvNd7eVJGtpnC2zNsx1
@cdeust
cdeust deleted the fix/docker-smoke-cache-export branch September 8, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants