cron: Enable CDN purging on the production batch plane - #92
Merged
Merged
Conversation
Task 9.7 repointed this plane at the production corpus but left the worker's empty `SCORECARD_API_BASE_URL` in place. That override existed so a plane writing `-test` buckets could not invalidate a CDN it had not published to, and its own commit scoped it to "until task 9.7". `getPurger` returns a no-op client on an empty base URL before it ever reads `FASTLY_PURGE_TOKEN`, so results would have been written and never purged; served responses carry an `age` well past their `max-age`, so the purge is what invalidates, not the TTL. Dropping the override rather than restating the URL here keeps the corpus a run writes and the CDN it purges in one file. An env var wins silently over `config-aws.yaml`, the documented rollback switch, so a rollback to `-test` would have kept purging production. `TestAWSOverlayBucketsAreTheProductionCorpus` now asserts `api-base-url` beside the four bucket names so the pair cannot drift. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Stephen Augustus <foo@auggie.dev>
Every failure mode on this path is silent. Purge errors are logged at Info and swallowed, so a run whose every purge failed still reports success; with purging switched off, `getPurger` returns a no-op client and logs nothing at all. The writes succeed either way, so the bucket looks correct while users are served stale scores. The assertion compares what the CDN returns against what the origin returns behind a cache-busting query string. Reading `age` would be weaker: entries have been observed more than 22 hours old under a 600s `max-age`, so it tracks POP routing more than invalidation. A preflight refuses to scan unless purging is wired up -- no env override on the worker, `api-base-url` set, token present -- so a misconfiguration names itself instead of looking like a purge that failed. The probe has to be a repository `getResults` serves from the cron bucket rather than a self-published Action result, which the date shape distinguishes. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Stephen Augustus <foo@auggie.dev>
Task 9.7 repointed the deployed ConfigMap at the production corpus, and this script writes wherever that points, so its header describing runs that "land in the four test buckets" had been wrong since. The closing hint printed a `-test` bucket that the run had not touched. The distinction matters more than a stale noun. data2 and rawdata are keyed by run and only add a prefix, but cron-results holds the latest-result pointers the live API serves, so every repository in the CSV becomes visible to users at whatever the run scores it -- and, now that purging is wired up, without waiting for a CDN entry to age out. The hint now reads the bucket back from the ConfigMap instead of carrying a second copy of the name, which is what let it drift. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Stephen Augustus <foo@auggie.dev>
Task 9.7 is marked complete, so the defect it missed belongs beside it rather than in a new task: the plane was repointed at the production corpus while the worker kept the empty `SCORECARD_API_BASE_URL` that existed only to stop a `-test` plane purging a CDN it had not published to. Results were written correctly and never made visible. Recorded with the evidence, since "the writes succeeded" is exactly what made this hard to see from the outside. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Stephen Augustus <foo@auggie.dev>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #92 +/- ##
==========================================
+ Coverage 42.71% 48.52% +5.81%
==========================================
Files 48 58 +10
Lines 2568 3229 +661
==========================================
+ Hits 1097 1567 +470
- Misses 1383 1504 +121
- Partials 88 158 +70
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Task 9.7 (#91) repointed the batch plane at the production corpus but left its
counterpart in
cron/k8s/worker.yaml: the emptySCORECARD_API_BASE_URLaddedwhile this plane wrote
-testbuckets, and scoped by its own commit to "untiltask 9.7".
getPurgerreturns a no-op client on an empty base URL before itreads
FASTLY_PURGE_TOKEN, so every result was written correctly and none wasever invalidated on the CDN.
Observed on the live service before the fix:
github.com/octocat/Hello-Worldwas rewritten in
ossf-scorecard-cron-resultsat 23:13 UTC with today's date,while the API served its
2026-08-24predecessor atage: 80717. The samerequest behind a cache-busting query string returned the new record — so the
origin was correct and only invalidation was broken. Note
agefar exceeds themax-age: 600the service sends, which is why purging, not TTL, is what makesa result visible.
The override is removed rather than set to the production URL. An env var
wins silently over
config-aws.yaml, which is the documented rollback switch,so a rollback to
-testwould have kept purging production. With it gone, thecorpus a run writes and the CDN it purges are named in one file and switched by
one apply.
TestAWSOverlayBucketsAreTheProductionCorpusnow assertsapi-base-urlbeside the four bucket names, so the pair cannot drift in eitherdirection.
Also here:
scripts/verification/verify-cdn-purge.sh— asserts CDN output against originoutput rather than reading
age, and refuses to run unless purging isactually wired up. Every failure on this path is otherwise silent: purge
errors are logged at Info and swallowed, and a disabled purger logs nothing.
run-sample-inventory.sh— its header still described runs landing in the-testbuckets and its closing hint named a bucket the run had not touchedsince 9.7. The hint now reads the bucket back from the ConfigMap instead of
carrying a second copy of the name.
Related
openspec/changes/provision-cron-awstask 9.7 — correction recorded intasks.mdbeside the existing completion note.How it was tested
make build,make test,make lint— all clean.api-base-urlfails withapi-base-url: got "", want "https://api.scorecard.dev". A guard that cannotfail is not a guard.
verify-cdn-purge.shrun against the live cluster correctly fails itspreflight (
worker sets SCORECARD_API_BASE_URL='') and exits withoutscanning; the same extraction against the fixed manifest returns empty, so
the positive path passes once this is applied.
kubectl apply --dry-run=serverreportsdeployment.apps/scorecard-batch-worker configured.shellcheckclean on both scripts.Not yet applied to the cluster. The manifest change is applied directly, but
merging keeps
:mainand the deployed state from diverging — the controller'sworker-updatesidecar restarts workers onto:mainon every scheduled run.Review focus
Whether dropping the env var is preferable to setting it explicitly. The
argument for dropping is the rollback interaction described above; the argument
against is that the manifest no longer shows the value. A comment on the
FASTLY_PURGE_TOKENblock covers the second point.Checklist
git commit -s)make buildandmake testpassmake lintis cleanBatch pipeline (
cron/):cron/andinternal/in either directioncron/internal/formatunchangedScan inventories: not touched.