ci(checks): a job that builds or runs an image of Docker Hub has the mirror as a second source - #448

A job that builds or runs an image of Docker Hub gets a second source for it, a mirror, so that a limit or a fault of Docker Hub does not turn the checks red.

This changes three workflow files, and adds one shared step, its script and a test file. No Dockerfile changes, no secret is used, and no setting of the repository changes. What starts being checked is below. Nothing stops being checked.

What users will notice

Nothing.

Why

• On 2026-10-09, from 21:06 to at least 21:31 UTC, Docker Hub refused pulls from the runners ("429 Too Many Requests ... You have reached your unauthenticated pull rate limit"), and its token service did not answer ("504 Gateway Timeout" and timeouts from auth.docker.io). Docker's status page showed no incident.
• 14 jobs build or run an image of Docker Hub. Five of them are required checks: the jobs of the install matrix, which run a distro image. Those five took ci(tests): the tests that need a running service and no model run on every pull request #444, ci(canary): the weekly check reads the source of each required check and the age of the newest night #446 and ci(replay): a change to text that every request carries needs its smoke result in the pull request #432 out of the merge queue, and each had to be queued again by hand. No pull request was at fault.
• A pull request from a fork (docs(ja): resync README + ARCHITECTURE with the English versions (#204) #439) was red in the same minutes.

What changes

One step, .github/actions/docker-hub-mirror, at the start of each of the 14 jobs, right after the checkout. It does two things:

1. It sets the one key registry-mirrors in the settings file of the runner's Docker service, to https://mirror.gcr.io, and starts the service again. A build then asks the mirror first and Docker Hub second. When the service does not come back, or does not name the mirror, the step is red with its fix and no later step runs. The step writes that one settings file and no other: the script scripts/docker_hub_mirror.py opens no file, it reads the text of the settings and prints the text with the mirror.
2. It pulls the images that the job runs with docker run by the mirror's own name, and gives them the name of Docker Hub. When the mirror does not give an image, the step says so and the job pulls it from Docker Hub, as before.

The four PR build check jobs have a builder of their own. They set it up in two steps: the first names the builder's own image at the mirror and may fail without failing the job; the second is the step of before and runs only when the first failed. Both give the builder the mirror for its base images.

Why the mirror in the settings is not enough: the runs by hand

The first design was the setting alone. A try showed that it does not carry docker run. Each run is on the branch try/docker-hub-mirror, which is not part of this pull request, with the addresses of Docker Hub pointed to nowhere in the runner's hosts file.

One thing is for the tries alone: the builder had the runner's own network there, because that is how the block reaches the builder's container.

What starts being checked

tests/infrastructure/test_docker_hub_mirror.py reads the workflow files:

• Every job that builds or runs an image for a pull request or in the merge queue has the step, after its checkout and before its first docker command. The four jobs that publish images, or start on a schedule or by hand, are listed in the test, each with the condition that keeps it from a pull request.
• The step is given the very images of Docker Hub that the job's docker run commands name, with the values of the matrix. An image that is run and not given would have Docker Hub as its one source again, with no sign.
• No job has a container: or services: image of Docker Hub. GitHub pulls such an image before the first step, so no step can help it.
• A builder job has the two setup steps, in that form.
• The script sets the one key and keeps every other key; for settings that are not one JSON object it prints nothing. It opens no file.
• The step writes the settings file of the Docker service and no other file. The one other name it takes is a file in the folder for temporary files, which only its tests set. Eight other names are refused before anything is read: a path outside that folder, a path that climbs out of it, another file of the service's folder, a relative name. The tests of the refusal run with a stand-in for sudo that runs nothing, so a broken refusal makes a test red and still touches no file.
• The new settings file has the mode 0644, and the file that the step makes on the way is removed.
• The step, run with stand-ins for the service and for docker: a service that does not start again, that gives no answer, or that does not name the mirror ends the step with an error that says the fix; an image that the mirror does not give does not.

Proof

• The new test file, the test files that read the same workflow files and the gates page, and the contract tests pass three times: with two Pythons, and with the names that a runner gives its jobs set (742 tests each time). The whole infrastructure and contract suites, in the container: 2,229 passed, none failed.
• 64 planted changes in the script, the step, the three workflow files and the gates page, each tried alone against the new test file, in a Linux container with a memory limit and a runner's names in its environment: each of the 64 turns at least one test red. 2 controls stay green.
• actionlint and zizmor find nothing in the three workflow files and in the step. shellcheck finds nothing in the shell of the step. The static gates of the lint job, the size check and the workflow YAML gate pass.
• The integrity check names ten things for a maintainer: five lint notes on the lines that use the step (zizmor asks for a form that actionlint does not read yet, as on the two upload steps), the one continue-on-error of the first setup step, and four files that configure the checks.

What changed after the first push

• Sonar named one thing in the first push, as a blocker of its security rules: the script took the path of the settings file from its command line and wrote there. No outside input reached it, and it was still a script that writes where its caller says. The script now takes no path at all, and the step, the one place that names the file, refuses every other file.
• One test of another file was red on the runner: tests/infrastructure/test_dev_results.py holds that the tests workflow uses two actions of this repository, the two that send results. The mirror step is a third one and sends nothing; the test now names it. The local runs of the first push had not run that file.

Cost and limits

• Each of the 14 jobs takes about 10 seconds more, for the start of the Docker service (9.7 s in a try).
• The five jobs of the install matrix run the mirror's copy of each tag, which can be older than the one of Docker Hub. The step prints the digest of each image that it took from the mirror. The four tags are not pinned by digest: the install matrix is there to try the distro images as they are.
• An image that a Dockerfile pins by its digest is the same image from the mirror: Docker checks the digest.
• One more outside service is in the path, as the first of two sources. On a day when the mirror and Docker Hub both give no answer, these jobs are red as before.
• When the mirror gives no image for a builder, its first setup step shows as failed in the job, and the job goes on with the second.
• The jobs that publish images are not changed.
• The codeql jobs were red that evening too, and for another cause: they ended after their entry had left the merge queue, and their upload found no branch. That is not changed here.
• The tests run the step with stand-ins. That a runner still behaves as in the runs by hand is shown by the jobs themselves, in each run.