runner-cleanup: event-driven _diag/pages wipe via systemd hooks#925
Conversation
The hourly runner-cleanup timer eventually clears stale page files,
but a runner that restarts within that hour still hits:
The file '/home/actions-runner-NN/_diag/pages/<uuid>.log' already exists.
Add two small pieces that turn this into an event-driven cleanup:
- runner-clean-pages: tiny helper, wipes $HOME/_diag/pages/ for
whatever user systemd invoked it as. Logs to journald via
systemd-cat. Idempotent.
- install-runner-hooks: one-shot installer. Finds every
/etc/systemd/system/actions.runner.*.service unit on the host
and drops a 10-clean-pages.conf override that adds:
ExecStartPre=-<helper>
ExecStopPost=-<helper>
The '-' prefix means cleanup failures don't block the runner.
ExecStopPost fires on every stop reason — clean exit, SIGTERM,
SIGKILL, OOM, machine reboot — so the page file is gone before the
next job lands. ExecStartPre covers the case where systemd was
bypassed (hard power-cut leaving stale pages on disk).
No long-lived daemon: systemd's own service supervision IS the
event hook. Hourly runner-cleanup keeps running as the catch-all
fallback.
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Extension of the event-driven cleanup added two commits ago. The
helper + installer were shipped but had to be deployed by hand.
Wire them into the runner-cleanup install path so a single
`armbian-config module_armbian_runners install` now puts everything
on the host in one shot:
- /usr/local/sbin/runner-cleanup (existing, hourly)
- /etc/systemd/system/runner-cleanup.timer (existing)
- /usr/local/sbin/runner-clean-pages (new, event-driven helper)
- 10-clean-pages.conf drop-ins for every actions.runner.*.service
(new, written by install-runner-hooks invocation)
The install-runner-hooks invocation runs AFTER daemon-reload so the
new drop-ins take effect on the next runner restart. Existing runner
units that started moments earlier in the same install pass don't
have the hooks active yet; they pick them up on their next restart
(operator or systemd-initiated). Hourly runner-cleanup keeps
covering the gap.
Non-fatal: install-runner-hooks failure prints a warning and the
runner-cleanup timer setup proceeds.
The bug, again
Even with hourly `runner-cleanup` (PRs #919/#920), `actions-runner-01` keeps hitting:
```
Error: The file '/home/actions-runner-01/_diag/pages/.log' already exists.
```
The hourly timer eventually clears it, but a runner that restarts within that hour still collides on a stale page UUID. We need event-driven cleanup.
What this adds
Two tiny pieces in `tools/modules/system/runner-cleanup/`:
The `-` prefix means cleanup failures never block the runner unit itself.
Why systemd hooks, not a daemon
`ExecStopPost` fires on every stop reason — graceful exit, SIGTERM, SIGKILL, OOM, machine reboot. `ExecStartPre` covers the case where systemd was bypassed (hard power-cut). Together they form an event-driven cleanup chain with zero long-lived daemon process — systemd's own supervision IS the daemon.
The hourly `runner-cleanup` timer continues to run as the catch-all fallback for whatever the hooks miss.
Deployment
On each runner host, after pulling the updated configng:
```bash
sudo install -m 0755 tools/modules/system/runner-cleanup/runner-clean-pages /usr/local/bin/
sudo bash tools/modules/system/runner-cleanup/install-runner-hooks
```
(or just copy both files to /usr/local/bin and run the installer)
Test plan