Every scanner vendor claims to catch everything. I wanted a way to check: honest, reproducible, and where the ground truth isn’t up for debate. So I built Guinea Pig: one CWE per directory, every vulnerable fixture paired with its remediation, a synthetically-generated catalogue of 1,028 secrets, and a benchmark that scores recall, false positives and noise. Here’s how it’s built, and what a late-June run actually said.

A build log and a tutorial for slit-scan photography: the one idea behind it, an algorithm I developed at night against Blender renders, two ways to shoot it for real (a train sliding past, a flower spinning on a home-made turntable), and a small side quest where I scraped the SNCF API so I’d stop wasting Saturdays waiting for a TGV.

I asked an agent to fix vulnerabilities and prove each fix works. It reached for rm -rf / and a reverse shell to do the proving. This is the story of how I stopped it: a safety contract baked into the skill, plus a purpose-built canary service that lets a test demonstrate an exploit path without ever firing a real one.

How I spent a few nights turning a pile of security scanners into one engine, and then let an agent do the part everyone gives up on: triage, prioritization, and writing the fix with a test that proves it.