Session-start sweep quarantines legitimate skill packs on plain-text matches in docs #5

Summary

The SessionStart sweep (verify session) moves any skill folder with a single HIGH heuristic finding into ${CLAUDE_PLUGIN_DATA}/quarantine/. The HIGH checks are plain-text pattern matches over every .md/.json/.ts/.js/.sh file in the folder, documentation included. Ordinary Solana docs trip them: a mention of ~/.config/solana/id.json, an official curl … | bash installer line, or curl in one paragraph and pnpm add -D (lowercased to -d ) in another.

Once quarantined, a pack is quarantined again every time it comes back, and verify approve restores it to the wrong place, where the next sweep quarantines it again.

Affected version

v1.0.0, verified at commit 2997280 (current main).

Where in the code (at 2997280)

• crates/engine/src/verify/heuristics.rs:15-16: scanned types include md, json, ts, js. :31-49: scan walks every file in the folder.
• heuristics.rs:122-131 with :226-237: secret_read (HIGH) if the lowercased file contains id.json, .config/solana, keypair.json, private_key, secret_key, …
• heuristics.rs:150-156 with :265-275: download_and_run (HIGH) if the file contains curl/wget anywhere and | sh/| bash anywhere, not necessarily in the same command.
• heuristics.rs:110-119 with :240-262: telemetry_post (HIGH) if the file contains curl/fetch anywhere and -d , --data or -x post anywhere.
• crates/engine/src/verify/mod.rs:229-237: eval_skill quarantines on any HIGH finding. There's no warn-only path.
• verify/mod.rs:502-508 (skill folders) and :579-594 (ext/<name> submodules): scan, then quarantine. :594 also drops the pin.
• verify/mod.rs:390-397: quarantine is keyed by folder name only (quarantine/<name>). The original path isn't recorded, and an existing entry with the same name is deleted (remove_dir_all, :394-396) before the move.
• crates/engine/src/main.rs:324-331: verify approve <name> restores into the first verify_skills_dirs entry, which is ~/.claude/skills by default (policy.rs:371, policy/default.policy.yaml:63), not the project's .claude/skills/ext/. It re-pins with scan: "approved" (main.rs:336-347), but nothing reads that marker, so the next sweep quarantines the restored folder again.
• Same defaults: the sweep also scans the user's personal ~/.claude/skills, even when the plugin is only enabled for one project.
• Docs vs code: README.md:74 and :130 describe "pin first-seen SHA, quarantine on drift", but first-seen content with a HIGH finding is quarantined instead of pinned. CHANGELOG.md:34 documents verify approve ext/<name>, but quarantined packs are stored as quarantine/<name> (verify/mod.rs:574-585), so that form can't find them.

Reproduction

A folder that contains only documentation:

mkdir -p demo/.claude/skills/docs-only
printf 'Load the payer with signerFromFile("~/.config/solana/id.json").\n' > demo/.claude/skills/docs-only/SKILL.md
safe-ai-skill verify check demo/.claude/skills/docs-only

Following heuristics.rs:122-131, this gives a HIGH secret_read finding and verdict block. At the next session start in demo/, the sweep moves the folder to quarantine/docs-only. (We derived this from the code and didn't run it against the binary.)

Real packs: we ported the HIGH checks to a small static script. It doesn't run the engine, and this wasn't a live session. Run over the ext/ packs pinned by solanabr/ai-kit, it flags 18 of 20 packs at ai-kit main (e52cfe2), including both packs the kit installs by default. Only eth-to-sol and solana-game pass. At ai-kit 4892cd0, which pinned 18 packs, it flagged 16 of 18. Examples:

• safe-solana-builder (9e94436), references/native-rust.md: its only HIGH hit is telemetry_post, from the Solana CLI install line sh -c "$(curl -sSfL https://release.anza.xyz/stable/install)" (L313) plus pnpm add -D solana-bankrun anchor-bankrun (L359).
• solana-dev (bb24c39):
  • secret_read on signerFromFile('~/.config/solana/id.json') (references/kit/overview.md L48).
  • download_and_run on the Surfpool installer curl -sL https://run.surfpool.run/ | bash (references/surfpool/overview.md L42).
  • telemetry_post on a local RPC call curl -X POST http://127.0.0.1:8899 (references/surfpool/cheatcodes.md L84).

Impact

• solanabr/ai-kit requires this plugin since feat: make safe-ai-skill a core part of the kit ai-kit#68. In an ai-kit project, the first session start moves the kit's default packs (solana-dev, safe-solana-builder) out of .claude/skills/ext/. In a clone of ai-kit with all its submodules, it moves all 18 flagged packs, and git then shows those submodules as deleted.
• Restoring a pack (git submodule update, the kit's pack installer) just gets it quarantined again at the next start. verify approve moves it into ~/.claude/skills/, where it gets quarantined again.
• Personal skills in ~/.claude/skills are moved out too. A later quarantine of any folder with the same name deletes the earlier quarantined copy.

Suggested fix

• Match only executable or instruction content (fenced shell blocks, scripts, package.json scripts), and require the tool and the marker to be in the same command, not anywhere in the file.
• Add an allowlist (name plus hash or SHA) and/or a warn-only mode. Make verify approve stick for the approved hash.
• Record each quarantined folder's original path and restore it there. Don't overwrite an existing quarantine entry.
• Consider scanning ~/.claude/skills only when the plugin is enabled at user scope.