Skill Validation Report

Validating skill: /home/runner/work/icskills/icskills/skills/deploy-to-cloud-engine

Structure

• Pass: SKILL.md found

Frontmatter

• Pass: name: "deploy-to-cloud-engine" (valid)
• Pass: description: (973 chars)
• Pass: license: "Apache-2.0"
• Pass: compatibility: (127 chars)
• Pass: metadata: (2 entries)

Markdown

• Pass: no unclosed code fences found

Tokens

Content Analysis

Contamination Analysis

• Warning: Language mismatch: config (1 category differ from primary)

Result: passed

Project Checks

✓ Project checks passed for 1 skills (0 warnings)

Skill and evals review — deploy-to-cloud-engine

Good skill. Covers genuinely new territory (nothing in icp-cli handles the --auth identity link, subnet targeting, or __META_* console metadata), end-to-end verified, and the without-skill baseline confirms the delta is real — baseline invents dfx deploy --network ic for every flow question. Validation passes; triggers are perfect 9/9 / 8/8.

I ran the full suite. Three failures in the WITH-skill results, analysed below.

Required before merge

1. printf '\n' | — the "Press Enter" wording will cause silent failures

Step 1 says "supply the Enter keypress (pipe an empty line)". On icp-cli 0.3.2 (verified in PR #217 agent-web-identity), redirecting stdin from /dev/null produces a bare EOF that does not satisfy the prompt — the command sits on Press Enter to log in indefinitely, no browser opens, and the flow wastes the first sign-in attempt. Piping an actual newline is the only reliable fix:

printf '\n' | icp identity link web <your-identity-name> --auth <console-origin>

Update both the Step 1 prose and Pitfall 1 to use this form, and add an explicit note: do NOT redirect from /dev/null.

2. Eval 2 — prompt/expected_behavior mismatch (false failure)

Prompt: "Give me only the command … No deploy steps."
Expected behavior: "Mentions a browser tab opens and the user completes the Internet Identity sign-in there."

These contradict. Asking for only the command and then checking for surrounding context is a design bug — the failure here is in the eval, not the skill. Two options: (a) remove that expected_behavior, or (b) drop the "No deploy steps" scope restriction and reword to "Give me the link command and what to expect when I run it."

3. Pitfall 6 too thin — causing real eval 8 failure (3/4)

The failure in "Adversarial: dfx to the engine": when correcting dfx misuse, the agent produced icp identity link web without <name> and --auth <console-origin>. Pitfall 6 currently reads only "This ecosystem uses icp, never dfx." — not enough to anchor the full command signature under adversarial conditions. Strengthen it to show the complete correction:

6. **Using `dfx`.** This ecosystem uses `icp`, never `dfx`. The correct sequence is:
   `icp identity link web <name> --auth <console-origin>` (Step 1), then
   `icp deploy -e ic --subnet <subnet-id>` (Step 3).

Suggested improvements (not blocking)

Missing inline icp.yaml example in Step 2. The text says "For a single inline icp.yaml, put the same block under each canister entry instead" but shows no YAML. Since agents use code blocks directly, prose-only guidance is a gap for the common case of single-file projects. Add a snippet:

# icp.yaml
canisters:
  frontend:
    recipe: ...
    settings:
      environment_variables:
        __META_PROJECT: "My App"
        __META_NAME: "Frontend"
        __META_MAIN_CANISTER: "true"
  backend:
    recipe: ...
    settings:
      environment_variables:
        __META_PROJECT: "My App"
        __META_NAME: "Backend"

Eval 10 — name the command explicitly. The failure was the agent said "redeploy" without naming icp deploy. Change the expected_behavior to check that the agent says icp deploy specifically — it matters that the right command is named.

Pitfall 4 — weak, not a failure mode. "Assuming a fixed identity name" is a clarification about a placeholder value, not a failure path. The identity name is already explained in Step 1. Either drop this pitfall or fold the note into the Step 1 inline explanation.

Prerequisites — strip the install detail. The npm install -g @icp-sdk/icp-cli @icp-sdk/ic-wasm, Node.js >= 22, and demo-icp fallback instructions duplicate what lives in icp-cli. This skill already defers there for everything else — trim to: "icp on $PATH — see the icp-cli skill to install."

Future cross-reference. PR #217 agent-web-identity covers icp identity link web --app <domain> for acting as a user's app-specific principal (oisy.com, NNS, etc.) — a distinct use case from --auth here. Once that lands, add it to ## Related Skills.

Thanks for the thorough review and for running the suite — all items addressed in 73d87d4.

Required

1. printf newline — Step 1 now shows printf '\n' | icp identity link web <name> --auth <console-origin> with an explicit "do not redirect stdin from /dev/null" note (bare EOF never satisfies the prompt). Pitfall 1 and the stall eval use the same form.
2. Eval 2 — took option (b): the prompt is now "Give me the command to link my icp CLI to my cloud engine console at https://opencloud.org, and what to expect when I run it. No deploy steps." (case renamed to "Link command and what to expect").
3. dfx pitfall — now anchors the full correction: icp identity link web <name> --auth <console-origin> (Step 1), then icp deploy -e ic --subnet <subnet-id> (Step 3 — deploy moved to Step 3 when the __META_* step landed as Step 2).

Suggestions

• Inline icp.yaml example — added, with one correction to the snippet from the review: per the v0.3.0 icp-yaml-schema.json, canisters is an array of {name, recipe, settings} items, not a map keyed by canister name. The map form would fail validation.
• Eval 10 — the expected behavior now requires naming icp deploy explicitly rather than accepting a bare "redeploy".
• Pitfall 4 — dropped; the placeholder explanation is folded into the Step 1 link instruction where the name is introduced.
• Prerequisites — trimmed to defer installation to the icp-cli skill, keeping only the icp --version check and the 0.3.0 verification note.
• PR Add agent-web-identity skill #217 cross-reference — agreed; will add agent-web-identity to Related Skills once it lands.

The three WITH-skill failures from your run map to required items 2–3 plus the eval-10 wording, so the suite should be re-run on the new state — checklist updated accordingly.