docs(spec): add metrics baseline spec (metrics-baseline) - #4
Conversation
Captures a structured, encrypted-at-rest baseline of Claude Code usage metrics so improvement-plan items can be re-measured against a fixed reference point. Schema covers per-project, per-machine, and main vs subagent dimensions; includes permission prompts, hook failures by name, slash command success rate, hot-file re-reads, conversation outcomes, and subagent type breakdown. Snapshots are encrypted with age using SSH-key recipients; schema and spec files stay plaintext. Designed so both full-data and anonymous HTML reports can be regenerated from the same snapshot by a deferred renderer.
There was a problem hiding this comment.
Pull request overview
Adds a new specs/metrics-baseline/ sibling spec that defines how to capture, encrypt, verify, and later re-measure a baseline snapshot of Claude Code usage metrics for attribution against improvement-plan changes.
Changes:
- Introduces baseline requirements (dimensions/metrics), design decisions, implementation task order, and a manual verification checklist.
- Specifies encrypted-at-rest snapshot storage using
age+ SSH recipients and a plaintext-protection guard.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| specs/metrics-baseline/requirements.md | Defines required dimensions/metrics, encryption/storage constraints, and re-measurement protocol requirements |
| specs/metrics-baseline/design.md | Documents rationale and key design choices (dimensions, encryption model, snapshot/report relationship) |
| specs/metrics-baseline/tasks.md | Provides an ordered implementation plan for schema, encryption surface, first snapshot, verification, and cross-linking |
| specs/metrics-baseline/test-spec.md | Manual verification checklist covering schema conformance, encryption hygiene, sanity checks, and diff hygiene |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Summary
Adds
specs/metrics-baseline/as a new sibling spec tospecs/claude-context/. Defines a structured, encrypted-at-rest baseline of Claude Code usage metrics so the items inproject_improvement_plan(memory) can be re-measured against a fixed reference point and deltas attributed to specific changes.What's in the spec
old_stringmismatches and hook failures broken down by hook name as their own lines), permission prompts (approve/deny per tool), stuck-loop sessions, subagent type breakdown, slash command usage and success rate, MCP usage, hot-file re-reads, conversation outcomes, interaction style indicators.baseline-YYYY-MM.md.ageusingagewith SSH-key recipients in plaintextrecipients.txt. Schema and spec files stay plaintext. Pre-commit guard rejects staged plaintextbaseline-*.md.Why now
This blocks the implementation of
claude-contextand other improvement-plan items whose effects should be measurable. Without a tracked baseline, "did this help?" stays unanswerable.Test plan
requirements.mdand confirm every dimension and metric is one you actually want measured.design.mdand confirm the encryption choice (age + SSH recipients, file-level) and the deferred-renderer / snapshot-as-data split are correct.tasks.mdand confirm the implementation order (schema → encryption surface → first snapshot → verify → cross-link → commit) matches your intended workflow.test-spec.mdand confirm the verification gates (schema conformance, encryption hygiene, sanity against memory numbers) are sufficient.