Skip to content

metrics-baseline: schema + encryption surface - #5

Merged
inkatze merged 9 commits into
mainfrom
metrics-baseline-schema
Apr 9, 2026
Merged

metrics-baseline: schema + encryption surface#5
inkatze merged 9 commits into
mainfrom
metrics-baseline-schema

Conversation

@inkatze

@inkatze inkatze commented Apr 9, 2026

Copy link
Copy Markdown
Owner

Summary

Implements tasks 1 and 2 of the metrics-baseline spec (#4). Producer script and first snapshot will follow on this same branch.

  • specs/metrics-baseline/snapshots/SCHEMA.md — defines every field a baseline-YYYY-MM.md must contain, with the three required dimensions (project / machine / main-thread vs subagent) declared once up front and applied section by section. Covers all required metrics from requirements.md (corpus scope, tool volumes, top tools split main/subagent, per-tool error rate, friction tallies with edit_old_string_mismatch as its own line, permission prompts, stuck loops, features, slash commands with success rate, MCP, hot-file re-reads, outcomes, interaction style) plus the optional signals and the mandatory methodology section.
  • specs/metrics-baseline/snapshots/recipients.txt — personal SSH pubkey (sourced from 1Password id_personal, matches git config user.signingkey).
  • specs/metrics-baseline/snapshots/README.md — documents the age -R recipients.txt encrypt flow and the decrypt flow (exporting the private key from 1Password to a tempfile, since age can't use the SSH agent).
  • lefthook.yml — new baseline-plaintext-guard pre-commit hook that rejects any staged plaintext baseline-*.md under snapshots/ with a loud error. Verified by staging a dummy plaintext file and confirming the hook exits 1.

Draft because the producer script, the first baseline-2026-04.md.age artifact, and the project_improvement_plan memory cross-link still need to land on this branch before merge.

Test plan

  • Lefthook guard verified locally (staged plaintext baseline-test.md, hook rejected with expected message)
  • Existing lefthook checks (yamllint, ansible-lint, syntax-check) still pass on the lefthook.yml edit
  • Schema review: confirm every required metric from requirements.md is represented with the right dimensions
  • README review: confirm encrypt/decrypt commands are correct for the 1Password-backed key workflow
  • Follow-up commits on this branch: producer script, encrypted baseline-2026-04.md.age, memory cross-link

inkatze added 2 commits April 8, 2026 22:35
- recipients.txt with personal SSH pubkey (from 1Password id_personal)
- snapshots/README.md documenting age encrypt/decrypt flow
- lefthook pre-commit guard rejecting staged plaintext baseline-*.md

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements the “schema + encryption surface” pieces of the metrics-baseline spec by defining a required snapshot schema, documenting the encrypt/decrypt workflow, adding an age recipients file, and enforcing “no plaintext snapshots” via a pre-commit hook.

Changes:

  • Add snapshots/SCHEMA.md defining the baseline snapshot fields and methodology section.
  • Add snapshots/recipients.txt plus snapshots/README.md documenting age + 1Password key workflow.
  • Add a lefthook pre-commit guard to block committing plaintext baseline-*.md snapshots.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
specs/metrics-baseline/snapshots/SCHEMA.md Defines the baseline snapshot schema, dimensions, and methodology requirements.
specs/metrics-baseline/snapshots/recipients.txt Adds an SSH public key recipient for age encryption.
specs/metrics-baseline/snapshots/README.md Documents snapshot encryption/decryption flows and recipient management.
lefthook.yml Adds a pre-commit guard to reject staged plaintext snapshot files.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md
Comment thread lefthook.yml Outdated
…path

Tighten SCHEMA dimensions policy back to requiring every volume and
friction metric to be split by project, machine, and thread. Add the
missing by_project/by_machine/by_thread encodings to sections 3, 4, 5,
6, 7, and 13 so producers have one canonical shape per metric.
Rename top_tools.main_thread to top_tools.by_thread.main for naming
consistency with the rest of the schema. Fix the lefthook plaintext
guard message to point at specs/metrics-baseline/snapshots/README.md.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread specs/metrics-baseline/snapshots/SCHEMA.md
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
…fields

Rename per_* to by_* in Section 1 for consistency with all other
sections. Add missing by_project/by_machine fields to Sections 8-12
so every claimed dimension has a concrete field definition.

@inkatze inkatze left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two issues found, both in SCHEMA.md, both fixed in 3cfee59:

  1. concern — naming inconsistency (line 34-36): Section 1 used per_project/per_machine/per_thread while every other section used by_*. Renamed to by_* for consistency.

  2. concern — missing dimension fields (Sections 8-12): These sections claimed dimensions (project/machine) in their footer lines but didn't define corresponding by_* fields in the table. Added the missing fields so the producer script has concrete field names to implement.

No blockers. LGTM with the above addressed.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread specs/metrics-baseline/snapshots/SCHEMA.md

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Sections 8 (features), 9 (slash commands), 10 (MCP usage),
11 (hot-file re-reads), and 12 (outcomes) were missing the
required thread dimension split. Add explicit by_thread fields
to each section to align with the mandatory three-dimension
requirement in requirements.md, design.md, and test-spec.md.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread specs/metrics-baseline/snapshots/SCHEMA.md
Comment thread specs/metrics-baseline/snapshots/README.md
Comment thread lefthook.yml
…counts

Add by_project, by_machine, and by_thread breakdowns to
conversation_counts.daily to satisfy the requirements that all
conversation-count metrics be emitted under all three dimensions.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
Comment thread specs/metrics-baseline/snapshots/SCHEMA.md Outdated
…nting

Add explicit zero-day padding requirement for daily dimensional series
(by_project, by_machine, by_thread) to match the parent daily field.

Clarify stuck_loops.by_tool counting semantics: a session increments
every tool bucket it has qualifying runs for, so the sum may exceed
the total session count.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread specs/metrics-baseline/snapshots/README.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@inkatze
inkatze marked this pull request as ready for review April 9, 2026 20:24
@inkatze
inkatze merged commit e33fec2 into main Apr 9, 2026
12 of 14 checks passed
@inkatze
inkatze deleted the metrics-baseline-schema branch April 9, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants