-
Notifications
You must be signed in to change notification settings - Fork 1
Home
Measuring AI implementation quality of Figma designs through controlled experiments.
Setup — Install and configure canicode (CLI, MCP token, Claude Code skills) Decision Log — Every major decision with context and rationale Rule Reference — Complete detection conditions, subTypes, and exemptions for all 16 rules Scoring Model — How scores are calculated, severity levels, grade thresholds Calibration — Pipeline structure, agents, debate highlights, fixture analysis Delta-driven Pipeline — How calibration + rule discovery close the feedback loop Calibration 01 — First full calibration: 10/24 fixtures, responsive comparison, avg 93.7% similarity Calibration 02 — size-constraints responsive strip: 6/24 fixtures, missing-size-constraint -5→-8, raw-value -3→-4
| # | Title | Key Finding | Date |
|---|---|---|---|
| 01 | Ablation Pilot | ΔT is noise, ΔO is the signal (small fixture only) | 2026-03-26 |
| 02 | Large-scale Ablation | Pilot conclusions reversed. Structure is dominant. ΔV is strongest signal | 2026-03-27 |
| 03 | Input Method Comparison | design-tree >> raw JSON. MCP direct render = 94% | 2026-03-27 |
| 04 | Viewport Test | Responsive reveals structure's true value. bad-structure drops -23%p at 500px | 2026-03-27 |
| 05 | Ablation Phase 1 | Strip experiments across 3 desktop fixtures | 2026-03-28 |
| 05b | Ablation Phase 2 | Generalization across 6 additional fixtures incl. mobile | 2026-03-28 |
| 06 | Calibration Data Validity | 3 of 6 responsive deltas invalid — size-constraints strip consistently hurts or is neutral | 2026-03-31 |
- design-tree @500px: 95% similarity (root width fix only, flex expands naturally)
- MCP Vite @500px: 92% (good, hardcoded widths), 75% (bad-structure)
-
MCP requires LLM post-processing for responsive — hardcoded
w-[375px]in child elements - design-tree is inherently responsive — root width removal is the only change needed
- Structure (auto-layout) is the only category that affects responsive — token, component, naming are viewport-independent
Help teams implement designer-provided Figma designs exactly as designed, with zero unnecessary AI token cost.
See #121 for full direction statement.
OAT (One-at-a-Time) Sensitivity Analysis — remove one design property at a time and measure the impact on AI implementation quality.
Getting Started
Releases
Reference
Round-Trip
Pipeline
Calibration Logs
Experiments