Deterministic repository receipts, analysis, review artifacts, and CI gates.
tokmd is the fast deterministic code-intelligence instrument in the Effortless
Metrics evidence stack. It turns a source tree into stable Markdown and JSON
artifacts: language and module summaries, file receipts, analysis reports,
diffs, policy gates, baselines, sensor reports, and LLM-ready context bundles.
Use it from the CLI first; wire the same surfaces into CI when you want automated receipts, comments, and gates.
cargo install tokmd --locked
tokmd --versionSee Install and try tokmd for the first-run path, or Install tokmd for release binaries, Nix, and CI usage.
# Install
cargo install tokmd --locked
# Inspect this repo
tokmd --format md --top 8
# Review this PR
tokmd cockpit --base origin/main --head HEAD --review-packet-dir .tokmd/review
# Prepare a coding-agent handoff
tokmd handoff --preset risk --budget 128k --strategy spread --out-dir .handoffFor CI adoption, start with the GitHub Action quickstart:
- uses: EffortlessMetrics/tokmd@v1
with:
version: '1.11.0'
paths: .
artifact: 'true'For browser/no-install trial, use Browser runner and then Browser to native when the job becomes PR review, handoff, or CI evidence.
For release-facing evidence, start with Release readiness. It checks package and version readiness without publishing, tagging, or creating releases.
For a job-oriented walkthrough, start with Install and try tokmd or Start Here. For the command-to-artifact reading order, use User paths.
| Surface | Output |
|---|---|
| Repository summary | Markdown tables for languages and modules |
| Receipts | JSON, JSONL, CSV, CycloneDX, HTML, SVG, Mermaid |
| Analysis | Risk, effort, complexity, duplication, git, and API-surface reports |
| Review reports | Cockpit reports, sensor reports, gate verdicts |
| Baselines | Ratchet-ready baseline JSON |
| LLM context | Bounded bundles, redaction, handoff directories |
For the receipt and packet names you will see in these workflows, use the artifact glossary. For which artifact to open first for a specific job, use User paths. For copy-ready command sequences, use Copy-Ready Workflows.
| If you need to... | Start with... | Typical output |
|---|---|---|
| summarize a repo | tokmd, module, export |
Markdown summary, file receipt |
| compare states | diff, run |
deterministic diff and receipts |
| analyze code health | analyze |
risk, effort, complexity reports |
| inspect syntax receipts | tokmd syntax (requires ast feature) |
advisory Tree-sitter syntax packet |
| review a PR | cockpit --review-packet-dir, GitHub Action review-packet |
review map, evidence, comment, packet manifest |
| gate policy in CI | gate, baseline, sensor |
verdicts, ratchets, sensor envelope |
| pack LLM context | context, handoff |
bounded bundle, handoff directory |
For local PR review, generate a cockpit review packet instead of reading CI logs or raw JSON first:
tokmd cockpit \
--base origin/main \
--head HEAD \
--review-packet-dir .tokmd/reviewStart with .tokmd/review/review-map.md for the review order and reproduction
commands, then read .tokmd/review/comment.md for the PR summary.
.tokmd/review/evidence.json and .tokmd/review/manifest.json carry the
machine-readable evidence state and hash-indexed packet inventory.
In a tokmd contributor checkout, verify the packet before using it as review evidence:
cargo xtask review-packet-check --dir .tokmd/reviewIf a PR changes source-of-truth docs, plans, ADRs, templates, .jules/goals/**,
or doc-artifact policy, generate the docs receipt first and import it:
cargo xtask doc-artifacts --check --json target/docs/doc-artifacts-check.json
tokmd cockpit \
--base origin/main \
--head HEAD \
--doc-artifacts-check target/docs/doc-artifacts-check.json \
--review-packet-dir .tokmd/reviewThis makes documentation-control evidence visible in the review packet; it does not turn cockpit into a merge verdict or promote advisory proof.
For proof imports, hosted Action artifacts, and what packet verification does and does not mean, see tokmd in Cockpit.
Use the root composite Action when you want tokmd receipts, PR summaries, artifacts, or gates in CI.
- uses: EffortlessMetrics/tokmd@v1
with:
version: '1.11.0'
paths: .For a copy-ready adoption workflow, see GitHub Action quickstart. For all Action modes, inputs, outputs, artifacts, checkout guidance, and release-asset behavior, see GitHub Action reference.
Raw LOC output is easy to generate and hard to reuse.
CI needs artifacts and gates. Review workflows need stable before/after surfaces. LLM workflows need bounded context instead of pasted terminal output.
tokmd makes repository shape repeatable, diffable, and machine-readable.
In the wider Effortless Metrics stack, tokmd produces code evidence.
evidencebus carries and inventories evidence from tokmd, mergecode, CI
sensors, gates, and other producers. tokmd stays useful on its own while
remaining a focused producer rather than the whole evidence backplane.
Representative summary output:
|Lang|Code|Lines|Bytes|Tokens|
|---|---:|---:|---:|---:|
|Rust|377263|470341|15334354|3833170|
|JSON|5405|5405|284012|70997|
|Markdown|3067|17273|567919|141930|
|JavaScript|1979|2233|64463|16111|
|TOML|1947|2387|71514|17855|
|Other|1978|2758|78806|19691|
|**Total**|391639|500397|16401068|4099754|| Command | Purpose |
|---|---|
tokmd |
Language summary for a repo or path set |
tokmd module |
Group stats by module roots such as crates/ or packages/ |
tokmd export |
File-level dataset for downstream pipelines |
tokmd run |
Save a full receipt set to a run directory |
tokmd analyze |
Derived metrics and enrichments |
tokmd badge |
Render SVG badges from receipt metrics |
tokmd diff |
Compare two runs, receipts, or refs deterministically |
tokmd context |
Pack code into an LLM context window |
tokmd handoff |
Build an LLM handoff bundle |
tokmd cockpit |
PR-review metrics with risk and evidence gates |
tokmd gate |
Evaluate TOML policy rules and ratchets |
tokmd baseline |
Capture a baseline for later ratchet comparisons |
tokmd sensor |
Emit a sensor.report.v1 envelope |
tokmd evidence-packet |
Emit a manifest for sensor artifacts such as analyze and context output |
tokmd tools |
Generate tool definitions for OpenAI, Anthropic, and JSON Schema consumers |
tokmd init |
Generate a .tokeignore template |
tokmd check-ignore |
Explain why a path is being ignored |
tokmd completions |
Generate shell completions |
tokmd-wasm and web/runner expose a narrower browser-safe slice:
langmoduleexport- browser-safe
analyzepresets on ordered in-memory inputs
Native filesystem flows, Git-history enrichers, gate, cockpit, sensor, evidence-packet, baseline, context, and handoff remain native-first.
The browser runner documents its in-memory GitHub ingest cache, supports local file/directory inputs, shows repo-load and worker-run progress, surfaces retry/rate-limit guidance, and supports session-only GitHub token auth without storing raw tokens in cache keys.
The machine-readable capability contract lives in docs/capabilities/wasm.json.
For the user-facing browser workflow and native-only boundaries, see Browser runner.
- It is not a formatter, linter, or build system.
- It is not a developer-scoring tool.
- It is not a vulnerability database or SAST replacement.
- It does not ask you to trust prose where a receipt can be emitted instead.
- Tutorial for first-run setup and basic workflows
- Install tokmd for Cargo, release binaries, Nix, and CI entry points
- Recipes for practical usage patterns
- Handoff bundles for coding-agent context and proof expectations
- Troubleshooting for common problems and fixes
- Contributing for local development and release work
- GitHub Action reference for Action inputs, outputs, modes, and release assets
- CLI Reference for flags, formats, and config
- Specification for current product contracts
- Schema for receipt contracts
- tokmd responsibilities for the wider sensor/receipt stack
- Philosophy for the design stance
- Architecture for the crate graph and boundaries
- ADRs for accepted architecture and contract decisions
- Design for system concepts and invariants
- Roadmap for the active horizon
MIT or Apache-2.0.