Skip to content

EffortlessMetrics/tokmd

Use this GitHub action with your project
Add this Action to an existing workflow or create a new one
View on Marketplace

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,848 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

tokmd

CI Codecov ripr

GitHub release crates.io downloads docs.rs

MSRV License: MIT OR Apache-2.0

Deterministic repository receipts, analysis, review artifacts, and CI gates.

tokmd is the fast deterministic code-intelligence instrument in the Effortless Metrics evidence stack. It turns a source tree into stable Markdown and JSON artifacts: language and module summaries, file receipts, analysis reports, diffs, policy gates, baselines, sensor reports, and LLM-ready context bundles.

Use it from the CLI first; wire the same surfaces into CI when you want automated receipts, comments, and gates.

Install

cargo install tokmd --locked
tokmd --version

See Install and try tokmd for the first-run path, or Install tokmd for release binaries, Nix, and CI usage.

Quick Start

# Install
cargo install tokmd --locked

# Inspect this repo
tokmd --format md --top 8

# Review this PR
tokmd cockpit --base origin/main --head HEAD --review-packet-dir .tokmd/review

# Prepare a coding-agent handoff
tokmd handoff --preset risk --budget 128k --strategy spread --out-dir .handoff

For CI adoption, start with the GitHub Action quickstart:

- uses: EffortlessMetrics/tokmd@v1
  with:
    version: '1.11.0'
    paths: .
    artifact: 'true'

For browser/no-install trial, use Browser runner and then Browser to native when the job becomes PR review, handoff, or CI evidence.

For release-facing evidence, start with Release readiness. It checks package and version readiness without publishing, tagging, or creating releases.

For a job-oriented walkthrough, start with Install and try tokmd or Start Here. For the command-to-artifact reading order, use User paths.

What tokmd Produces

Surface Output
Repository summary Markdown tables for languages and modules
Receipts JSON, JSONL, CSV, CycloneDX, HTML, SVG, Mermaid
Analysis Risk, effort, complexity, duplication, git, and API-surface reports
Review reports Cockpit reports, sensor reports, gate verdicts
Baselines Ratchet-ready baseline JSON
LLM context Bounded bundles, redaction, handoff directories

For the receipt and packet names you will see in these workflows, use the artifact glossary. For which artifact to open first for a specific job, use User paths. For copy-ready command sequences, use Copy-Ready Workflows.

Choose a Path

If you need to... Start with... Typical output
summarize a repo tokmd, module, export Markdown summary, file receipt
compare states diff, run deterministic diff and receipts
analyze code health analyze risk, effort, complexity reports
inspect syntax receipts tokmd syntax (requires ast feature) advisory Tree-sitter syntax packet
review a PR cockpit --review-packet-dir, GitHub Action review-packet review map, evidence, comment, packet manifest
gate policy in CI gate, baseline, sensor verdicts, ratchets, sensor envelope
pack LLM context context, handoff bounded bundle, handoff directory

Review a PR

For local PR review, generate a cockpit review packet instead of reading CI logs or raw JSON first:

tokmd cockpit \
  --base origin/main \
  --head HEAD \
  --review-packet-dir .tokmd/review

Start with .tokmd/review/review-map.md for the review order and reproduction commands, then read .tokmd/review/comment.md for the PR summary. .tokmd/review/evidence.json and .tokmd/review/manifest.json carry the machine-readable evidence state and hash-indexed packet inventory.

In a tokmd contributor checkout, verify the packet before using it as review evidence:

cargo xtask review-packet-check --dir .tokmd/review

If a PR changes source-of-truth docs, plans, ADRs, templates, .jules/goals/**, or doc-artifact policy, generate the docs receipt first and import it:

cargo xtask doc-artifacts --check --json target/docs/doc-artifacts-check.json

tokmd cockpit \
  --base origin/main \
  --head HEAD \
  --doc-artifacts-check target/docs/doc-artifacts-check.json \
  --review-packet-dir .tokmd/review

This makes documentation-control evidence visible in the review packet; it does not turn cockpit into a merge verdict or promote advisory proof.

For proof imports, hosted Action artifacts, and what packet verification does and does not mean, see tokmd in Cockpit.

GitHub Action

Use the root composite Action when you want tokmd receipts, PR summaries, artifacts, or gates in CI.

- uses: EffortlessMetrics/tokmd@v1
  with:
    version: '1.11.0'
    paths: .

For a copy-ready adoption workflow, see GitHub Action quickstart. For all Action modes, inputs, outputs, artifacts, checkout guidance, and release-asset behavior, see GitHub Action reference.

Why It Exists

Raw LOC output is easy to generate and hard to reuse.

CI needs artifacts and gates. Review workflows need stable before/after surfaces. LLM workflows need bounded context instead of pasted terminal output.

tokmd makes repository shape repeatable, diffable, and machine-readable.

In the wider Effortless Metrics stack, tokmd produces code evidence. evidencebus carries and inventories evidence from tokmd, mergecode, CI sensors, gates, and other producers. tokmd stays useful on its own while remaining a focused producer rather than the whole evidence backplane.

What It Looks Like

Representative summary output:

|Lang|Code|Lines|Bytes|Tokens|
|---|---:|---:|---:|---:|
|Rust|377263|470341|15334354|3833170|
|JSON|5405|5405|284012|70997|
|Markdown|3067|17273|567919|141930|
|JavaScript|1979|2233|64463|16111|
|TOML|1947|2387|71514|17855|
|Other|1978|2758|78806|19691|
|**Total**|391639|500397|16401068|4099754|

Command Surface

Command Purpose
tokmd Language summary for a repo or path set
tokmd module Group stats by module roots such as crates/ or packages/
tokmd export File-level dataset for downstream pipelines
tokmd run Save a full receipt set to a run directory
tokmd analyze Derived metrics and enrichments
tokmd badge Render SVG badges from receipt metrics
tokmd diff Compare two runs, receipts, or refs deterministically
tokmd context Pack code into an LLM context window
tokmd handoff Build an LLM handoff bundle
tokmd cockpit PR-review metrics with risk and evidence gates
tokmd gate Evaluate TOML policy rules and ratchets
tokmd baseline Capture a baseline for later ratchet comparisons
tokmd sensor Emit a sensor.report.v1 envelope
tokmd evidence-packet Emit a manifest for sensor artifacts such as analyze and context output
tokmd tools Generate tool definitions for OpenAI, Anthropic, and JSON Schema consumers
tokmd init Generate a .tokeignore template
tokmd check-ignore Explain why a path is being ignored
tokmd completions Generate shell completions

Browser And WASM

tokmd-wasm and web/runner expose a narrower browser-safe slice:

  • lang
  • module
  • export
  • browser-safe analyze presets on ordered in-memory inputs

Native filesystem flows, Git-history enrichers, gate, cockpit, sensor, evidence-packet, baseline, context, and handoff remain native-first.

The browser runner documents its in-memory GitHub ingest cache, supports local file/directory inputs, shows repo-load and worker-run progress, surfaces retry/rate-limit guidance, and supports session-only GitHub token auth without storing raw tokens in cache keys.

The machine-readable capability contract lives in docs/capabilities/wasm.json.

For the user-facing browser workflow and native-only boundaries, see Browser runner.

What tokmd Is Not

  • It is not a formatter, linter, or build system.
  • It is not a developer-scoring tool.
  • It is not a vulnerability database or SAST replacement.
  • It does not ask you to trust prose where a receipt can be emitted instead.

Go Deeper

Tutorial

  • Tutorial for first-run setup and basic workflows

How-To

Reference

Explanation

  • Philosophy for the design stance
  • Architecture for the crate graph and boundaries
  • ADRs for accepted architecture and contract decisions
  • Design for system concepts and invariants
  • Roadmap for the active horizon

License

MIT or Apache-2.0.