probex

Built by agents, for agents. Exact answers to structural questions about Elixir source — so the tokens go into the task, not into sed windows and grep guesses.

Ask for the body of a function, every clause of a multi-clause definition, the precise range of a describe block, a table of contents for a file you have not opened, what a file imports, or the uncovered lines of a test run attributed to the functions that contain them:

$ probex body lib/accounts.ex register_user/2
lib/accounts.ex:148-163  def register_user/2   [defmodule MyApp.Accounts]
  def register_user(attrs, opts \\ []) do
    ...
  end

Reading source — body, clauses, outline, directives — parses; it never compiles. No mix, no dependency resolution, no application boot: it works on files that do not currently build and cannot be broken by a half-finished refactor. cover is the one exception; it needs compiled beams.

grep finds where something starts. It cannot tell you where it ends — so an agent guesses a window, and the guessing is where the tokens go:

1. grep -n to locate the function;
2. sed -n '148,200p' to guess its extent;
3. the window clips the block, or drags in three neighbours — read again, wider;
4. repeat per function, per file, per task.

Every guessed window is paid for in tokens whether or not it was the right window, every miss is paid for twice, and each attempt is a full agent round-trip. Mining one week of agent transcripts found 2,369 windowed reads with a median window of 54 lines — for answers that are usually a dozen lines, exactly bounded, and known to the parser all along. Nor is guessing safe: line-based heuristics break on heredocs containing the word end, on sigils containing whole fake function definitions, and on keyword-form bodies that have no end at all.

The parser already knows all of this. probex just asks it: exact blocks, smaller payloads, no re-reads — and one call for as many files and selectors as the question needs.

It started with a disappointed human watching the token bill: transcript after transcript of agents grep-ing for a function, guessing a sed -n 'a,bp' window, clipping the block, and reading again. The ask was one sentence — this is wasteful, improve it. Every decision after that was the agents':

1. Agents chose the tool. Coding agents working a large Elixir monorepo left behind thousands of shell calls — grep -n to find a function, a guessed sed -n 'a,bp' window to read it, another window when the first clipped the block, throwaway scripts to scrape coverage HTML. Mining those transcripts decided the feature set: body exists because the windowed read was the single most repeated shape in the corpus, and cover because an inline coverage parser was the most reinvented wheel. No feature came from a human wishlist.
2. Evidence over intuition, iterated daily. Every invocation can be logged (PROBEX_LOGFILE), and probex stats turns the log into the next decision: which commands get reached for, which fail, and — the sharpest finding — which options exist and are never used. On top of the log, an explicit skill ran at the end of each day to mine that day's agent sessions for probex gaps — the reads it still could not answer, the workarounds agents fell back to — and to propose the next extensions. An early roadmap ranked by intuition got its top pick wrong; after that, nothing shipped without measured demand.
3. Agents improve it for agents. Feedback comes from the agents using the tool, filed as structured records with transcript evidence attached, and the fixes are made by agents with the test suite as the north star. probex prompt closes the loop: the tool's instructions ship from the binary itself, and a test fails when a command or option exists that the prompt does not teach.

The human's role: ask once, run the loop, and pay for the tokens.

git clone https://github.com/pnezis/probex.git
cd probex
sudo ln -s "$PWD/probex" /usr/local/bin/probex

Or skip sudo and link into any directory already on your PATH:

ln -s "$PWD/probex" ~/.local/bin/probex

Verify before moving on:

$ probex --help | head -1
probex — AST queries over Elixir source. Parses only; never compiles.

Requirements: Elixir. That's it — no dependencies, no build step. Developed against Elixir 1.18 / OTP 27; it relies on token_metadata: true from Code.string_to_quoted/2, so 1.13+ should work.

Then teach it to your agents:

$ probex prompt --write
added to   CLAUDE.md
added to   .github/copilot-instructions.md

probex <command> --help is the reference for each command's selectors and options — and a test holds every usage text to the flags the parser actually accepts, so the docs cannot drift from the tool.

probex body lib/accounts.ex register_user/2      # by name/arity
probex body lib/accounts.ex 'describe:refunds'   # by title; also test:, def:, call:
probex body lib/accounts.ex L142                 # the block containing line 142
probex body lib/accounts.ex L142,88,301          # a batch, deduplicated
probex body lib/accounts.ex 'validate/2..normalise/1'   # a contiguous run
probex body lib/accounts.ex @default_assigns     # a multi-line attribute
probex body lib/accounts.ex defstruct            # the struct definition
probex body lib/accounts.ex 'defimpl:Inspect'    # a protocol implementation
probex body test/accounts_test.exs preamble      # line 1 to the first def/test

One call takes as many files and selectors as needed — each run of files followed by a run of selectors is a group, and every selector applies to every file in it:

probex body lib/a.ex lib/b.ex changeset/2 valid?/1   # both blocks, in both files
probex body lib/ handle_info/2                       # every match under a directory
probex body 'lib/**/*_live.ex' mount/3               # every match under a pattern
probex body lib/accounts.ex:142                      # a line, as grep -n prints it

Across many targets a miss is counted rather than listed, an unparseable file does not stop the rest, and the exit is 2 only when nothing matched anywhere. --head <n>/--tail <n> page long bodies and name what they left out; --decorators and --docs add the @spec run and the @doc.

probex clauses lib/accounts.ex normalise

Clauses need not be adjacent; each prints with its own range. Same grammar and flags as body.

probex outline lib/accounts.ex
probex outline test/accounts_test.exs --kind describe,test

Every block with its exact range, no bodies; every range feeds straight back into body. DSL-heavy files stay legible: multi-line macro calls (defparsecp, scope, schema) and module attributes are blocks too.

probex directives lib/accounts.ex

Answers "where does this name come from" in one call, with the local name each directive introduces.

mix test --cover                      # export coverdata first
probex cover                          # the gap map
probex cover my_app --fun register_user/2 --body   # did my new test go green?
probex cover my_app --top 10          # worst functions, worst first
probex cover my_app --summary         # provenance and totals only
probex cover my_app --explain lib/accounts.ex:247   # what cover measures there

Works with single projects, umbrellas and workspace monorepos, with no configuration. Every report opens with which export it read and when, and says stale when a source is newer than the export. Totals reconcile with mix test.coverage, including :ignore_modules — and when no mix.exs can be tied to the export (a renamed CI artifact, say), the header says the ignore list was not applied instead of reporting unfiltered totals as authoritative.

This is the one place probex runs your code: it evaluates the owning project's mix.exs (in that project's directory, MIX_ENV=test) to read its coverage settings, because real projects reach them through indirection no parser can follow. Worth knowing before pointing cover at a repository you don't trust. --root and --coverdata skip discovery entirely.

probex prompt --write     # into AGENTS.md / CLAUDE.md / GEMINI.md / copilot-instructions.md

Prints or installs the instructions an agent needs, fenced by markers so re-running after an upgrade updates the block in place. Shipped from the binary because hand-copied instructions drift, and an option the docs omit is an option that does not exist.

probex stats --since 7d

Reads the usage log back: invocations by command, options used and never used, selector forms reached for, failures by message. The one command a human asked for, and the exception that proves the loop: agents consume the log's JSON directly, and stats renders the same evidence for the human deciding what to build next.

Set PROBEX_LOGFILE to record one JSON object per invocation; unset, nothing is written or read.

export PROBEX_LOGFILE=~/.probex/usage.jsonl

{"ts":"2026-08-08T09:12:44Z","cwd":"/src/app","command":"body","args":["lib/accounts.ex","register_user/2"],"exit":0}

Concurrent runs can share one file (single atomic append), logging never changes behaviour or exit status (any write failure is swallowed), and exit 2 means a user error — an ambiguous selector, a missing app — not a crash.

• Parses, never compiles — except cover, which is labelled as such above.
• No silent no-ops. Unrecognised flags are refused with the real ones listed; a pattern matching nothing is an error, not an empty run; a coverage join that resolved no sources exits 2 instead of reading as fully covered.
• Errors are the next command. An ambiguous name lists the candidates and the call that resolves them; a miss says what the file does hold; a missing path gets a did you mean. The next call is the answer — not another probe you pay for.
• Exact boundaries, verified. A whole-repository sweep asserted the computed end line of every block against the real end token: 53,991 blocks across ~4,600 files, 0 mismatches, 0 parse failures.
• Output never lies about truncation. --head/--tail name the lines they left out, and a reader closing the pipe early gets output cut off on stderr rather than a clipped block that reads as whole.

• No .heex support — colocated templates are invisible to every command.
• directives reports directives, not effects: what a use injects is invisible without compiling.
• Computed module names are reported as written (defmodule __MODULE__.Child shows as such); definitions inside them are found normally.
• A mix.exs that cannot be evaluated is skipped by cover, so its :ignore_modules go unapplied.
• A report cannot tell which test run produced its coverdata — age is checked, scope is not; a .coverdata file records nothing about the run.
• Usage logging runs inside the VM, so it cannot record what never reaches it (a missing elixir, a killed process). The log is evidence, not a census.

bin/test                              # everything
PROBEX_SKIP_INTEGRATION=1 bin/test    # skip the suite that compiles mix projects
elixir test/probex_test.exs           # or any single suite directly

No dependencies and no setup. Twelve suites, including integration tests that compile real fixture projects and run mix test --cover against them.

Alpha, but not experimental — extracted from an internal tool used daily by a fleet of coding agents against a production monorepo. Contributions welcome; the honest backlog is in Limitations.

MIT