Kin
Install

Public proof

What we measured, and what we will not claim.

We ran this once against a task set we froze first, and we are publishing all of it, including the part where Kin only ties. If you would rather check the ripgrep number yourself than take our word for it, the commands are below.

Inspect the public proof package

The ripgrep demonstration

A one-line edit, and the entities it affects.

Clone ripgrep at release tag 15.2.0, add one line to the resolve_binary signature, and ask Kin what it touched. You get 13 impacted entities within 3 hops, including 3 direct callers of the signature you changed. The graph was already built, so nothing compiled to answer that. Run the commands below and you should get the same thing we did.

reproduce the demonstration
git clone --single-branch --branch 15.2.0 https://github.com/BurntSushi/ripgrep
# tag 15.2.0 pins commit e89fff89ac9af12e8d4ce9d5fd07beb408ca730f
cd ripgrep && kin init && kin embed
# in crates/cli/src/decompress.rs, add this parameter line to the resolve_binary signature:
#   search_dirs: Option<&[PathBuf]>,
kin impact resolve_binary --depth 3

The edit is one added line, +1/-0, to the resolve_binary signature. Name-mode search starts answering about two minutes after kin init begins. The full index and embed pass is slower, roughly 20 minutes and up to 7 GB of memory, and the terminal goes quiet once the linking bar finishes. That silence had us worried the first time too, and it is normal. You need macOS or Linux for this one, since kin embed has no native Windows path yet.

Published result

26/26

tasks, three identical Kin passes

One result, and all it shows is determinism.

Across a frozen set of 26 Multi-SWE-Bench Go tasks, the Kin arm returned the same retrieval result on every task in all three passes. Same input, same answer.

  • Against the grep-driven baseline arm, the same run is a statistical tie on file-level accuracy and at parity on cost. The 95% confidence interval includes 0, and McNemar exact gives p = 0.125. That is not a win, and we are not going to claim accuracy or token savings from it.
  • The run used Kin commit 2508da69, which long predates the current release line. Newer release code, including the shipping v0.5.x binaries, is not covered by this result.
  • This is our own measurement, published so you can pick it apart. The harness is still private, so you cannot reproduce the benchmark with one command yet.

Method

How we set the run up.

I fixed the protocol before running anything, and scored both arms against the same task denominator.

  1. 01

    Frozen task suite

    A hashed set of 26 Multi-SWE-Bench Go tasks, fixed before the run, so the denominator cannot drift between arms or passes.

  2. 02

    Symmetric arms

    Model, seed, context budget, tool-call cap and scorer all stay put. The retrieval tool is the only thing we changed.

  3. 03

    Three Kin passes

    If the Kin result had not reproduced byte-identically all three times, the run would have been void.

  4. 04

    Real tool traces

    A result only counts when the proof package records the Kin calls behind it.

Claim boundary

What we are not claiming.

None of this is fine print. These are the edges of what we actually measured.

  • No headline accuracy claims. The linked research bundle reports raw precision, recall, and F1 both ways, including metrics where Kin trails, and none of it becomes a product claim pending independent verification.
  • No claim that Kin broadly beats grep, text search, Git, or GitHub.
  • No token-reduction, time-to-merge, bug-catch, or merge-gate accuracy claim.
  • No claim that advisory review automatically blocks a merge.
  • No claim that the measured commit proves the behavior of newer release bytes.

Inspect the source

If you are going to repeat the claim, read the evidence first.

The public package has the benchmark boundary and the governing artifacts in it. It also lists the work we have not finished checking.

Open the proof package