Skip to content
zeyu-zhengPublic

About

No description, website, or topics provided.

Resources

Stars

18 stars

Watchers

2 watching

Forks

Latest commit

 

History

9 Commits

Folders and files

Repository files navigation

Carnegie Mellon University          L3 Lab          Anysphere

FAR: Find, Attempt, and Recommend

Overview | The Cascade | Data Preparation | Setup | Running | Citation

FAR is a literature-to-review cascade. Given a large literature corpus and a research direction, FAR finds relevant open conjectures, attempts to prove or disprove them, and recommends promising conjecture-resolution pairs for expert review. The pipeline has three stages:

  • Find — label which papers lie in the direction, extract unresolved statements from them, and check which are well posed and still open.
  • Attempt — search for resolutions across the extracted conjecture pool, with each attempt yielding either a claimed resolution or no result.
  • Recommend — judge each claimed resolution for correctness, grade it for significance, and put forward those worthy of expert review.

Overview

Frontier-model reasoning is a limited resource, and expert mathematical review is scarcer still. In most existing AI-for-math workflows, human effort is concentrated at the two ends: choosing a problem worth attempting, and reviewing the resulting model outputs. FAR replaces the first with a research direction and compresses the second with an automated triage cascade.

From choosing a problem to choosing a direction

Figure: The upper part is the problem-level interface used by most AI-for-math workflows; the lower part is ours — the mathematician supplies a direction, and FAR supplies the problems and candidate resolutions.

The Cascade

Inspired by search and recommender systems, FAR narrows a large collection through progressively more expensive stages, so later stages can afford more compute per item.

From papers to recommendations for expert review

Figure: Upper row, a search or recommender cascade. Lower row, the analogous FAR pipeline.

       corpus
         │
  Find   ├─ label ────── does this paper lie in the research direction?
         ├─ extract ──── recover explicit unresolved statements
         └─ check ────── open / solved / invalid  ────────────────►  pool P
         │
Attempt  └─ solve ────── one attempt per conjecture: KNOWN / NEW / FIX / NONE
         │
Recomm.  ├─ judge ────── is the claimed resolution correct?  PASS / FAIL / KNOWN
         └─ grade ────── how significant?  KNOWN / TYPE1 / TYPE2 / TYPE3  ──►  artifacts A

TYPE2 and TYPE3 results form the artifact set put forward for expert review.

Stage names

The code uses the names from the paper.

Paper Module Output
Find, "Finding relevant papers via labeling" src/label.py data/labeled.jsonl
Find, "Extracting and recovering conjectures" src/extract.py data/extracted.jsonl
Find, "Checking validity and status" src/check.py data/checked.jsonl
Attempt, "Attempting for Candidate Resolutions" src/solve.py data/solved.jsonl
Recommend, "Judging" src/judge.py data/judged.jsonl
Recommend, "Recommending for review" src/grade.py data/graded.jsonl

Find prompts and per-task agent messages are in src/prompts.py. Solve, Judge, and Grade system prompts are maintained directly in agents/prover.md, judge.md, and grader.md, matching appendix A of the paper, with their opencode settings in agents/agents.json. Output schemas and validators are in src/schemas.py.

The research direction

Label takes a free-text direction and answers a single yes/no per paper, so the granularity is yours to set:

--direction "combinatorics"
--direction "extremal graph theory"
--direction "additive number theory over function fields"

Data Preparation

FAR reads one input: a corpus of paper text, stored as one Arrow IPC file (.arrow) with a text column, one paper per row. By default, the scripts read data/raw/corpus.arrow. To use another path, set CORPUS at the top of the script you are running, or pass it directly:

python src/pipeline.py --stage all --corpus /path/to/corpus.arrow

For debugging, data/debug.arrow contains the extracted full text of 20 real arXiv mathematics papers. Pass --corpus data/debug.arrow and --direction combinatorics to use it.

Setup

Please use Python 3.10 or newer. You can create a conda environment for FAR by running:

conda create -n far python=3.12 -y
conda activate far
pip install -r requirements.txt

The API keys and endpoints are read from environment variables. ~/.env is also loaded automatically, so you can keep the variables there instead of exporting them; to start from the template, run cp .env.example ~/.env and fill it in. Each of the three Find stages takes its own pair, so they can run on different providers:

export LABEL_API_KEY=...      LABEL_BASE_URL=...
export EXTRACT_API_KEY=...    EXTRACT_BASE_URL=...
export CHECK_API_KEY=...      CHECK_BASE_URL=...

Solve, Judge, and Grade run through the opencode CLI, which reads the key of the model's provider, such as OPENAI_API_KEY for openai/gpt-5.5; opencode's free models need none. Use OpenCode 1.18.32. Install it using the official instructions and ensure it is on PATH. It can be replaced by any other agent backbone — Codex, Claude Code, Cursor — by editing src/agent.py.

To run these three stages in Docker instead (--mode docker), start Docker and pull the image once; opencode is then not needed on the host. Each container gets the variables set in ~/.env, so keep the provider's key there:

docker pull docker/sandbox-templates:opencode-0.7.0

Running

The four scripts in scripts/ cover the three phases and the whole pipeline. You can run them from the repository root:

bash scripts/run_all.sh      # everything, streamed -> run_all.log
bash scripts/find.sh         # label -> extract -> check -> find.log
bash scripts/attempt.sh      # solve -> attempt.log
bash scripts/recommend.sh    # judge -> grade -> recommend.log

You can also run a single stage against an earlier stage's output:

python src/pipeline.py --stage check --corpus /path/to/corpus.arrow --input data/extracted.jsonl
python src/pipeline.py --stage grade --corpus /path/to/corpus.arrow --input data/judged.jsonl

Common flags are --direction for the research direction, --limit N for a trial on a few rows, --resume true to continue an interrupted run, --judges N for independent judge passes per solution, and --<stage>-jobs for concurrency. --mode docker runs each agent attempt in its own container. python src/pipeline.py --help lists all of them.

The defaults reproduce the configuration reported in the paper:

Stage Model Reasoning effort Web search
label gpt-oss-120b — —
extract gemini-3.5-flash — —
check gemini-3.1-pro — yes
solve openai/gpt-5.5 xhigh yes
judge openai/gpt-5.5 xhigh yes
grade openai/gpt-5.5 xhigh yes

Solve, Judge, and Grade have no web-search flag, since opencode grants its agents web access by default.

A run writes two directories: data/ for the stage outputs, one .jsonl per stage, and .far/ for the agent workspaces, one per conjecture. Each stage there keeps its answer, plus an output_NNN.log and a session_NNN.json per opencode run.

Citation

@article{zheng2026problem,
  title={The problem is the problem: Towards scalable mathematical discovery},
  author={Zheng, Zeyu and Zhang, Shengtong and Avigad, Jeremy and Tetali, Prasad and Welleck, Sean},
  journal={arXiv preprint arXiv:2608.16977},
  year={2026}
}

License

This project is licensed under the Apache License 2.0.

About

No description, website, or topics provided.

Resources

Stars

18 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages