ASI-Evolve — Let AI Do the Research, You Keep the Insight

"What if you could run a tireless AI researcher on your hardest problem — one that reads the literature, designs experiments, runs them, and learns from every failure?"

That's ASI-Evolve. It is a general agentic framework that closes the loop between knowledge → hypothesis → experiment → analysis — and repeats it autonomously, round after round, until it finds something that works.

We built it for AI research. But the loop doesn't care about domain.
A financial analyst, a biomedical engineer, a climate scientist, or a game developer can all plug their own problem into ASI-Evolve and let it search for better solutions than any human has time to manually explore.

The paper validated ASI-Evolve on three hard AI problems — each requiring expensive compute, open-ended search, and multi-dimensional feedback:

These are not toy benchmarks. These are real, frontier-level results — produced autonomously.

Repeat the loop and improve each round.

Three agents drive this loop:

• Researcher — reads the database and cognition store, proposes the next candidate
• Engineer — executes the candidate and collects structured metrics
• Analyzer — distills outcomes into transferable lessons for future rounds

Two memory systems keep the loop from going in circles:

• Cognition Store — inject your domain knowledge, papers, or heuristics upfront so the AI doesn't start from zero
• Experiment Database — every trial is stored with its motivation, code, result, and analysis; parent selection uses UCB1, greedy, random, or MAP-Elites island sampling

You don't need to be an AI researcher. You need:

1. A problem where better code = better outcome — optimization, algorithm design, pipeline tuning, simulation strategy
2. An evaluation script — something that takes a program and returns a score
3. Some domain knowledge — papers, rules of thumb, known good approaches

If you have those three things, ASI-Evolve can search the space for you.

Examples across domains:

Scenario: You're an ML infra engineer. Your LLM serving system uses a fixed continuous-batching scheduler. You want to automatically discover a better request-scheduling policy that maximizes throughput while keeping P99 latency under budget — without spending weeks hand-tuning heuristics.

git clone https://github.com/GAIR-NLP/ASI-Evolve.git
cd ASI-Evolve
pip install -r requirements.txt

mkdir -p experiments/infer_scheduler/prompts

experiments/infer_scheduler/
├── input.md          ← problem description
├── config.yaml       ← API key + overrides
├── initial_program   ← baseline scheduler to evolve from
├── init_cognition.py ← inject your domain knowledge
├── evaluator.py      ← benchmark the candidate scheduler
└── eval.sh           ← shell wrapper called each round

# LLM Serving Scheduler

Write a Python function `schedule(queue, gpu_mem_free_gb)` that selects
a batch of requests from `queue` (list of dicts with keys: seq_len, priority,
wait_ms) given available GPU memory, and returns a list of selected request ids.

Optimize for: tokens/sec throughput (primary), P99 latency ms (must stay < 500ms).

def schedule(queue, gpu_mem_free_gb):
    """FCFS baseline — just take requests in arrival order."""
    budget = int(gpu_mem_free_gb * 1024)  # rough token budget
    selected, used = [], 0
    for req in sorted(queue, key=lambda r: r["wait_ms"], reverse=True):
        if used + req["seq_len"] <= budget:
            selected.append(req["id"])
            used += req["seq_len"]
    return selected

from cognition.store import CognitionStore

store = CognitionStore(storage_dir="experiments/infer_scheduler/cognition_data")
store.add([
    {
        "title": "Continuous Batching",
        "content": "Orca-style iteration-level scheduling avoids head-of-line blocking "
                   "by preempting long requests and inserting shorter ones mid-flight.",
    },
    {
        "title": "Sequence Length Bucketing",
        "content": "Grouping requests by similar sequence length reduces padding waste "
                   "and improves GPU utilization significantly.",
    },
    {
        "title": "Priority + Starvation",
        "content": "Pure priority scheduling causes starvation. Age-weighted priority "
                   "(priority * log(1 + wait_ms)) balances latency and throughput.",
    },
])

python experiments/infer_scheduler/init_cognition.py

import importlib.util, sys, json
from benchmark import run_serving_simulation  # your internal benchmark harness

def load(path):
    spec = importlib.util.spec_from_file_location("candidate", path)
    mod = importlib.util.module_from_spec(spec); spec.loader.exec_module(mod)
    return mod.schedule

if __name__ == "__main__":
    fn = load(sys.argv[1])
    result = run_serving_simulation(scheduler_fn=fn, trace="traces/sharegpt_1k.jsonl")
    # score = throughput; constraint violation penalizes automatically
    score = result["tokens_per_sec"] if result["p99_latency_ms"] < 500 else 0
    print(json.dumps({"score": score, "metrics": result}))

# eval.sh
#!/bin/bash
python /path/to/experiments/infer_scheduler/evaluator.py "$1"

python main.py \
  --experiment infer_scheduler \
  --steps 40 \
  --sample-n 3 \
  --eval-script /path/to/experiments/infer_scheduler/eval.sh

ASI-Evolve will iterate through scheduling policies — from simple heuristics to multi-factor priority functions with dynamic chunking — and write a structured lesson after every trial. After 40 rounds you have a ranked database of every policy tried, why each worked or failed, and the best-performing code ready to deploy.

ASI-Evolve/
├── main.py                         ← entry point
├── config.yaml                     ← global defaults
├── pipeline/                       ← Researcher, Engineer, Analyzer agents
├── cognition/                      ← cognition store (embedding + FAISS)
├── database/                       ← experiment database (nodes + sampling)
├── utils/
└── experiments/
    ├── circle_packing_demo/        ← included runnable demo
    └── best/circle_packing/        ← top programs from our ablation runs

pip install -r requirements.txt

Requirements:

• Python 3.10+
• bash and python3 on your system path
• Any OpenAI-compatible API endpoint (GPT-4o, Claude, Gemini, local models via LiteLLM, etc.)
• Optional: Weights & Biases for experiment tracking

The included demo evolves a program to pack 26 circles in a unit square — a clean benchmark used in our ablation studies.

# Initialize the cognition store
python experiments/circle_packing_demo/init_cognition.py

# Run 10 evolution steps
python main.py \
  --experiment circle_packing_demo \
  --steps 10 \
  --sample-n 3 \
  --eval-script /path/to/experiments/circle_packing_demo/eval.sh

ASI-Evolve reaches SOTA-level circle-packing results in as few as 17 rounds.

Configuration merges in this order (later overrides earlier):

1. config.yaml at repository root
2. experiments/<name>/config.yaml
3. An explicit file passed with --config

Key settings:

If you use ASI-Evolve in your work, please cite:

@misc{asi_evolve_2026,
  title   = {ASI-Evolve: AI Accelerates AI},
  author  = {Xu, Weixian and Mi, Tiantian and Liu, Yixiu and Nan, Yang and Zhou, Zhimeng and Ye, Lyumanshan and Zhang, Lin and Qiao, Yu and Liu, Pengfei},
  year    = {2026},
  note    = {SJTU / SII / GAIR. https://github.com/GAIR-NLP/ASI-Evolve}
}