Skip to content
orze-aiPublic

About

No description, website, or topics provided.

Resources

Stars

7 stars

Watchers

0 watching

Forks

Repository files navigation

orze

PyPI License orze-pro

A GPU experiment orchestrator for ML research.

Orze runs experiments on GPUs: schedule ideas → train → evaluate → report → repeat. It coordinates GPUs via filesystem locks, works across machines, and gives you a complete leaderboard, notifications, and analysis — out of the box.

Website: orze.ai

Research execution API: prepare proposals concurrently and evaluate ready work.

Research exploration API: branch-local discovery and replay, with Dream-RSI analysis and historical counterexamples.

Research results and handoff: verified outcomes, reusable code, and machine recovery notes (中文).

Install

curl -sL https://orze.ai/install | bash

That's it. It installs orze, detects your GPUs and codebase, generates training scripts and experiment ideas, and starts running — all in one command.

Pass environment variables for additional options:

# LLM-powered setup
ANTHROPIC_API_KEY=sk-ant-... curl -sL https://orze.ai/install | bash

# With pro (autopilot)
ORZE_PRO_KEY=ORZE-PRO-xxx curl -sL https://orze.ai/install | bash

# Custom project path
curl -sL https://orze.ai/install | bash -s /nfs/my-project

orze vs orze-pro

orze is a complete, production-ready tool. orze-pro adds autopilot — so experiments run while you sleep.

Feature orze (free) + orze-pro
GPU scheduling & multi-node ✓ ✓
Idea queue (ideas.md + SQLite) ✓ ✓
Hyperparameter sweep (auto-expand grid) ✓ ✓
Leaderboard report ✓ ✓
Notifications (Telegram/Slack) ✓ ✓
Admin dashboard & MCP server ✓ ✓
Retrospection (plateau detection) ✓ ✓
Cross-experiment regression analysis ✓ ✓
Failure analysis & categorization ✓ ✓
Checkpoint GC ✓ ✓
Sealed eval protection ✓ ✓
Service watchdog (auto-restart + containers) ✓ ✓
Autonomous research agents (Gemini/GPT/Claude) ✓
The Professor (paper lake, cross-domain search, strategy) ✓
Engineer (implement ideas, fix bugs) ✓
Auto-fix failed experiments ✓
Code evolution on plateau ✓
Meta-research (strategy adjustment) ✓
FSM orchestration (7 procedures) ✓
Data analyst & thinker (auto-injected) ✓

Research Loop Comparison

orze free + orze-pro
How ideas are generated Smart Suggestions — rule-based: detects regressions, generates scale sweeps, perturbations Research Agents — LLM-driven: reads all results, forms hypotheses, designs novel experiments
How failures are handled You read the failure log Auto-fix: LLM diagnoses and patches the error
How plateaus are handled Smart Suggestions tries parameter variations Code Evolution: LLM modifies your train script
Does research stop? Continuous by default; stop via CLI or queue policy Continuous by default; finite campaigns use research_budget.max_agent_cycles
Requires API key? No Yes (Gemini/OpenAI/Anthropic)

Compatibility

orze orze-pro Notes
4.6.x 0.13.x Current release

Quick Start

After install, orze auto-detects GPUs and starts running experiments.

AI CLI users (Claude Code, Cursor, Codex):

do @ORZE-AGENT.md

CLI Reference

# Project lifecycle
orze init [path]              # initialize a new project
orze start                    # start as background daemon
orze stop                     # stop gracefully
orze restart                  # stop + start
orze --check                  # validate config, files, GPUs, API keys
orze --launch-status          # fast stop/pause policy JSON; no GPU access
orze --uninstall              # full cleanup, preserves research results

# Operations
orze run-idea IDEA --gpu N    # run one admitted queue item on one physical GPU
orze upgrade                  # reinstall from source + restart daemon
orze admin migrate            # migrate legacy layout to .orze/
orze service install          # auto-restart on crash (systemd)

# CPU-only harness efficiency receipt
python -m orze.benchmarks.harness_efficiency --work-dir .orze

# Pro
orze pro activate <key>       # activate license
orze pro status               # check license info
orze pro deactivate           # remove license
orze sop list                 # list available SOPs

File Layout

your-project/
├── orze.yaml                 # Project config (single source of truth)
├── train.py                  # Your training script
├── ideas.md                  # Experiment queue
├── GOAL.md                   # Research objective
├── RESEARCH_RULES.md         # Agent constraints
├── configs/base.yaml         # Default hyperparameters
├── .env                      # API keys (gitignored)
├── ORZE-AGENT.md             # AI CLI instructions
├── ORZE-RULES.md             # Agent guardrails
├── venv/                     # Training dependencies
├── .orze/                    # Runtime state (gitignored)
│   ├── state/version.json    # Layout version
│   ├── logs/                 # Role logs
│   ├── locks/                # Filesystem locks
│   ├── rules/                # Migrated rule files
│   ├── mcp/                  # MCP server configs
│   ├── receipts/             # Execution evidence
│   ├── triggers/             # One-shot role triggers
│   ├── heartbeats/           # Per-host liveness
│   ├── backups/              # ideas.md backups
│   └── feedback/             # Failure feedback
├── procedures/               # User procedure overrides (pro)
├── fsm/runner.py             # FSM orchestrator (pro)
└── orze_results/             # Research outputs
    ├── idea-0001/metrics.json
    ├── methods/              # Generated code
    └── knowledge/            # Analysis insights

Multi-node

Start orze in the same shared folder on any machine — nodes auto-join the research pool.

# Node 1
ssh node1 "cd /nfs/project && orze start"

# Node 2
ssh node2 "cd /nfs/project && orze start"

Key Features

  • Measured control-plane scaling — indexed queue/config-identity paths with a reproducible efficiency acceptance receipt
  • 4C qualification boundary — a public terrain, anti-gaming rule, and promotion contract that separates source evidence from task-proven research yield
  • Config Inheritance — Child ideas inherit parent configs; specify only what changes
  • HP Sweep — lr: [1e-4, 3e-4] auto-expands into all combinations
  • Failure Protection — Stops automatically when failure rates spike
  • Fail-closed Data Boundaries — Kernel-isolated held-out paths and optional training-network denial plus keyed-fingerprint manifest separation (configuration and limits); managed runs can bind those controls to exact output bytes with model lineage receipts
  • Cross-Experiment Analysis — Detects regressions, tradeoffs, and suggests actions
  • Rich Notifications — GPU VRAM, per-dataset breakdown, verified results, target/gap tracking
  • Admin Panel — Real-time web dashboard at http://localhost:8787
  • Clean Uninstall — orze --uninstall removes runtime files, preserves results

The Contract

Your training script receives:

python train.py --idea-id idea-001 --results-dir orze_results --ideas-md ideas.md --config base.yaml

Required output: orze_results/{idea_id}/metrics.json:

{"status": "COMPLETED", "test_accuracy": 0.92, "training_time": 142.5}

See SKILL.md for the full technical specification.

Admin Panel

Auto-launches at http://localhost:8787. No extra install needed.

admin-panel admin-queue admin-leaderboard

Telegram Notifications

notifications:
  enabled: true
  on: [completed, failed, new_best]
  channels:
    - type: telegram
      bot_token: "YOUR_BOT_TOKEN"
      chat_id: "YOUR_CHAT_ID"
tg

Service Management

orze service install -c orze.yaml    # auto-restart on crash + manage containers
orze service status                  # check health
orze service audit                   # verify effective unit + pinned runtime
orze service uninstall               # remove

The watchdog runs every minute (crontab) or every 5 minutes (systemd). It restarts orze on crash/stall and manages Docker containers defined in orze.yaml:

For systemd installations, the watchdog timer is the sole restart decision owner. The main service does not unconditionally restart itself, so .orze_disabled and .orze_stop_all remain authoritative during deliberate stops and persistent startup failures cannot become restart storms. Installation pins the imported Orze/Orze-Pro package trees. Before every systemd start, Orze checks those hashes plus the effective unit properties (including drop-ins) and fails closed on runtime drift or an active stop latch. Reinstall the service intentionally after a reviewed package upgrade.

The watchdog also records repeated launch failures in a per-host, mode-0600 state file under results_dir. The second consecutive failure with the same categorical reason emits a rate-limited watchdog_failure_loop alert; a changed reason or restored service health resets the sequence. The receipt and alert contain only a stable reason code, count, and SHA-256 fingerprint—not subprocess output, command lines, paths, environment values, or notification secrets.

Direct/manual launches can opt into the same exact interpreter and package-tree identity check with a controller_runtime block in orze.yaml. Run orze service capture-runtime from the reviewed executable to print a canonical block to paste under that key, then run orze --check to verify it and see the active interpreter and import root. This in-process pin detects drift only in contract-aware Orze versions; it cannot prevent an older, pre-contract executable from ignoring an unknown key. Use the managed systemd service and its independent ExecStartPre check when downgrade resistance is required.

Every controller also takes a kernel-backed exclusive lease on each physical GPU in its invocation scope before startup checks or GPU telemetry. Controllers with disjoint scopes can coexist; any overlap fails closed. Training, evaluation, pre/post, post-hoc, and GPU smoke-test children inherit the lease descriptor, so a controller crash or clean detach cannot make a still-running child's device appear available to another Orze controller. The startup smoke test probes only the explicit invocation/allowed_gpus scope and uses CPU when no GPU scope is declared. These host-local leases coordinate Orze processes running as the same OS user; unrelated external launchers remain outside the Orze control boundary. Lease contention returns exit status 2 with a categorical gpu_lease_contended: physical_gpu=N reason.

orze run-idea IDEA --gpu N is the fail-closed manual campaign path. It requires an exact runtime pin, an authoritative QUEUED lake row, a current decision-admission receipt, and an allowed physical GPU. It honors every stop and pause latch, runs no research roles or auto-fix agents, does not reconcile or report unrelated queue entries, and restricts GPU telemetry and child CUDA visibility to that one device. Exit status zero additionally requires a COMPLETE lifecycle plus configured evaluation, post-script, benchmark, and model-lineage evidence. It is not a ranking claim.

Public or benchmark-comparable campaigns should also make the evidence policy explicit. These switches make configuration invalid—and admission impossible— until the corresponding full contracts are configured:

managed_run:
  require_data_separation: true
  require_model_lineage: true
  require_benchmark_contract: true
  require_explicit_untainted_metrics: true

With this policy, an absent or merely implicit no-leakage claim cannot produce a successful command status. Data-separation evidence is revalidated at the terminal boundary; model-lineage and benchmark receipts retain their existing fail-closed validation. The defaults remain false for ordinary local runs that make no public or benchmark-comparable claim.

containers:
  paperdog:
    image: orzeai/paperdog:latest
    ports:
      - "8000:8000"

Containers are auto-pulled and recreated when a new image is available.

Citation

@article{li2026autoresearching,
  title={Auto Researching, not hyperparameter tuning: Convergence Analysis of 10,000 Experiments},
  author={Li, Xiaoyi},
  journal={arXiv preprint arXiv:2603.15916},
  year={2026}
}

License

Apache 2.0 — orze is and will always be free and open source.

orze-pro (autopilot features) is commercially licensed.

About

No description, website, or topics provided.

Resources

Stars

7 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages