Eleos

Project status: Eleos is experimental software. Its acceptance gates are designed to reject unverified changes, but operating it still grants an LLM substantial access to the local development environment. Review the security model before running it.

Eleos does not select behavior through adapters keyed to a programming language, build system, test runner, or profiler. It never selects behavior from a project-type enum.

During init, the LLM inspects the repository and submits one structured contract:

• whether acceptance means optimization or repair;
• which build inputs and artifacts matter;
• which commands prepare the experiment;
• which exact commands are timed;
• how each sample is reset;
• which command produces correctness output;
• which observations can guide optimization;
• which validation commands and output assertions must pass.

Transactional edit authority is owned by the harness rather than inferred by the LLM. For normal sessions it covers the whole project root; paths outside that root remain unavailable to transactional source edits.

One session performs one kind of accepted work. An optimization contract cannot turn into repair merely because its benchmark is not yet usable, and a repair contract does not also optimize performance. A prerequisite repair and the subsequent optimization are separate sessions.

The harness validates paths and commands, presents the proposal, automatically runs its preflight, and then interprets the accepted contract without knowing what language or build system is involved. The initial repository map is context-bounded; the LLM expands only relevant paths with generic discovery tools, so generated data in a large repository cannot flood init.

inspect -> define contract -> preflight -> observe -> hypothesize
        -> diagnose or edit -> measure candidates -> verify -> consolidate
        -> continue from evidence

No change is accepted from an LLM claim. Optimization requires measured improvement, stable output, and declared validation. Repair requires a preflight-reproduced failure, then a candidate that exits successfully while preserving stable output and passing all declared validation. Repair completion assertions qualify the nonzero baseline's failure stage only; a successful candidate may remove those failure symptoms and is judged by its zero exit, current-sample output/artifacts, exact baseline output, and validation. LLM request and hypothesis counts are audit sequences, not execution budgets. Init and Agent work continue while material evidence can still be obtained, and stop only on a verified objective, user cancellation, or a concrete unavailable external condition. When causal evidence is insufficient, strategy may select a diagnostic direction instead of forcing a candidate edit. Read-only diagnostics or transactionally rolled back instrumentation produce durable decision evidence but are never eligible for candidate acceptance. Repeated equivalent mechanical failures without a new discriminator close that implementation path rather than creating another retry script. A branch blocked by a missing tool or authority ends only that direction and returns its evidence to strategy. The session ends as blocked only when strategy determines that no material alternative remains.

When a contract declares preparation and editable build inputs, preflight temporarily makes those inputs stale and runs reset plus preparation through the exact contract Runner before accepting the baseline. Their timestamps are restored afterward. This rejects build commands that exit successfully only because artifacts were already current. Fresh reference/candidate failures retain their exact benchmark variant, phase, exit status, timeout, and diagnostic tail for the next strategy request.

At phase boundaries, Eleos keeps the LLM's working context small without discarding the evidence needed for the next action. The selected branch receives a bounded, deduplicated set of exact discovery results from strategy. That evidence survives rejected branches and session resume while the source tree is unchanged; a committed candidate invalidates old source observations. After every completed direction, the separate Observer LLM receives only that direction and its mechanical result in a fresh conversation and produces a short semantic summary. Every later strategy request receives the complete summary index, including the earliest attempts. Detailed recent outcomes remain bounded in the live prompt, while the complete tool stream and hypothesis history stay archived and searchable.

For opaque failures, the Agent receives a compact inventory of diagnostic executables discovered on the current platform. run_diagnostic can inspect that inventory, run a tool with an explicit argv, or keep an observer running around one exact target sample. Diagnostic output and dumps/traces written to the reported session diagnostic directory are archived as evidence but never affect candidate acceptance; only run_target can verify a change.

During branch and consolidation work the Agent also has run_shell, a full host shell with arbitrary accessible working directories, structured environment overrides, stdin, output decoding, and explicit timeouts. It is intended for real build commands, local experiments, process inspection, and debugger orchestration that cannot be expressed by the immutable target contract. Complete output is archived under the session; the TUI shows the exit status, output tail, and log path. Shell results never satisfy candidate acceptance, and source changes continue through transactional edit/create tools.

Eleos is not a sandbox. Init and Agent shell tools run with the permissions of the current operating-system user and may execute commands in any accessible working directory. Transactional source-edit tools and declarative runtime plans have narrower project boundaries, but those boundaries do not reduce the technical authority of the host shell.

Run Eleos only in an environment where that authority is acceptable. Prefer a disposable worktree, container, virtual machine, or dedicated low-privilege account. Do not expose production credentials, deployment control planes, SSH agents, or unrelated writable data to an Eleos process.

The configured LLM provider may receive repository metadata, selected source content, prompts, commands, and command output. A session also persists detailed local evidence under .eleos/sessions, including transcripts and tool results. Treat both provider traffic and session archives as potentially sensitive; inspect archives before sharing them. Provider usage may incur cost.

From a source checkout:

cd eleos
python3 -m venv .venv
source .venv/bin/activate
python -m pip install .

On Windows PowerShell, activate the environment with .\.venv\Scripts\Activate.ps1 instead.

Eleos does not provide an LLM account, API key, proxy, or bundled offline model. A provider and API key are required. Before starting either the TUI or headless mode, choose a provider, obtain an API key from that provider, and expose it to the Eleos process through environment variables. The --model value must use the matching LiteLLM provider prefix.

For example, choose one provider configuration in the same terminal that will launch Eleos. Replace every placeholder, including the model name:

# OpenAI
export OPENAI_API_KEY="<your-openai-api-key>"

# Anthropic
export ANTHROPIC_API_KEY="<your-anthropic-api-key>"

# An OpenAI-compatible service
export OPENAI_API_KEY="<your-provider-api-key>"
export OPENAI_API_BASE="https://provider.example/v1"

PowerShell uses $env:OPENAI_API_KEY = "<key>", $env:ANTHROPIC_API_KEY = "<key>", and $env:OPENAI_API_BASE = "https://provider.example/v1". Other providers use the environment variables and model prefixes documented by LiteLLM.

Never paste an API key into the TUI input, an Eleos prompt, a project file, or a command argument. Prefer your shell's secret facility or a dedicated low-privilege environment; environment variables are inherited by commands launched from that terminal.

The chosen model must reliably support tool calling and enough context for repository work. The Observer uses the same model and credentials unless --observer-model selects another provider/model combination.

After installing Eleos and configuring the provider and API key, start the TUI by omitting --headless:

python -m eleos /path/to/project --model openai/<model>

Describe the target, representative workload, objective, constraints, and correctness requirements. Eleos inspects the project, presents SESSION_PROPOSAL, and automatically runs the contract preflight.

The repository includes a deliberately naive prime-summing program and semantic tests under examples/slow_primes. This example exercises LLM-authored init, automatic preflight, transactional editing, candidate measurement, correctness validation, acceptance, and session reporting.

First verify the unoptimized project from the repository root:

cd examples/slow_primes
python -B slow_primes.py 20000
# 21171191

python -B -m unittest -q
# Ran 2 tests ... OK
cd ../..

The initial implementation uses trial division up to each candidate value. On the development machine it takes about 1.7 seconds for this workload; absolute time will vary by machine.

With the provider environment variables set, launch the example without --headless:

python -m eleos \
  --mode runtime \
  --run-command "python -B slow_primes.py 20000" \
  --test-command "python -B -m unittest -q" \
  --build-input slow_primes.py \
  --constraint "preserve sum_of_primes(limit) semantics for arbitrary non-negative limits" \
  --constraint "do not add third-party dependencies" \
  --benchmark-runs 3 \
  --min-improvement-percent 5 \
  --model openai/<model> \
  examples/slow_primes

PowerShell users can put the command on one line or replace each trailing \ with a backtick. If a session picker appears, choose New Session. When the Agent pane asks what to optimize or repair, enter this complete request:

Optimize slow_primes.py. Measure the exact workload that sums all primes below 20000
and make it at least 3x faster. Preserve the printed result and the general
sum_of_primes(limit) behavior for arbitrary non-negative limits. Keep the existing unit
tests passing and do not add third-party dependencies.

Init then shows repository and provider progress, submits and renders the exact SESSION_PROPOSAL, and runs preflight automatically. No confirmation click is required. Once Agent work starts, ordinary text in the Agent input is queued as context for the next model request; /reject reason rejects and rolls back the active direction. The Observer input or /ask question can explain recorded evidence but cannot change the source or acceptance decision. Press F2 to inspect the current model context.

For the same test without Textual, keep the provider variables set and pass the complete objective on the command line:

python -m eleos --headless \
  --mode runtime \
  --target slow_primes.py \
  --workload "sum all primes below 20000" \
  --objective "Make the exact workload at least 3x faster" \
  --run-command "python -B slow_primes.py 20000" \
  --test-command "python -B -m unittest -q" \
  --build-input slow_primes.py \
  --constraint "preserve sum_of_primes(limit) semantics for arbitrary non-negative limits" \
  --constraint "do not add third-party dependencies" \
  --benchmark-runs 3 \
  --min-improvement-percent 5 \
  --model openai/<model> \
  examples/slow_primes

The run should proceed through these mechanically visible stages:

1. Init inspects the example, runner-verifies the proposed commands, and prints a SESSION_PROPOSAL that times python -B slow_primes.py 20000, preserves stdout 21171191, and runs the unit tests as validation.
2. Automatic preflight executes that exact contract and records the baseline. It is reused as the initial Agent evidence rather than immediately benchmarked again.
3. Agent investigates and edits slow_primes.py transactionally. Each candidate must keep exact stdout, pass both unit tests, exceed the 5% per-candidate threshold, and contribute toward the 3x session objective.
4. On success, Eleos prints Done. Best: ... (...x speedup) and writes the final report into the example's .eleos session archive. If the model cannot produce a verified improvement, Eleos retains the original or last verified source and reports the blocker instead of accepting an unverified edit.

Inspect the accepted result and its evidence:

git diff -- examples/slow_primes/slow_primes.py
python -B examples/slow_primes/slow_primes.py 20000
python -B -m unittest discover -s examples/slow_primes -q
python -m eleos examples/slow_primes --list-sessions

# Substitute the session ID reported above.
python scripts/render_session.py \
  examples/slow_primes/.eleos/sessions/<session-id> \
  --output session.html

To repeat the demonstration from the deliberately slow baseline:

git restore -- examples/slow_primes/slow_primes.py

LLM decisions and measured speedups are intentionally not deterministic. The fixed parts of this test are the workload, correctness output, validation suite, acceptance rules, transaction behavior, and archived evidence.

A repair request is ordinary natural language; no exit code is required:

Reproduce the shutdown failure, find and fix its underlying cause, make the workload
exit successfully, and preserve the completed semantic output.

The TUI input remains active throughout the session. Plain text entered during init, preflight, strategy, branch work, verification, or consolidation is queued for the next LLM request and retained across phase transitions. It does not cancel the provider request, tool, command, or measurement already in progress. /reject is the explicit exception: it immediately rejects the current direction, cancels its pending model/tool execution, rolls back its transactional edits, and moves on; /reject reason records the reason with the rejected hypothesis.

A separate read-only Observer explains strategy results and branch/consolidation transitions in its own TUI conversation pane. Enter a question in the Observer input, or use /ask <question> from the Agent input, to ask about recorded session events. Every non-empty Observer query calls the Observer LLM. Each request receives the complete direction-summary index, a few recent Observer turns, bounded recent session evidence, and focused older evidence when lexical retrieval finds it. Retrieval only controls the context supplied to the model; there is no local relevance gate, including when no session events have been recorded yet. Observer has no project tools and never enters the Agent input mailbox or controls Worker decisions. Its isolated per-direction summaries are the sole deliberate input from Observer calls to Worker: they form the complete strategy-memory index without sharing either model's conversation history. Select a different model with --observer-model; otherwise it uses --model.

After preflight, Eleos locks a timeout cap for candidate trials. The automatic cap is max(300 seconds, 3 x baseline timed wall time) and never raises a lower timeout already declared by the contract. It applies to timed, metric, output, observation, and validation commands, but not to untimed preparation such as a required build. Override it with --trial-timeout-s SECONDS when the workload has a known bound.

For automation, headless mode runs the same LLM-authored init before optimization:

python -m eleos --headless \
  --objective "Make the requested incremental build workflow 20% faster" \
  --constraint "preserve public behavior" \
  --model anthropic/<model> \
  /path/to/project

--target, --workload, and --mode are optional init hints, not adapter selectors or execution policy. If omitted, the init LLM infers their display/context values from the objective, repository evidence, and executable contract. If a hint conflicts with verified project evidence, the contract value wins. Commands, measurement scope, and acceptance always come from the contract.

The model calls submit_session_contract with a generic goal and typed plans.

Commands are argv arrays, never implicit shell strings. Cleanup is not a command: reset paths may name files, symbolic links, or directories, but must stay under dedicated .eleos/build/... or .eleos/measurements/... storage. This lets the LLM choose project semantics without receiving unbounded deletion authority.

Init can use run_contract_command to execute an exact proposed CommandSpec through the same runner as preflight, including argv, cwd, environment overrides, timeout, process-group handling, optional verbatim stdin, and captured output. Explicit interpreters can receive scripts through stdin when their command-string argv parsing is not stable. run_shell is broader host-shell exploration and is deliberately not considered proof of equivalent argv behavior.

Repair contracts declare completion_assertions, fresh_output_paths, or both. Timed stdout/stderr assertions prove the workload reached the relevant stage. Fresh paths are snapshotted after preparation and must contain a file created or changed by that timed sample, so stale external output cannot validate a failure. External freshness paths are observed read-only and are never reset or deleted by the harness. If the workload's semantic evidence exists only in captured process output, output_command_input routes the current timed stdout, stderr, or their combination to the output probe's stdin for deterministic summarization.

The effective sample count remains harness-controlled. A contract may suggest a count, but the baseline pilot and explicit CLI policy determine and lock it.

flowchart TB
    User[User objective and constraints] --> Init[LLM init]
    Inventory[Generic file inventory] --> Init
    Read[Read-only exploration tools] --> Init
    Shell[Full init shell] --> Init
    Exact[Exact contract command runner] --> Init
    Init --> Contract[Validated SessionContract]
    Contract --> Runtime[Generic contract interpreter]
    Runtime --> Runner[Async argv runner]
    Runtime --> Gate[Output and assertion gate]
    Contract --> Agent[Optimization / repair agent]
    Agent --> Tools[Read / search / edit / measure]
    Tools --> Runtime
    Agent --> Evidence[Sessions / candidates / hypotheses / benchmarks]
    Evidence --> Observer[Read-only Observer]
    Observer --> User

The boundary is deliberate:

• LLM: project understanding, experimental design, hypotheses, and code changes.
• Harness: schemas, path safety, process execution, timing, rollback, and acceptance.

An unfamiliar project normally requires only a different LLM-authored contract. If the current primitives cannot express a needed capability, Eleos extends the generic contract and interpreter rather than adding a project adapter.

Every run writes an authoritative session archive:

.eleos/sessions/<session-id>/
  session.json
  checkpoint.json
  preflight.json
  transcript.jsonl
  benchmark-observations.jsonl
  diagnostics/
  cache-stats.json
  conversation.md
  report.md

Render the full session as HTML:

python scripts/render_session.py \
  /path/to/project/.eleos/sessions/<session-id> \
  --output session.html

The renderer combines LLM requests, tool calls, harness events, contract data, benchmark samples, cache statistics, and the final report.

After preflight, Eleos atomically checkpoints the accepted contract, locked benchmark policy, exact correctness baseline, compacted Agent trunk, and mechanical progress only at transaction-free strategy boundaries. Provider transport failures retry three calls per automatic burst with capped exponential backoff. If the burst still fails, Agent work pauses in place; /continue retries without changing the prompt, while queued ordinary input is applied to the retry request. A stopped process can resume any resumable archive by ID; latest is only a convenience alias over the same archive index:

python -m eleos /path/to/project --list-sessions
python -m eleos /path/to/project --resume <session-id>
python -m eleos /path/to/project --headless --resume latest

An interrupted branch has a durable rollback journal. Resume restores it only when the file still matches Eleos's recorded write; a later external edit causes an explicit conflict instead of being overwritten.

Init also supplies a concise display-only session title as contract metadata. It never affects runtime behavior. When the TUI starts without an explicit --resume and finds resumable archives, it opens a session picker showing title, status, last activity, and session ID. Choose an archive to continue, or choose New Session to enter init.

Provider controls are ELEOS_LLM_TIMEOUT_S (per-call timeout, default 180 seconds), ELEOS_LLM_RECONNECT_ATTEMPTS (calls per automatic burst, default 3), and ELEOS_LLM_RECONNECT_INITIAL_S / ELEOS_LLM_RECONNECT_MAX_S (backoff bounds).

--headless                    run without Textual
--model MODEL                 Worker model routed through LiteLLM
--observer-model MODEL        read-only Observer model (defaults to Worker model)
--list-sessions               list archived sessions and resumability
--resume ID                   resume an archived session ID or latest
--trial-timeout-s SECONDS     candidate command timeout cap (0 selects baseline policy)
--target TEXT                 target/workflow hint for init
--workload TEXT               representative workload hint
--objective TEXT              performance target or repair outcome
--mode runtime|build          optional display-category hint for init
--constraint TEXT             binding constraint; repeatable
--context-file PATH           UTF-8 workflow/background document; repeatable
--benchmark-runs N            explicit sample count; 0 uses harness policy
--min-improvement-percent N   acceptance threshold
--no-cache                    disable exact LLM/cache reuse
--inspect                     print generic repository inventory only

Explicit --run-command, --build-command, --test-command, --artifact, and --build-input values are passed to init as binding user constraints. They do not select a built-in backend.

• Natural-language constraints guide the model but are not all mechanically provable.
• Validation quality is only as strong as the accepted session contract and baseline output oracle.
• Observation commands return raw diagnostics. Eleos does not parse tool-specific profiler formats.
• Performance results remain workload- and machine-specific.
• A repair baseline must fail reproducibly; candidates cannot normalize the exit code, suppress the failure, alter the reproducer, or weaken contract assertions.

For a long or operationally complex benchmark, put the background in a UTF-8 Markdown file and pass it with --context-file. Include the exact preparation/deployment commands, environment variables, timed boundary, representative inputs, validation commands, success assertions, files expected to be relevant, and important side effects. Init explores the repository, distills that document into a proposed contract, and shows the exact plan before execution. The full document remains in the session transcript while later Agent phases receive compact contract rationale and evidence.

If the objective concerns one stage inside a larger application, the contract may use a numeric metric probe over artifacts emitted by that exact workload. Eleos will not silently treat whole-application time as the stage metric.

python -m pip install -e ".[dev]"
python -m compileall -q eleos
python -m pytest -q

Read AGENTS.md before changing init, measurement, or acceptance behavior. The durable invariants are in docs/harness-design-contracts.md, and the init-time session contract is described in docs/session-detector.md.

Eleos is licensed under the Apache License 2.0. See LICENSE.