_{v0.8 Strict SR is shown as official LeWM / paired random, rounded from the mean over seeds 0, 1, and 42.}

Minimal evaluator repair when comparability matters. Task-semantic precision when completion matters.
CLEAR-LeWM removes pre-solved cases, repairs demonstrated evaluator defects, freezes paired manifests, and exposes every success decision for audit.

Junhan Sun1    Guofeng Zhang1,†    Hao Zhao2,†
1State Key Laboratory of CAD&CG, Zhejiang University  |  2Tsinghua University  |  †Corresponding authors

Website · Releases · Results · Modes · Specification · Submit Results · Checkpoints

Single-task INTACT E1 (history). Red marks the best SR for each task and mode.

Unified INTACT E5. CEM references are single-task, not epoch-matched. Protocol and seed statistics.

• 2026-09-14 - Added public INTACT Direct and Guarded A results on the frozen CLEAR v0.8 Moderate/Strict manifests (RTX 4090, three training seeds and three evaluation seeds).
• 2026-09-13 - Added the INTACT paper citation for researchers building on CLEAR-LeWM's auditable evaluation.
• 2026-09-09 - Merged PR #7 from @ma-kjh, adding configurable Adam and DINO-WM optimizer profiles, device-safe stable-worldmodel==0.1.0 support, and deterministic planner regression tests while keeping CEM canonical.
• 2026-08-28 - Released v0.8.0, closing the Reacher action-repeat termination leak and publishing an audited 84-run RTX 4090 reference matrix with corrected documentation and media.
• 2026-07-24 - Accepted the first community result submission in PR #1 from @zerotul782231, covering DINOv2 No-Proprio and GCBC Joint LeWM checkpoints under the frozen v0.5 protocol.
• 2026-07-23 - Released v0.5.1 with the complete three-seed, 48-run reference matrix after v0.5.0 unified the benchmark around separate Moderate and Strict modes.
• 2026-07-22 - Released v0.3.0, introducing task-semantic completion rules, deterministic manifests, matched random controls, and task-specific audit guides.
• 2026-07-21 - Released v0.2.1 with audited FAST loading and explicit runtime modes after v0.2.0 established the reproducible three-tier evaluation standard.
• 2026-07-21 - Published v0.1.0, the initial four-task CLEAR-LeWM evaluation release with deterministic manifests, initial-success filtering, and paired random baselines.

The historical stack mixes genuinely difficult control with evaluator effects: initially solved start-goal pairs, incorrect Reacher angle topology, early dm-control termination before CLEAR can score the rollout, and a TwoRoom rewrite whose endpoint-only collision check can admit an invalid wall crossing. Cube also has a high random floor because many sampled windows do not move the cube.

CLEAR-LeWM separates two scientific questions instead of forcing one rule to answer both.

Moderate is the closest corrected continuation of the released benchmark. Strict is the stronger semantic claim. They must be reported as separate columns. Exact inequalities and runtime gates are normative in EVALUATION_SPEC_V08.md.

The primary table uses pinned official high-epoch LeWM checkpoints and paired random policies. All runs were independently executed on RTX 4090 GPUs with 100 episodes, 300 x 30 CEM, top-k 30, solver batch size 1, and strict 303/303 tensor loading.

Mean +/- sample standard deviation across complete paired runs:

Mean +/- sample standard deviation across complete paired runs:

The JSON files in results/v0.8/runs/ are the source of truth. They include all episode outcomes, manifest hashes, criteria, solver settings, environment fingerprints, checkpoint hashes, and topology diagnostics.

Moderate preserves the complete pusher-plus-block goal state. Strict asks the task-semantic question: is the T itself placed correctly? A full-image latent cost can therefore differ from Strict completion, and methods should disclose their planning target. Read the PushT guide.

Moderate follows OGBench's 4 cm cube-position task. Strict adds a 3 cm position gate and 15 degree orientation modulo all 24 proper cube rotations. Neither mode scores terminal robot pose. Read the Cube guide.

Moderate repairs the shoulder/wrist topology while preserving joint matching. Strict scores the physical fingertip endpoint for two consecutive steps. Both modes disable the environment's own task termination throughout rollout, including after reset and recompilation, so only the declared CLEAR predicate can terminate scoring. Read the Reacher guide.

Both modes reject polluted source windows and execute corrected swept-disk physics. Strict additionally requires a legal room crossing, goal-side arrival, and an 8 px endpoint. Read the TwoRoom guide.

git clone --recurse-submodules https://github.com/DavidSunok/CLEAR-LeWM.git
cd CLEAR-LeWM
python -m venv .venv
source .venv/bin/activate
pip install -e '.[dev,lewm]'
python scripts/prepare_official_checkpoints.py --cache-dir "$STABLEWM_HOME"

Evaluate the identical Strict pair set with random and official LeWM:

clear-lewm evaluate \
  --manifest manifests/v0.8/tworoom/strict-seed42-n100.json \
  --policy random --cache-dir "$STABLEWM_HOME" \
  --dataset-path /path/to/tworoom.h5 \
  --solver-batch-size 1 \
  --output results/tworoom-v08-strict-random.json

clear-lewm evaluate \
  --manifest manifests/v0.8/tworoom/strict-seed42-n100.json \
  --policy official/tworoom/weights.pt --policy-label official-lewm \
  --cache-dir "$STABLEWM_HOME" \
  --dataset-path /path/to/tworoom.h5 \
  --num-samples 300 --n-steps 30 --topk 30 \
  --solver-batch-size 1 --strict-checkpoint \
  --random-results results/tworoom-v08-strict-random.json \
  --output results/tworoom-v08-strict-lewm.json

Canonical v0.8 reference runs use pure CEM with --actor-warmstart off; no alternative inference contract is mixed into the reference table.

The upstream LeWM Adam planner can be selected explicitly with --planner adam. CLEAR applies a narrow compatibility adapter for stable-worldmodel==0.1.0: a full-horizon initialization is moved to the solver device before CUDA sample noise is added. Omitting --planner preserves the canonical CEM default. Alternative planners are experimental/non-reference profiles for robustness studies, not canonical v0.8 reference runs.

--planner dinowm-gd applies DINO-WM's optimizer profile to LeWM visual latents: random-normal initialization, 1,000 manual SGD updates at learning rate 1, Gaussian action noise 0.003, and terminal visual-latent MSE with mean reduction. It does not reproduce the DINO-WM model: the adapter has neither the DINO encoder contract nor DINO-WM's proprioceptive objective. Actor-prior initialization is unsupported; an explicit --actor-warmstart on is rejected. The default auto setting resolves to off for this planner. Use --n-steps for shorter diagnostic runs. Result JSON records the initialization, source revision, model scope, finite-value checks, and aggregate solver runtime. No performance advantage is claimed.

FAST is an optional derived reader, not a new dataset. It decodes once into verified row-major memory maps while preserving complete action chunks and episode boundaries.

The four-task geometric-mean loader speedup is 17.87x. These figures exclude conversion and model compute. See PERFORMANCE.md.

Every matched comparison must share the manifest, dataset fingerprint, environment, policy seed, control budget, solver budget, solver batch size, and protocol mode. MuJoCo, Pymunk, dm-control, Gymnasium, PyTorch, CUDA, cuDNN, task source, checkpoint source, and evaluator source are fingerprinted.

The pinned official source revisions and hashes are in checkpoints/official-v0.5.json. Binary weights and datasets remain outside ordinary Git.

Public methods may submit auditable Moderate and/or Strict result bundles. CI verifies structure, canonical manifest hashes, trace arithmetic, provenance, and topology. It does not imply independent reproduction or endorsement.

This table compares published models with their stated inference methods on the same RTX 4090 CLEAR v0.8 Moderate/Strict manifests; it is not an actor-off, matched-compute model ablation. Official LeWM, DINOv2 No-Proprio LeWM, and GCBC Joint LeWM use pure CEM 300 x 30 (no actor prior). The three public INTACT checkpoint families use either Direct action prediction (no sampled search) or actor-guided Guarded A 128 x 3. Each evaluation seed 0/1/42 contributes 100 episodes. LeWM/DINOv2/GCBC use the published checkpoint for each task; INTACT additionally averages training seeds 0/42/3072, with all three evaluation seeds per training seed.

All entries are success rates (%). The macro is an equal-weight four-task mean, shown only for models with all four tasks. INTACT's +/- is the sample standard deviation across its three training-seed macro means; it is not the episode-level uncertainty or directly comparable to the three-evaluation-seed variation of the other models. DINOv2 and GCBC do not have released Reacher checkpoints.

Direct uses a five-action receding horizon and the checkpoint-matched actor. The history-enabled actor receives only actions before the sampled start, then its own executed actions; the no-previous-action actor ignores that history channel. No current or future expert action is supplied. These are CLEAR v0.8 scores, not Official Direct scores. Per-task variation and training-seed means for INTACT are in the INTACT evaluation record; the official/DINOv2/GCBC per-evaluation-seed variation, hashes, and episode traces are under results/v0.8/.

Canonical community entries use policy seed 42 and 100 episodes per task/mode. Values are model / paired random (excess). CI validates the bundle structure, canonical manifests, trace arithmetic, provenance, and topology; the verification label states whether execution was independently reproduced.

A dash means that task/mode was not submitted. Supplementary evidence that does not match the canonical manifest/seed contract remains in each method card and is not mixed into this table.

Read the submission guide.

If you find CLEAR-LeWM useful, please consider citing our INTACT paper:

@misc{sun2026intact,
  title         = {INTACT: Isomorphic Intent-to-Action Learning for Search-Free World Models},
  author        = {Sun, Junhan and Zhao, Hao and Zhang, Guofeng},
  year          = {2026},
  eprint        = {2607.26056},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2607.26056}
}

CLEAR-LeWM is MIT licensed. Upstream LeWM, stable-worldmodel, OGBench, DMC, Pymunk, MuJoCo, PLDM, and DINO-WM components remain under their own licenses and attribution requirements.