CodeLeWM: A Reproducible, Claim-Gated Code World-Model Study with Negative Action-Use Results and a Narrow Downstream Reranking Slice

We study whether a JEPA-style latent code world-model can rerank candidate program edits without executing them, scoring candidates by predicted latent transitions rather than by generating patches. We release CodeLeWM, a manifest-backed research artifact: a ∼45M-parameter transition model with an EMA target encoder and LeJEPA-family SIGReg anti-collapse.

We evaluate it under a configuration-bound, control-first claim gate: the acceptance thresholds and the no-action, lexical, and shuffled-action controls are fixed in the training configuration before launch, and a machine-readable gate decides which claims may open. The outcome is one falsified hypothesis, non-collapsed diagnostic signals, and a single narrow downstream win that does not open an aggregate claim.

The original commit-edit action-use hypothesis is negative: on 1,000 held-out retrieval examples, commit-edit text-action ranking (Recall@1 0.263, MRR 0.370) loses to a no-action baseline (0.441, 0.533). A separate execution-trace substrate is cleaner diagnostically: execution retrieval beats every control (Recall@1 0.254/0.273 vs. all controls below 0.024), and single-point mutation surprise reaches AUC 1.0 over 260 scored pairs. These are training-health signals, not downstream utility.

The only open downstream slice is HumanEval WS-D, a deterministic mutation-distractor reranking pack of n = 47 problems: CodeLeWM reaches 46/47 (0.979) on both seeds, beating the strongest non-trivial control, shuffled-action (0.894/0.915), by +8.5 and +6.4 points (4 and 3 problems; 95% bootstrap CIs [0.0, 19.1] and [0.0, 14.9] that include zero), and the no-action control by +10.6 and +8.5 points. The two seeds are a training-stability check on the same 47 problems, not independent replications. The aggregate model-quality claim stays closed: MBPP-Plus WS-D is saturated on its 17 problems because CodeLeWM, no-action, and lexical all reach 17/17, so the slice has zero headroom and is non-informative for any lift.

Representation probes do not beat controls, broad semantic surprise has no scorable pairs, and the downstream rows expose no standalone p_pass score.

We therefore present CodeLeWM as a reproducible negative-and-diagnostic study with one narrow positive slice, not as evidence that the model generally improves coding, together with a released anti-saturation falsification harness whose gate is deliberately still closed. The main contribution is a reproducible artifact and a falsification protocol: code-world-model reranking claims need no-action, lexical, and shuffled-action controls