CASMO (Confidence-Adjusted Signal-to-noise Momentum Optimizer) is a drop-in
replacement for torch.optim.AdamW that scales each coordinate's update by the
signal-to-noise ratio of its gradient.
from casmo import CASMO
optimizer = CASMO(model.parameters(), lr=1e-3, weight_decay=0.01)| Document | What it covers |
|---|---|
| Getting Started | Install, first training loop, choosing robustness, monitoring |
| API Reference | Every parameter, method, and exception |
| Migration Guide | Coming from AdamW, or upgrading from CASMO v0.3 |
| FAQ | Concepts, usage, performance, troubleshooting |
| Theory | The derivation of AGAR and the confidence map |
| Redesign evidence | The experiments that produced the v0.4 design |
Adam adapts to gradient magnitude and cannot tell a strong consistent signal from large
random noise. CASMO tracks the gradient's first moment m (signal) and its centered
variance s (noise), and forms a per-coordinate signal fraction
AGAR = m̂²/(m̂² + ŝ) ∈ [0,1]. That fraction drives two multipliers: trust, an
absolute per-tensor pace that shrinks when gradients are noise-dominated, and focus, a
relative per-coordinate weight that shifts the step toward reliable directions. The
robustness dial sets how strongly the absolute axis applies — 0 behaves like AdamW,
1 is maximally noise robust.
CASMO(params, lr=1e-3, robustness=1.0) # noisy labels, long-tail, continual learning
CASMO(params, lr=1e-3) # default 0.5 -- general purpose
CASMO(params, lr=1e-3, robustness=0.0) # clean data, DP-SGD, maximum speed5 seeds, mean ± stdev, clean test labels. Reproduce with
python research/validate_redesign.py.
| Regime | Adam | CASMO |
|---|---|---|
| 30% label noise | 0.675 ± 0.020 | 0.810 ± 0.008 (ρ=1.0) |
| 15% label noise | 0.764 ± 0.022 | 0.857 ± 0.016 (ρ=1.0) |
| Clean data | 0.931 ± 0.010 | 0.934 ± 0.007 |
| High LR (steps) | 70 | 38 (ρ=0.5) |
| Gradient noise σ=0.5 | 0.641 ± 0.037 | 0.383 ± 0.035 (ρ=1.0) |
The last row is a real limitation, not an omission: label noise and isotropic gradient
noise want opposite policies. Use low robustness for DP-SGD-style workloads.