Tags: SauersML/gam
Tags
release: gam 0.3.117 / gamfit 0.1.203 crates.io catch-up release for the `gam` crate. gam was last published at 0.3.116 (= gamfit 0.1.199); everything shipped to the gamfit wheel since then (0.1.200–0.1.203: #979 batched coord_corrections perf, the Phase 0+1 cross-fit warm-start foundation, the noise-floor inner-Newton guard, the BMS separation false-positive fix) is engine work that lives in the `gam` crate and was not on crates.io. This bumps the crate to 0.3.117 and tags v0.3.117 so crates.io consumers get it. No code change — only the version bump + CHANGELOG entry. gamfit stays at 0.1.203 (already published to PyPI); pyproject.toml is untouched, so the wheel publish gate is a no-op on this push. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fix(spatial): degrade joint κ non-convergence to baseline, not fatal #… …1126 A 1-D Gaussian measure-jet smooth `y ~ s(x, bs="mjs")` was unfittable through the formula/FFI path (`fit_from_formula`, the path `gamfit.fit` takes) even though the `gam` CLI fits the identical data, aborting with `IntegrationFailed { reason: "... anisotropic analytic optimization did not converge after 80 iterations ..." }`. Root cause ---------- The joint spatial-κ optimizer (`run_exact_joint_spatial_optimization`) is a *refinement* layered on top of an always-valid frozen baseline geometry (the REML-seeded length scales in `best`). It is best-effort by nature, yet a run that hit its iteration cap without certifying a stationary point — and without clearing the relative-to-cost acceptance gate — was escalated to a fatal error (`spatial_optimization.rs` bail). The existing baseline fallback only fired when the optimizer *converged but worsened* the profiled score, never when it failed to converge. The formula path provokes exactly this: `materialize` builds `FitOptions` with the tightened outer tolerance `tol = 1e-10` (the #893 `w=c ⇔ c-fold replication` invariance tolerance), where the CLI uses `tol = 1e-6`. At the looser tolerance the same optimizer converges in ≤80 iters; at the tighter one it is left mid-descent at the cap with a large projected gradient, and the non-convergence aborts the whole fit. Approach -------- Make the joint optimizer best-effort, matching how mgcv treats smoothing-parameter optimization. `run_exact_joint_spatial_optimization` now returns a `SpatialJointOutcome`: * `Optimized` — converged, or terminal iterate accepted under the mgcv-style relative-to-cost REML criterion (unchanged behavior); * `NonConverged` — ran to a *finite* terminal cost but neither converged nor cleared the gate. The caller keeps the frozen baseline geometry rather than aborting. A genuinely non-finite terminal cost (NaN/inf in the gradient/Hessian wiring) still surfaces as `Err` — that is a real numerical blowup, not ordinary slow convergence, and must not be masked by the fallback. The two non-adoption paths (non-convergence, and converged-but-worse) now share one `fit_frozen_baseline_geometry` helper that re-fits at `best`'s lambdas + frozen spec and stamps the certified baseline REML score, so the spatial-κ result gate stays consistent. Why this over aligning the tolerance: loosening the formula path to `tol = 1e-6` would weaken the #893 replication invariance for every fit and would only paper over this one dataset (the report notes ~60% of random datasets fail the Python path). The robustness fix is general — it protects every kernel-representer basis (Matérn, Duchon, mjs) on any data where the κ optimizer needs more than 80 iters — and preserves the tight tolerance. Scope: the sibling latent-coordinate joint optimizer (the SAE/manifold path) is deliberately left fatal on non-convergence — there the latent coordinates *are* the model, not an optional refinement, so no valid baseline geometry exists to fall back to. Verification ------------ * `bug_hunt_measure_jet_formula_fit_aborts_at_tight_outer_tol` (the committed repro) now passes. * New `measure_jet_formula_fit_recovers_signal_on_nonconvergence` attacks the same root cause from the prediction side on an independent jittered-grid dataset: it asserts the fallback geometry recovers sin(2πx+0.7) to RMSE < 0.15 (a flat/degenerate fit would be ~0.71), proving the fallback is genuinely usable, not merely non-aborting. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test(bms): sync seed-screen invariant to production cap=8 floor The BMS marginal-slope outer seed-config screen invariant test pinned `screen_max_inner_iterations == 2`, but the production `outer_seed_config` deliberately raised the screening cap 2->8 in d388d12 ("first viable reachability floor"): two cycles sit below the observed KKT reachability floor for these startup seeds, rejecting every candidate and then immediately paying a second pass at cap=8. The invariant test was left asserting the stale pre-change value, so it has failed since d388d12 (already red at the 0.1.204 baseline). Update the pinned value to the intended production setting (8) and document why two cycles is too low. This is a stale-assertion sync, not a weakening: the assertion remains an exact equality and no tolerance or quality bound is relaxed; it now detects unintended drift from the deliberate, documented cap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fix(build): read tracked-file list from .git/index, not the workspace Wheel builds failed with: error: 1 ban violation across 1 rule ── tracked file over 10k lines (split the file; issue #780 line-count gate) cuda_installer-linux-6.8.0-1052-azure-x64-12.2.2/ cuda_installer-linux-6.8.0-1052-azure-x64_12.2.2.run:16117691: 16117691 lines; limit is 10000 The Build and Release All workflow runs a "Install CUDA" step that drops the 16M-line installer payload into the workspace at <root>/cuda_installer-*/, and the line-count audit then panics on it. Root cause: the audit was supposed to inspect "repo files" but collect_repo_files walked the filesystem under the manifest root and included every file not on a hardcoded skip list. The skip list (target, node_modules, dist, …) cannot enumerate every artifact a workflow step may write into the workspace, so the audit was always one workflow change away from breaking. The previous incarnation shelled out to `git ls-files`, which was correct in concept (only tracked files) but unreliable in practice: inside maturin's manylinux/musllinux Docker images the bind-mounted source trips git's safe.directory check and the command returns non-zero, panicking the build that way instead. Fix: parse .git/index directly. The git index is the canonical definition of "what belongs to this repo" — it lists exactly the staged tree, ignores anything CI dropped into the workspace, and needs no external git binary (so it works inside maturin Docker without any safe.directory dance). Supports index v2, v3, and v4 (v4 path-prefix compression and the offset-style varint), plus the linked-worktree case where .git is a file pointing to the real gitdir. Drops both the workspace-pollution bug and the git-command dependency in one change.
fix(family-resolver): accept mgcv-style `family(link)` syntax
`resolve_family` only canonicalized via `replace('_', "-")`, so
`binomial(logit)` (mgcv's standard R spelling for "binomial family with
the logit link") fell through to the `_ =>` arm and the workflow
aborted with `unknown family 'binomial(logit)'`. Three in-repo test
files pass this string straight through to the resolver
(`sphere_logit_predict_finite_at_pole` in `broad_sweep_batch_i`,
`sphere_wahba_binomial_logit_fit_succeeds_and_predicts_finite` in
`sphere_binomial_logit_fit`, and the m4 sphere binomial pseudo test),
and the docs explicitly advertise `family(link)` as the spelling
mgcv users carry over.
Root cause: the canonicalizer ignored the parenthesized form; the
match table only ever saw `family-link` / `family_link` / `family`.
Fix: after lowercase + underscore-to-hyphen, if the name has a `(...)`
tail, peel it and re-emit as `head-inner` (whitespace-trimmed, with a
trailing `-` on the head removed so `negative_binomial(log)` lands
correctly on `negative-binomial-log`). Names whose head or inner part
is empty fall back to the literal lowered string and the original
"unknown family" error path, so genuinely malformed inputs still get
the same diagnostic.
Adds a unit test pinning `binomial(logit)` / `Binomial(Logit)` /
`binomial(LOGIT)` / `binomial( logit )` / `binomial(probit)` /
`Binomial(CLogLog)` / `negative_binomial(log)` to their canonical
`(response, link)` pairs.
https://claude.ai/code/session_01NUaW5HqqLicxEPnECRQtZG
release: gam 0.3.116 / gamfit 0.1.199 Rolls up the unreleased work since 0.3.115. Highlights: - Interval-censored survival from a formula via SurvInterval(L,R,event) (#1108); full multinomial prediction inference — intervals/SE/posterior_predict/Wald (#1101); expectile GAMs (#1100); magic Poisson auto-detect (#1065); RMST output; exact full-conformal prediction intervals (#1054/#942); per-term LR tests with Lawley-Bartlett (#1063); residual-cascade auto-route (#1032). - SAE/manifold interpretability: SPD/Grassmann/Stiefel/Poincare response geometries (#1061); ConstantCurvature + curvature-as-estimand inference (#1104/#944); Riesz debiased functionals, per-atom curvature CI, checkpoint dynamics, atom-smooth e-value significance (#1097/#1099/#1102/#1103/#1055). - Solver robustness: compass-search removed entirely (could hang) with callers on exact-gradient BFGS; arrow/Schur Woodbury fails loud over silent NaN; no wall-clock fake-convergence; gradient-tolerance-gated cost-stall; Gershgorin spectral ridge; no fake-PSD covariance clamping. - Perf: CPU-resident reduced-Schur SAE matvec (#1017); parallel competing-risks CIF assembly (#1082). Build, rustfmt, and the clippy correctness/suspicious gate are green. Known perf/coverage gaps (#1082, #1116, #979) are tracked, not shipped-surface correctness defects. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fix(#1099): wire delta-method curvature SE through to python curvatur… …e_report (unify with dictionary_report) The python ManifoldSAE.curvature_report serializer still draped the old profile-CI 'unavailable' framing (ci_method/verdict='unavailable', flatness_lr_stat/p_value, ci_lo/hi=None, 'fixed-kappa profile criterion' note) around the plug-in kappa_hat — the category-error #1099 rescoped away and #1115 deleted everywhere else. #1115 (39f5639/e5131a6f9) removed the AtomCurvatureCi delta-method SE machinery entirely: a sup-norm curvature BOUND is not an estimand with a profiled criterion, and the SE was conditioned on the generated latent coordinates as if known (omitting the generated-regressor channel) and under-covered. So the honest unification is the point bound only, matching the source of truth (CertificateInputs::per_atom_kappa_hat, shared with dictionary_report) — NOT re-introducing the deleted under-covering CI. New python dict shape: {note, atoms:[{atom, kappa_hat}]}. Dropped the ci_available/ci_method/level/verdict/flatness_lr_stat/flatness_p_value/ ci_lo/ci_hi/lo_at_bound/hi_at_bound keys from the SAE curvature path (the separate Model.curvature profile-CI estimand path is untouched). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fix(#1119): assemble block-diagonal joint Hessian for orthogonal disp… …ersion location-scale families WIP: NB/Gamma/Tweedie noise_formula fits returned None for the joint coefficient Hessian (only Beta built it), which stranded the multi-block outer-REML path ('multi-block families must provide a joint outer path'), forced a degraded ρ-seed escalation, and left covariance/EDF unset so predict could not run. Build the (block-diagonal, zero-cross) joint Hessian for all four members and declare likelihood_blocks_uncoupled for the Fisher-orthogonal members so the directional-derivative dispatch accepts it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fix(ci/benchmark): install R unconditionally so the runtime-cache-hit… … path can still verify The Benchmark Suite workflow has been failing on every cache-hit bootstrap since the runtime cache was introduced. `bootstrap-runtime` only invoked `./.github/actions/bench-setup` (which installs R via `setup-bench-r`) when the runtime cache was MISSED, but the immediately-following `Verify staged R library is complete` step always shells out to `Rscript` to confirm the cached r-lib tree contains every required package. On every cache hit the bootstrap step therefore died with `Rscript: command not found` before the verify check could run, blocking the entire bench fan-out. Add an unconditional `Set up R for runtime verification` step before the verify check so Rscript is available regardless of the cache state. The r-lib toolcache makes the second install a fast no-op on cache miss where `bench-setup` already installed R in the same job. Recurring failure: Benchmark Suite job 'Bootstrap benchmark runtime' on the cache-hit path.
PreviousNext