{"schema":"voidly-forecast-onset-skill/v1","model":"forecast-v1 (XGBoost + isotonic) — legacy 7-day current-regime score","evaluation_status":"historical_unvalidated","production_skill_verified":false,"headline":"Current shutdown-onset skill and probability-interval coverage are unvalidated.","plain_english":"An exploratory May 2026 audit found a strong persistence signal and poor new-onset timing. Its historical figures do not establish current served model skill.","label":{"name":"target_7day","construction":"7-day sliding window — 1 if any censorship/mixed incident in [T+1, T+7]","autocorrelation_day_to_day":0.989,"note":"Adjacent days share 6 of 7 look-ahead days, so the label barely moves day-to-day. High AUC on this label rewards persistence, not forecasting skill."},"metrics":{"v1_stratified_auc_leaky":0.954,"v1_stratified_auc_note":"Reported by /v1/forecast/model/info. Comes from a shuffled train_test_split that scatters adjacent autocorrelated days across folds — inflated, not deployment-representative.","v1_forward_temporal_auc":0.589,"v1_forward_temporal_f1":0.132,"v1_forward_temporal_note":"Exploratory past-to-future 60-day split. Issued model/version receipts and independently mature outcomes for current production were not verified.","predict_yesterday_baseline_auc":0.957,"predict_yesterday_baseline_f1":0.922,"predict_yesterday_baseline_note":"Trivial baseline forecast[t] = label[t-1]. Beats the model because the sliding window overlaps by construction — high score here is an autocorrelation artifact, not skill.","transition_row_auc":0.328,"transition_row_f1":0.105,"transition_row_note":"Exploratory subset where the label changes. Its below-0.5 AUC is a historical diagnostic, not a qualified estimate of current new-onset skill."},"correct_use":"Treat the forecast as an experimental current-regime score with SHAP drivers. Probability intervals are withheld pending coverage validation; do not use it as a new-shutdown alert.","variants_share_this":"The multi-horizon (1d/7d/30d) and per-region labels also use sliding windows with autocorrelation. Historical hourly evaluation used a temporal holdout, but current served skill is unvalidated.","finding_url":"/atlas/findings/forecast-v2-momentum-vs-persistence-2026-05"}