Skip to content

Open-source clinical ML field guide

Machine Learning & AI for Neurologists

Follow the full path from cohort definition to monitored clinical use—without confusing a polished model score with trustworthy evidence or causal benefit.

Begin with the clinical ML map Evidence register Choose by goal

  1. 1DefineCohort + outcome
  2. 2SplitPrevent leakage
  3. 3ValidateCalibrate + travel
  4. 4OperateMonitor decisions

CC BY 4.0 · Educational only — not medical advice · Source on GitHub

Expand model path
Clinical model path from cohort definition through validation and decision

Choose a route

FoundationsRebuild the minimum mathUse notation, probability, calculus, linear algebra, and optimization as a reference. OrientationSee the whole ML lifecycleFrame the task, split honestly, compare baselines, validate, and monitor. EvaluationRead performance correctlyConnect thresholds, calibration, prevalence, and decision consequences. Clinical usePlan for real-world failureInterrogate missingness, shift, fairness, leakage, workflow, and drift.

Build trust in layers

Four validation layers: data validity, internal validity, external validity, and clinical utility
Validation is cumulative. Later-stage performance cannot repair a biased cohort, contaminated split, or unstable label.
Layer Core question Evidence to inspect
Data validity Are the cohort, time zero, predictors, and labels credible? Sampling, label process, missingness, measurement, provenance
Internal validity Does performance survive honest resampling? Leakage-free split, simple baseline, calibration, uncertainty
External validity Does the model travel across place, time, and prevalence? Temporal and geographic validation, subgroup calibration, shift analysis
Clinical utility Does using the model improve decisions or outcomes? Thresholds, net benefit, workflow study, prospective impact, monitoring

Contents