LGD/EAD IRB Capital Modelling Framework

▶ Live validation dashboard — the frozen model evidence, rendered.

This repository implements a rigorously engineered Internal Ratings-Based (IRB) loss-given-default (LGD) and exposure-at-default (EAD) modelling framework for Fannie Mae single-family mortgage portfolios, modelled to align with CRR (Capital Requirements Regulation) and EBA Guidelines.

The model architecture:

• Realized LGD calculated from cash-flow resolution (Bluhm Eq. 1.7/1.8)
• Two-stage pipeline: cure classifier + severity regressor
• Downturn LGD calibration via HPI-anchored stress window (EBA/RTS/2018/04)
• Margin of Conservatism (MoC) per EBA/GL/2017/16

Note: A Credit Conversion Factor (CCF) module for revolving credit is retained as a deliberate demonstration of EAD methodology beyond the fully-drawn mortgage case (where EAD ≡ UPB per CRR Art. 166). It is not exercised in the mortgage pipeline.

• Source: Fannie Mae Single-Family Loan Performance Data
• Raw file: data/raw/fannie_sf_loan_perf.txt
• Format: Pipe-delimited, 113 fields, no header row
• Loading: lgd_engine.data_loader.load_fannie_raw() converts to data/processed/fannie_sf_loan_perf.parquet on first run
• Fields used: orig_upb, last_upb, oltv, fico, orig_rate, zb_code, lpi_date, disp_date, net sale/foreclosure/preservation/recovery/tax/misc costs, mi_proceeds

Data Gitignore: Raw and processed data files are excluded from git (data/ is in .gitignore). Users must acquire datasets independently from the source above.

No loan-level redistribution: the Fannie Mae licence forbids sharing the data or products derived from it, so this repository ships aggregates only — bin counts, per-bin means and summary metrics. The dashboard artifacts under app/artifacts/ are aggregates; the loan-level frames the notebooks read (reports/*_sample.parquet) and the fitted model binaries are not published. Rebuild them locally after acquiring the raw data:

python scripts/freeze_presentation_samples.py   # loan-level notebook samples
python scripts/build_app_artifacts.py           # aggregate dashboard artifacts

Tests that depend on those local files skip automatically on a fresh clone.

Discriminatory Power (Out-of-Time Sample, n=697,364):

• Champion Model (Delinquency Features, Tuned): Spearman rank correlation 0.5225
• Baseline (Origination Features Only): Spearman 0.4931
• Improvement: +295 basis points (Spearman 0.4931 → 0.5225), well above the ±18 bp Fisher-z noise floor and statistically significant (non-overlapping CIs)

Calibration (Out-of-Time Sample):

• Bias: +0.025 (mean pred − realized) — conservative over-prediction, retained as a Category-B Margin of Conservatism
• Halved from +0.048 by regularizing the severity stage; the residual is an out-of-time recovery-regime shift, not an in-sample error

Realized LGD (OOT Recovery Period, vintage > 2009):

• Mean LGD: 19.65% (no discount)
• Validates two-stage architecture: ~10–15% of defaulted loans achieve full recovery (cure classifier)

Downturn LGD (HPI-Anchored Stress Window, 2007–2011):

• EBA Reference Value: 34.18%
• Margin of Conservatism: 2.29% (Category-B parameter-uncertainty buffer for the post-2009 recovery-regime shift; CRR Art. 179(1)(f), EBA/GL/2017/16 §4.4)
• Final Adjusted LGD: 36.48%

Treatment: Fully-drawn, non-revolving, amortizing mortgages require no CCF model. EAD ≡ UPB (Unpaid Principal Balance) at default per CRR Article 166.

Empirical Validation: 88.57% of 536,169 liquidated loans fall within ±2% of unity ratio (last_upb / upb_at_default). Median ratio exactly 1.0. Full analysis in reports/ead_treatment.md.

• Python 3.11+ (tested; project uses 3.14 in dev environment)
• Install dependencies: pip install -r requirements-dev.txt (full pipeline). requirements.txt holds the dashboard-only runtime deps used by the deployed app.

1. Acquire Data (manual step, files not in repo)

   # Fannie Mae data
   # Download from https://www.fanniemae.com/research-and-analysis/datasets
   # Place raw file at: data/raw/fannie_sf_loan_perf.txt

2. Create Data Directories

   mkdir -p data/raw data/processed
   touch data/.gitkeep

Six notebooks walk through the full methodology, end-to-end:

1. 00_primer.ipynb — Data overview, default cohort construction, cash-flow timing
2. 01_data_and_lgd_targets.ipynb — Realized LGD calculation, 0% vs. 5% discount rates
3. 02_lgd_two_stage_and_downturn.ipynb — Cure classifier, severity regressor, HPI-anchored downturn window, MoC
4. 03_ead_identity.ipynb — EAD ≡ UPB validation, ratio analysis, no CCF requirement
5. 04_backtest_explain_sql.ipynb — Out-of-time backtest, Spearman/bias/MAE metrics, SHAP feature importance, SQL reconciliation
6. 05_phase2_experiments.ipynb — Feature engineering experiments (6 blocks tested), champion selection, isotonic recalibration

jupyter notebook
# Open each notebook in sequence; all load frozen samples of the real Fannie Mae portfolio

pytest tests/ -v
# Expected: 123 passed
# Validates: feature timing (no post-default leakage), LGD calculation, downturn logic

Live: https://lgd-irb.streamlit.app

A Streamlit app renders the frozen validation evidence across the six-part IRB spine (overview, discrimination, calibration, downturn/MoC, explainability, governance). It reads only committed artifacts — no raw data, no model fit at runtime.

• Regenerate artifacts: python scripts/build_app_artifacts.py (idempotent, seeded).
• Run locally: streamlit run app/cockpit.py
• Deploy: Streamlit Community Cloud pointed at app/cockpit.py; the KTH theme is picked up from .streamlit/config.toml. Only app/artifacts/ (~40 KB of aggregates) is shipped — the multi-GB raw data is never touched.

.
├── lgd_engine/
│   ├── __init__.py
│   ├── data_loader.py          # Fannie Mae pipe-delimited load, parquet cache
│   ├── lgd_targets.py          # Realized LGD (cash-flow, discount-rate)
│   ├── features.py             # Feature engineering (no post-default leakage)
│   ├── lgd_model.py            # Two-stage cure + severity, TwoStageLGD class
│   ├── downturn.py             # HPI-anchored stress window, MoC, reference value
│   ├── ead_model.py            # EAD validation (legacy CCF code retained)
│   ├── backtest.py             # OOT split, Spearman/bias/MAE/RMSE metrics
│   ├── explain.py              # SHAP feature importance
│   └── utils.py                # Constants (SEED=42), helpers
│
├── app/
│   └── cockpit.py              # Streamlit validation dashboard (6-tab IRB spine)
│
├── notebooks/
│   ├── 00_primer.ipynb                    # Data overview & default cohort
│   ├── 01_data_and_lgd_targets.ipynb      # Realized LGD demo
│   ├── 02_lgd_two_stage_and_downturn.ipynb # Two-stage + downturn + MoC
│   ├── 03_ead_identity.ipynb              # EAD ≡ UPB validation
│   ├── 04_backtest_explain_sql.ipynb      # Backtest + SHAP + SQL
│   └── 05_phase2_experiments.ipynb        # Feature engineering trials
│
├── tests/
│   ├── test_data_loader.py
│   ├── test_lgd_targets.py
│   ├── test_features.py           # Enforces strict feature timing
│   ├── test_lgd_model.py
│   ├── test_downturn.py
│   ├── test_backtest.py
│   ├── test_ead_model.py
│   └── test_explain.py
│
├── reports/
│   ├── experiments.md             # Experiment trial log (all 6 feature blocks)
│   ├── ead_treatment.md           # EAD ≡ UPB empirical validation
│   └── p6_lgd_ead_report.tex      # 8–10pp IRB memo, article mapping, limitations
│
├── data/
│   ├── raw/                       # Gitignored; user must supply
│   │   └── fannie_sf_loan_perf.txt
│   ├── processed/                 # Gitignored; auto-generated on first load
│   │   └── fannie_sf_loan_perf.parquet
│   └── .gitkeep
│
├── requirements.txt               # Dashboard runtime deps (Streamlit Cloud)
├── requirements-dev.txt           # Full pinned modelling pipeline deps
├── README.md                      # This file
└── .gitignore                     # Excludes data/, notebooks checkpoints, pycache

Every model component maps to a specific CRR or EBA article:

Full mapping table with implementation notes appears in reports/p6_lgd_ead_report.tex (Section 3).

• This project models LGD and EAD for mortgage portfolios only; no combined PD × LGD × EAD loss number is calculated
• Expected Loss formula shown only symbolically as Bluhm Eq. 1.3 (EL = PD × LGD × EAD), with PD as a placeholder

• Cure threshold: LGD ≤ 1% (negligible loss)
• Real cure rates vary by product; threshold calibrated to the observed Fannie portfolio
• Production models should validate empirically against institutional workout data

• Models assume PD and LGD are independent; real portfolios may show negative correlation
• IRB Foundation Approach permits independence assumption; not required to model correlation explicitly
• Impact: IRB capital estimates may understate risk when PD spikes and LGD correlates upward during severe downturns

• All LGD calculations discount cash flows at 5% per annum
• Real portfolios use market-implied rates or funding costs; 0–6% range common across banks
• Sensitivity provided in NB01; 1% discount change shifts LGD by ~2–3 pp

• Fannie Mae data: US residential mortgages only; no European/Nordic property market dynamics or regulatory frameworks
• Basel context: Models are aligned with CRR/EBA (European), not US OCC/Federal Reserve guidance
• Portability: Code structure is generic; parameters (discount rate, MoC weights) are user-configurable

Why a US dataset under an EU rulebook? This pairing is deliberate. Loan-level mortgage data carrying the fields an LGD/EAD model requires — drawn balance, realised recoveries, foreclosure timing, mortgage insurance, and servicer identity — is not publicly available for European portfolios, where such records stay bank-internal or access-restricted. The Fannie Mae single-family dataset is one of the few open sources at this scale that carries these fields, which keeps the entire analysis reproducible from public data. The methodology is jurisdiction-agnostic: the two-stage LGD, downturn calibration, and Margin of Conservatism all follow the Basel III framework that the EU (CRR/EBA) and the US implement in parallel. CRR and the EBA Guidelines serve as the reference rulebook because they publish the most prescriptive and openly citable requirements for LGD and EAD estimation (for example, EBA/GL/2017/16 on downturn LGD and EBA/RTS/2018/04 on estimation methods), which gives concrete targets to build and test against. Because the calibration is parameterised, applying the framework to a European portfolio is a change of input data and parameters rather than a redesign.

• This project: 12-year severity lookback (2000–2012 Fannie Mae subset)
• EBA/RTS/2018/04 Art. 3: Requires 20-year minimum
• Gap: 8-year shortfall reflects US data availability; estimates may understate tail-risk LGD in rare multi-decennial stress cycles
• Recommendation: Extend to 20+ years where available; conduct sensitivity on lookback length

• All features (oltv, fico, orig_rate) are origination attributes, not workout-period observables
• Strict enforcement via unit test test_features.py::test_no_post_default_leakage
• Real models may use post-default observables (e.g., appraisal at sale start); this framework constrains to pre-default only

• Downturn window and equity-based features use single national house-price index
• Regional booms and busts averaged away; localized stress (state- or MSA-level) understated
• Production use should substitute regional HPI panel

An experiment harness (Hydra + MLflow) tested six feature blocks on a frozen out-of-time split (vintage > 2009, n = 697,364). Only delinquency trajectory features (months_dlq_at_default, max_dlq_12m, dlq_episodes_12m) cleared the ±18 bp noise floor with Spearman 0.5264 (untuned) vs. 0.4931 baseline. Mortgage-insurance, servicer, and engineered-ratio blocks were flat or negative and not adopted.

Full MLflow trial log (every run, successes and failures):

Metrics are reproduced verbatim from the tracked run log at reports/experiments.md; each row carries its own git_sha and MLflow run_id for audit.

Two levers were tested against the raw block's +0.048 over-prediction bias. Isotonic recalibration (rank-preserving), fit out-of-fold to avoid in-sample optimism, moved bias only 0.048 → 0.046 — because the over-prediction is an out-of-time regime shift (post-2009 recoveries improved relative to the GFC-era training vintages), not an in-sample miscalibration, so a monotone map fit on training data does not transfer to the OOT sample. Hyperparameter tuning, which regularizes the severity stage, instead halves bias 0.048 → 0.025 at a ~38 bp Spearman cost (0.5264 → 0.5225, within the Fisher-z CI band). Because IRB LGD feeds capital directly (RWA = EAD × LGD × 12.5), a level bias flows into own-funds requirements while the marginal rank difference does not — so the tuned delinquency model is champion. The residual +0.025 is conservative (over- not under-statement) and retained as a Category-B Margin of Conservatism (EBA/GL/2017/16 §4.4), governed by a segment-level guardrail: any segment with under-prediction (bias ≤ 0) triggers escalation and rollback. The out-of-fold recalibrator remains in the harness as a corrected calibration primitive for future vintages where the bias is in-sample.

See reports/experiments.md for full trial log (all successes and failures).

• MI-netting effect: Realized LGD computed net of mortgage-insurance proceeds. Insurance recoveries lower observed LGD on insured loans; MI-feature block adds little discrimination because benefit is already priced into target. Portfolios that book MI differently (gross LGD or MI as separate offset) may see different results.
• National-HPI limitation: Single national house-price index averages regional booms/busts. Regional stress understated. Production use should substitute regional HPI panel.

All code tested via pytest:

pytest tests/ -v --cov=lgd_engine --cov-report=term-miss

Test categories:

• Data integrity: Parquet caching, column alignment
• LGD calculation: Discount rate, cost aggregation, signed conventions
• Feature timing: No post-default leakage (enforced)
• Model API: Cure probability, severity prediction, two-stage composition
• Downturn logic: HPI window identification, MoC weights
• Backtest: OOT split, metrics (Spearman, bias, MAE, RMSE)
• EAD validation: Ratio analysis, identity check

Coverage target: ≥80%; achieved 95%+ across core modules.

See requirements.txt for pinned versions. Key libraries:

• numpy, pandas, pyarrow: Numerical and tabular data handling
• scikit-learn: Logistic regression (cure classifier)
• statsmodels: GLM severity model
• lightgbm: Gradient-boosted severity model
• scipy: Statistical distributions and metrics
• shap: Feature importance via Shapley values
• duckdb: In-process SQL for reconciliation
• matplotlib: Visualization (notebooks, app)
• streamlit: Web app framework for scorecard
• jupyter: Interactive notebooks
• pytest: Unit testing framework

• Code style: PEP 8; type hints encouraged
• Git workflow: Branch per feature/fix; tests must pass before merge
• Commit messages: Conventional Commits (feat:, fix:, docs:, etc.)
• Documentation: Docstrings for all public functions; README updated with model changes

Internal use only. Regulatory filing requires institutional sign-off.

For questions on model methodology, regulatory compliance, or data access, see reports/p6_lgd_ead_report.tex (Section 4: Known Limitations & Scope Gaps).

Report date: 2026-07-22
Python version: 3.11 (developed under 3.14)
Framework version: aligned with IRB requirements (CRR Art. 181, EBA/GL/2017/16, EBA/RTS/2018/04)