Do SAE Features Actually Help Detect Jailbreaks?
A Systematic Benchmark of Interpretability-Based Safety Methods
Website • Paper • Demo • Dataset
We benchmark 8 detection methods across 4 paradigms (raw probes, SAE probes, logit-based, external classifiers) on 6 datasets and 4 models spanning 2B to 70B parameters. We define the Detection Gap — best SAE AUROC minus best non-SAE AUROC — and show it is negative on every model tested. We trace the cause to the reconstruction objective and propose InterpGuard, a two-stage recipe that detects with raw activations and explains with SAE features.
A simple linear probe on raw activations (0.949 AUROC) outperforms every SAE-based method and LlamaGuard-3 (0.885):
| Model | Params | SAE | Best Raw | Best SAE | Detection Gap |
|---|---|---|---|---|---|
| Gemma-2-2B-it | 2B | Gemma Scope JumpReLU 16K | 0.949 | 0.712 | -0.237 |
| Llama-3.1-8B-Instruct | 8B | Llama Scope JumpReLU 32K | 0.867 | 0.477 | -0.391 |
| Gemma-3-4B-it | 4B | Gemma Scope 2 JumpReLU 16K | 0.922 | 0.709 | -0.213 |
| Llama-3.3-70B-Instruct | 70B | Goodfire BatchTopK 65K | 1.000 | 0.949 | -0.051 |
The gap persists across layers, transfer settings, 4x wider SAEs, nonlinear classifiers, and MI-ranked features. Fine-tuning the SAE encoder with a classification-aware objective nearly closes it, confirming the problem lies in the training objective, not the architecture.
A practical two-stage framework:
- Detect with raw activation probes — 0.957 AUROC
- Explain with SAE feature activations — 98% of harmful samples have safety-relevant concepts in their top-10 features
An LLM-as-judge evaluation across three frontier models reveals that current SAE labels identify that a prompt is harmful but not what kind of harm (1.14/5 composite quality), pointing to label granularity as the key bottleneck.
git clone https://github.com/ronyrahmaan/saeguardbench.git
cd saeguardbench
uv sync # or: pip install -e ".[dev]"saeguardbench gap # Detection Gap across all models
saeguardbench leaderboard # Full leaderboard (sortable, filterable)
saeguardbench demo # Launch Gradio demouv run python scripts/extract_activations.py # Cache activations
uv run python scripts/extract_sae_only.py # SAE features
uv run python scripts/train_detectors.py # Train & evaluate
uv run python scripts/generate_figures.py # Publication figures
# Or all at once:
uv run python scripts/run_all.pyAll experiments use Hydra YAML configs in configs/ and save results as JSON with full metadata (config, timestamp, git hash).
| Experiment | Script | Description |
|---|---|---|
| Activation extraction | extract_activations.py |
Cache raw model activations |
| SAE feature extraction | extract_sae_only.py |
Run activations through SAE encoder |
| Detector training | train_detectors.py |
Train 8 methods per model |
| Cross-dataset transfer | cross_dataset_transfer.py |
20 train/test pairs |
| Layer sweep | layer_analysis.py |
Detection Gap across all layers |
| SAE width ablation | topk_ablation.py |
16K vs 65K features |
| Classification-aware SAE | classification_aware_sae.py |
Fine-tune encoder with detection loss |
| InterpGuard | hybrid_experiment.py |
Two-stage detect + explain |
| LLM-as-judge | multi_llm_judge.py |
Evaluate explanations with 3 LLMs |
| Safety subspace PCA | safety_subspace_analysis.py |
Where the lost signal goes |
| Adaptive attacks | run_adaptive_attacks.py |
CFA2, obfuscated activations |
| Over-refusal | measure_overrefusal.py |
FPR on OR-Bench benign prompts |
| Latency | latency_benchmark.py |
Inference time comparison |
Cross-model experiments
# Llama-3.1-8B-Instruct (~16GB RAM)
python scripts/extract_llama.py && python scripts/llama_validation.py
# Gemma-3-4B-it (~24GB RAM, bfloat16)
python scripts/wave5_gemma3.py
# Llama-3.3-70B-Instruct (2x H100)
python scripts/wave5_70b.py| Model | Hardware | Time |
|---|---|---|
| Gemma-2-2B (full suite) | Apple M4 Pro 24GB / 8GB+ GPU | ~4 hours |
| Llama-3.1-8B | 16GB+ RAM | ~2 hours |
| Gemma-3-4B | 24GB+ unified memory | ~3 hours |
| Llama-3.3-70B | 2x H100 SXM (80GB each) | ~2 hours |
| Dataset | Samples | Role |
|---|---|---|
| JailbreakBench | 200 | Primary evaluation (semantically paired) |
| HarmBench | 320 | Attack diversity (ICML 2024) |
| AdvBench | 520 | GCG-optimized prompts |
| SORRY-Bench | 450 | 45 fine-grained harm categories |
| WildJailbreak | 2,000 | In-the-wild attacks |
| WildGuardTest | 1,699 | Independent validation |
| OR-Bench | 1,319 | Over-refusal test (benign only) |
All prompts are available on HuggingFace as Parquet files with the datasets library.
saeguardbench/
├── src/saeguardbench/ # Core library
│ ├── models.py # Activation extraction (TransformerLens, HF, MLX)
│ ├── sae.py # SAE loading (Gemma Scope, Llama Scope, Goodfire)
│ ├── datasets.py # Dataset loading and preprocessing
│ ├── detectors/ # All 8 detection methods
│ ├── evaluation.py # AUROC, F1, FPR, bootstrap CI, Cohen's d
│ ├── attacks.py # Adaptive attacks (CFA2, obfuscated activations)
│ ├── neuronpedia.py # Neuronpedia label retrieval
│ └── visualization.py # Publication-quality figures
├── scripts/ # 40+ experiment scripts
├── tests/ # 115 unit tests
├── configs/ # Hydra YAML configs
├── figures/ # Generated figures (PNG)
├── results/ # Experiment outputs (JSON/CSV)
├── notebooks/ # Analysis notebooks
├── docs/ # GitHub Pages website
├── (paper available on Zenodo)
├── CITATION.cff # Citation metadata
└── CONTRIBUTING.md # Contribution guide
pytest tests/ -v # 115 tests
pytest tests/ --cov=saeguardbench # with coverageIf you find SAEGuardBench useful, please cite:
@article{rahman2026saeguardbench,
title={Do {SAE} Features Actually Help Detect Jailbreaks?
{A} Systematic Benchmark of Interpretability-Based Safety Methods},
author={Rahman, Md A},
journal={arXiv preprint DOI: 10.5281/zenodo.19535387},
year={2026}
}This project is licensed under the MIT License.