An LSTM learns how brain activity evolves over time and uses that context to sharpen standard source estimates such as dSPM.
Simulated burst travelling across five sources (red: true peak source). CMNE suppresses background activity that dSPM leaves in. Generated by examples/quickstart_simulation.py.
- How it works
- Installation
- Quick start
- Reproducing the paper
- Running on modest hardware
- API overview
- Changes from the original code
- Citation
Methods such as MNE, dSPM and sLORETA estimate every time sample on its own. Brain activity, however, unfolds over time: what happens now depends on what happened just before. CMNE uses this context in three steps.
-
Estimate. Compute a standard source estimate
$\hat q_t$ (dSPM), rectify it and z-score each source. -
Predict. Give the last
$k$ CMNE estimates$b_{t-k}, \dots, b_{t-1}$ to an LSTM network, which predicts the next estimate. - Correct. Use the prediction, scaled to a maximum of one, as a per-source weight on the current estimate:
For the first
In the paper, CMNE gave higher source-space SNR, lower spatial dispersion and smaller localisation errors than dSPM, both on simulated epileptiform spikes and on recorded auditory steady-state responses (ASSR).
| Hardware | CPU, NVIDIA CUDA or Apple Silicon (MPS), chosen automatically; bfloat16 on CUDA |
| Memory | Training windows are generated on the fly, so memory grows with the sensor data rather than the source space |
| Speed | The inverse operator is folded into a single matrix; many trials run as one batch |
| Deployment | Export to ONNX and run with ONNX Runtime, without PyTorch |
| MNE-Python | Works directly with Epochs, SourceEstimate and inverse operators |
| Fidelity | Implements Eqs. 9–13, the control estimate and the metrics of Eqs. 22–26, tested against literal reference implementations |
pip install cmne # PyTorch + MNE-Python
pip install "cmne[onnx]" # adds ONNX export and ONNX Runtime
pip install "cmne[viz]" # adds matplotlib for the examplesFor GPU training, install the matching PyTorch build first; CMNE picks it up automatically.
Development install
git clone https://github.com/chdinh/cmne && cd cmne
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytestcmne demoThis trains a small model on simulated EEG on the CPU and compares it with dSPM at the peak of the activity:
dSPM peak error 46.2 mm spatial dispersion 58.4 mm
CMNE peak error 0.0 mm spatial dispersion 40.0 mm
import cmne
from mne.minimum_norm import apply_inverse
# epochs: mne.Epochs, inv: mne.minimum_norm.InverseOperator
kernel = cmne.inverse_kernel(epochs.info, inv, lambda2=1 / 9, method="dSPM")
# Train on single-trial dSPM estimates (paper settings: k = 80, d = 1280)
model = cmne.fit(
epochs[train], kernel, look_back=80, num_units=1280, n_steps=250, validation=epochs[test]
)
model.save("cmne.pt")
# Apply to an averaged response
stc = apply_inverse(epochs[test].average(), inv, 1 / 9, "dSPM", pick_ori="normal")
result = cmne.apply_cmne(stc, model)
result.cmne.plot(subject="sample", subjects_dir=subjects_dir)apply_cmne returns three estimates:
| Field | Content |
|---|---|
result.sensing |
rectified, z-scored input estimate |
result.prediction |
raw LSTM prediction |
result.cmne |
contextual estimate |
They are SourceEstimate objects if the input was one, NumPy arrays otherwise. To process many signals at once, pass an array of shape (n_signals, n_sources, n_times).
cmne.export_onnx(model, "cmne.onnx")
predictor = cmne.OnnxPredictor("cmne.onnx") # PyTorch not required
result = cmne.apply_cmne(stc, predictor)cmne train --raw sub01_raw.fif --inv sub01-inv.fif --events sub01-eve.fif -o cmne.pt
cmne export cmne.pt -o cmne.onnx
cmne apply --raw sub01_raw.fif --inv sub01-inv.fif --events sub01-eve.fif \
--model cmne.onnx --n-average 20 -o results/cmne <command> --help lists all options, including epoch window, rejection thresholds, MEG-only, look-back and number of units.
The ASSR data set used in the paper (MEG + EEG, ~4.3 GB) is available on OSF. The script downloads it on first use:
python examples/assr_paper.py --data ~/cmne_data # paper settings
python examples/assr_paper.py --data ~/cmne_data --units 256 # faster, for laptops| Setting | Value (paper and default) |
|---|---|
| Inverse | dSPM, SNR 3, loose orientation 0.2, normal component |
| Normalisation | rectified, z-scored per source (Eq. 9) |
| Look-back |
80, 1280 |
| Training | 250 minibatches of 30 epochs × 20 windows |
| Split | 85 % training, 15 % test (indices shipped with the data) |
| Evaluation | average of 20 held-out epochs |
Most parameters sit in the LSTM input layer, which grows with the number of units
num_units |
Parameters | Size (fp32) | Suitable for |
|---|---|---|---|
| 128 | 3.4 M | 13 MB | any laptop |
| 256 | 6.8 M | 27 MB | laptop CPU, Apple Silicon |
| 640 | 18.0 M | 72 MB | entry-level GPU, M-series Mac |
| 1280 (paper) | 39.4 M | 157 MB | GPU with ≥ 6 GB |
If memory is still tight, lower epochs_per_batch and windows_per_epoch (default 30 × 20 = 600 windows per step).
Inference speed at paper scale on an Apple M-series CPU (python benchmarks/bench_inference.py):
| Backend | 1 signal | 20 signals, batched |
|---|---|---|
| PyTorch | 43 ms per step | 7.6 ms per step and signal |
| ONNX Runtime | 41 ms per step | 18.9 ms per step and signal |
| Function / class | Purpose |
|---|---|
inverse_kernel |
Inverse operator as one (n_sources, n_channels) matrix |
fit → CMNEModel |
Train the LSTM |
CMNEModel.save / load / predict |
Store, load and run a model |
apply_cmne → CMNEResult |
Contextual estimate (Eqs. 10–13) |
control_estimate |
The paper's control estimate without LSTM |
export_onnx, OnnxPredictor |
ONNX export and inference |
peak_localization_error, spatial_dispersion, source_snr |
Metrics (Eqs. 22–26) |
simulate_data, fetch_assr |
Simulated data and the ASSR data set |
Version 0.2 is a rewrite of the 2017–2022 research code, which is kept in archive/. Besides moving from TensorFlow/Keras to PyTorch, it fixes:
- Normalisation mismatch. The network was trained on signed z-scored dSPM but applied to rectified, max-normalised dSPM. Both now use Eq. 9.
-
Weighting. The estimate was multiplied by the raw, possibly negative prediction instead of the normalised weight of Eq. 11. The old behaviour remains available with
normalize_weights=False. - Windowing. One valid training window per epoch was dropped.
-
Crashes.
train()used undefined variables and returned nothing;Dataindexed a method and compared arrays with== None;Settingsfailed if the results folder did not exist. -
Hard-coded sizes. The evaluation assumed 5,124 sources and
$k$ = 80; both now come from the model. -
Robustness. Division by zero for constant sources; the data download relied on shell
mv/rm.
If CMNE is useful for your work, please cite:
Dinh C, Samuelsson JG, Hunold A, Hämäläinen MS, Khan S. Contextual MEG and EEG Source Estimates Using Spatiotemporal LSTM Networks. Frontiers in Neuroscience 15:552666 (2021). doi:10.3389/fnins.2021.552666
@article{dinh2021cmne,
title = {Contextual {MEG} and {EEG} Source Estimates Using Spatiotemporal {LSTM} Networks},
author = {Dinh, Christoph and Samuelsson, John G. and Hunold, Alexander and
H{\"a}m{\"a}l{\"a}inen, Matti S. and Khan, Sheraz},
journal = {Frontiers in Neuroscience},
volume = {15},
pages = {552666},
year = {2021},
doi = {10.3389/fnins.2021.552666}
}An earlier version is available as a preprint: arXiv:1909.02636.
MIT © 2017–2026 the CMNE authors.