Add SynapticDelay: per-channel axonal/synaptic delay with a learnable delay time - #453
Merged
Merged
Conversation
…elay time Closes the 2021 "Axonal Delays" enhancement request (jeshraghian#60). snnTorch had no delay mechanism: a spike emitted at step t always reached the next layer at step t. `snn.SynapticDelay` is a plain nn.Module dropped between a connection and a neuron (`fc -> SynapticDelay -> Leaky`) that lags each channel by its own delay d_c >= 0. * Fractional delay via interpolation, so d_c is a continuous, trainable quantity. `kernel="linear"` is an exact 2-tap kernel (local gradient, good for fine delays); `kernel="gaussian"` gives gradient across a wide delay range, anneal sigma down during training (Hammouamri et al., 2024). * Two call modes sharing one kernel: step mode `(batch, channels)` with a ring buffer + `reset_delay()` (mirrors `Leaky.reset_mem()`), and sequence mode `(time, batch, channels)` as a causal depthwise conv1d along time to pair with `LeakyParallel`. Both give the same output for the same delay. * `learn_delay` toggles nn.Parameter vs buffer; delay clamped to [0, max_delay] each forward, as `beta` is clamped to [0, 1]. * Additive only - no existing neuron signature changes. 17 unit tests; full suite 227 passed / 2 xfailed (was 210 / 2).
Closed
tritsystem
added a commit
to tritsystem/snntorch
that referenced
this pull request
Sep 20, 2026
Resolve the conflict in snntorch/_neurons/__init__.py against jeshraghian#453 (SynapticDelay): keep both the customneuron and synapticdelay exports. Full suite: 258 passed, 2 xfailed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
SynapticDelay: per-channel axonal / synaptic delay with a learnable delay timeCloses #60.
Motivation
snnTorch has no delay mechanism. A spike emitted at step t always reaches
the next layer at step t. There is no way to give a pathway a
transmission lag, and no way to learn one — even though axonal/synaptic
delays are a standard feature of biological circuits and, since Hammouamri,
Khalfaoui-Hassani & Masquelier (Learning Delays in Spiking Neural Networks
using Dilated Convolutions with Learnable Spacings, ICLR 2024), a
competitive trainable parameter for temporal tasks.
What this adds
One module,
snn.SynapticDelay— a plainnn.Moduleplaced between aconnection and a neuron:
d_c ≥ 0, realised by interpolation sod_cis continuous.kernel="linear"— exact 2-tap kernel, local gradient (±1 step), bestfor fine delays / fine-tuning.
kernel="gaussian"— gradient across awide delay range; anneal
sigmafrom wide to narrow during training tolearn a large delay from scratch (the DCLS recipe).
(batch, channels), called pertime step, keeps a ring buffer and a
reset_delay()that mirrorsLeaky.reset_mem(). Sequence mode(time, batch, channels)runs thewhole sequence at once as a causal depthwise
conv1dalong time — pairswith
LeakyParallel. The two modes give the same output for the samedelay.
learn_delaytogglesnn.Parametervsregister_buffer; the delay isclamp(0, max_delay)-ed every forward,as
betais clamped to[0, 1].SynapticDelaydoes not subclass
SpikingNeuron(it has no membrane / threshold), so itis reset explicitly via
reset_delay()rather than throughutils.reset()— same pattern as callingLeaky.reset_mem()yourself.Evidence
Pre-registered acceptance bars (
gradcheck, RED/GREEN, honest baselines):delay=kappears at t+k, step and sequence mode; the two modes agree to< 1e-6on random per-channel fractional delays.gradcheckpasses w.r.t. the input (float64); analytic∂out/∂delaymatches central finite differences to2.9e-10.gaussian+sigmaanneal → 7.000 (loss ×0.07);linearfrom init 6.0 → 7.000 (loss → 0).learn_delay=Falsecontrol stays at init.lag=9after channel 0's"). Oracle (delay fixed at 9): 1.00. Learnable delay (init 5, annealed): 0.64, beats the delay-disabled ablation (0.54, chance) on 5/5 seeds. The oracle−learned gap is real: joint delay+head optimisation from a poor delay init does not reach the oracle; Bar 3 shows the annealing recipe recovers the exact delay when it is trained in isolation.SynapticDelaycomposes withLeaky(step) andLeakyParallel(sequence), both.backward()cleanly.17 new unit tests in
tests/test_snntorch/test_synapticdelay.py.Scope / follow-ups (deliberately out)
per-channel. A
[out, in]kernel is a natural follow-up.RLeaky/RSynaptic.