Skip to content

Add SynapticDelay: per-channel axonal/synaptic delay with a learnable delay time - #453

Merged
ixfd64 merged 3 commits into
jeshraghian:masterfrom
tritsystem:feat/synaptic-delay
Sep 20, 2026
Merged

ixfd64 merged 3 commits into
jeshraghian:masterfrom
tritsystem:feat/synaptic-delay

Conversation

@tritsystem

Copy link
Copy Markdown
Contributor

Add SynapticDelay: per-channel axonal / synaptic delay with a learnable delay time

Closes #60.

Motivation

snnTorch has no delay mechanism. A spike emitted at step t always reaches
the next layer at step t. There is no way to give a pathway a
transmission lag, and no way to learn one — even though axonal/synaptic
delays are a standard feature of biological circuits and, since Hammouamri,
Khalfaoui-Hassani & Masquelier (Learning Delays in Spiking Neural Networks
using Dilated Convolutions with Learnable Spacings
, ICLR 2024), a
competitive trainable parameter for temporal tasks.

What this adds

One module, snn.SynapticDelay — a plain nn.Module placed between a
connection and a neuron:

self.fc1   = nn.Linear(num_in, num_hidden)
self.delay = snn.SynapticDelay(max_delay=16, delay=1.0,
                               channels=num_hidden, learn_delay=True)
self.lif1  = snn.Leaky(beta=0.9)
...
cur1 = self.delay(self.fc1(x))          # step mode: (batch, N)
spk1, mem1 = self.lif1(cur1, mem1)
  • Fractional, trainable delay. Each channel is delayed by its own
    d_c ≥ 0, realised by interpolation so d_c is continuous.
    kernel="linear" — exact 2-tap kernel, local gradient (±1 step), best
    for fine delays / fine-tuning. kernel="gaussian" — gradient across a
    wide delay range; anneal sigma from wide to narrow during training to
    learn a large delay from scratch (the DCLS recipe).
  • Two call modes, one kernel. Step mode (batch, channels), called per
    time step, keeps a ring buffer and a reset_delay() that mirrors
    Leaky.reset_mem(). Sequence mode (time, batch, channels) runs the
    whole sequence at once as a causal depthwise conv1d along time — pairs
    with LeakyParallel. The two modes give the same output for the same
    delay.
  • snnTorch conventions. learn_delay toggles nn.Parameter vs
    register_buffer; the delay is clamp(0, max_delay)-ed every forward,
    as beta is clamped to [0, 1].
  • Additive only. No existing neuron signature changes. SynapticDelay
    does not subclass SpikingNeuron (it has no membrane / threshold), so it
    is reset explicitly via reset_delay() rather than through
    utils.reset() — same pattern as calling Leaky.reset_mem() yourself.

Evidence

Pre-registered acceptance bars (gradcheck, RED/GREEN, honest baselines):

Bar Result
1 — static integer delay is exact impulse at t with delay=k appears at t+k, step and sequence mode; the two modes agree to < 1e-6 on random per-channel fractional delays.
2 — the delay parameter is differentiable gradcheck passes w.r.t. the input (float64); analytic ∂out/∂delay matches central finite differences to 2.9e-10.
3 — it learns the delay target delay 7.0, wrong init 1.0: gaussian + sigma anneal → 7.000 (loss ×0.07); linear from init 6.0 → 7.000 (loss → 0). learn_delay=False control stays at init.
4 — it helps on a temporal task single-spike alignment task (label = "is channel 1's spike exactly lag=9 after channel 0's"). Oracle (delay fixed at 9): 1.00. Learnable delay (init 5, annealed): 0.64, beats the delay-disabled ablation (0.54, chance) on 5/5 seeds. The oracle−learned gap is real: joint delay+head optimisation from a poor delay init does not reach the oracle; Bar 3 shows the annealing recipe recovers the exact delay when it is trained in isolation.
5 — zero regression + composition full test suite 227 passed / 2 xfailed (was 210 / 2); SynapticDelay composes with Leaky (step) and LeakyParallel (sequence), both .backward() cleanly.

17 new unit tests in tests/test_snntorch/test_synapticdelay.py.

Scope / follow-ups (deliberately out)

  • Per-synapse (weight-matrix-shaped) delays, the full DCLS form — this is
    per-channel. A [out, in] kernel is a natural follow-up.
  • Delay in the recurrent path of RLeaky / RSynaptic.
  • NIR export of the delay parameter.

…elay time

Closes the 2021 "Axonal Delays" enhancement request (jeshraghian#60). snnTorch had no
delay mechanism: a spike emitted at step t always reached the next layer at
step t. `snn.SynapticDelay` is a plain nn.Module dropped between a connection
and a neuron (`fc -> SynapticDelay -> Leaky`) that lags each channel by its
own delay d_c >= 0.

* Fractional delay via interpolation, so d_c is a continuous, trainable
  quantity. `kernel="linear"` is an exact 2-tap kernel (local gradient,
  good for fine delays); `kernel="gaussian"` gives gradient across a wide
  delay range, anneal sigma down during training (Hammouamri et al., 2024).
* Two call modes sharing one kernel: step mode `(batch, channels)` with a
  ring buffer + `reset_delay()` (mirrors `Leaky.reset_mem()`), and sequence
  mode `(time, batch, channels)` as a causal depthwise conv1d along time to
  pair with `LeakyParallel`. Both give the same output for the same delay.
* `learn_delay` toggles nn.Parameter vs buffer; delay clamped to
  [0, max_delay] each forward, as `beta` is clamped to [0, 1].
* Additive only - no existing neuron signature changes. 17 unit tests;
  full suite 227 passed / 2 xfailed (was 210 / 2).
@ixfd64
ixfd64 merged commit 8500fd8 into jeshraghian:master Sep 20, 2026
3 checks passed
@ixfd64 ixfd64 mentioned this pull request Sep 20, 2026
@ixfd64 ixfd64 added the enhancement New feature or request label Sep 20, 2026
tritsystem added a commit to tritsystem/snntorch that referenced this pull request Sep 20, 2026
Resolve the conflict in snntorch/_neurons/__init__.py against jeshraghian#453
(SynapticDelay): keep both the customneuron and synapticdelay exports.
Full suite: 258 passed, 2 xfailed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Axonal Delays

2 participants