Client: olm v1.7.0 → v1.8.0 → v1.8.2, darwin/arm64 (macOS, Apple silicon), run headless under a launchd job that restarts the client on exit.
Server: Pangolin (hosted, current).
Log window: 2026-07-15 → 2026-08-09, ~26 days of continuous operation.
Over that window the client took 28 nil-pointer panics across four distinct crash sites. All four are lifecycle bugs: something dereferences the websocket client, a peer, or a monitor after it has gone away or before it was ever assigned. They are reproducible in the sense that they recur on their own; none needs unusual input.
I run olm under a supervisor, so each panic is "just" a reconnect — but each reconnect also re-installs ~158 host routes, so the crashes are visible as periodic estate-wide route churn. Happy to run a debug build or a patched binary against any of these.
Everything below is copied verbatim from the supervisor log; I have only replaced our hostname/org/client-id with placeholders.
1. websocket.(*Client).Close called on a nil receiver, from StartTunnel's deferred cleanup
Count: 19 (18× v1.7.0, 1× v1.8.2). The oldest of the four and still present.
This one is the easiest to characterise: it always follows a connection that never came up — an HTTP 401 on the websocket, or the server being unreachable — and then a shutdown signal. StartTunnel's deferred cleanup (func6) calls Close() on a client that was never assigned, so the receiver is nil.
v1.8.2:
Received shutdown signal, stopping tunnel
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x28 pc=0x1028e22ac]
goroutine 65 [running]:
github.com/fosrl/olm/websocket.(*Client).Close(0x0)
github.com/fosrl/olm@v1.8.2/websocket/client.go:230 +0x1c
github.com/fosrl/olm/olm.(*Olm).StartTunnel.func6()
github.com/fosrl/olm@v1.8.2/olm/olm.go:676 +0x24
github.com/fosrl/olm/olm.(*Olm).StartTunnel(_, {...})
github.com/fosrl/olm@v1.8.2/olm/olm.go:681 +0xe10
created by github.com/fosrl/cli/cmd/up/client.clientUpMain in goroutine 1
github.com/fosrl/cli/cmd/up/client/client.go:671 +0x36d4
The identical panic in v1.7.0, immediately after Authentication error: 401 WebSocket connection unauthorized:
Authentication error: 401 WebSocket connection unauthorized
Received shutdown signal, stopping tunnel
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x28 pc=0x104da637c]
goroutine 25 [running]:
github.com/fosrl/olm/websocket.(*Client).Close(0x0)
github.com/fosrl/olm@v1.7.0/websocket/client.go:210 +0x1c
github.com/fosrl/olm/olm.(*Olm).StartTunnel.func6()
github.com/fosrl/olm@v1.7.0/olm/olm.go:667 +0x24
Same defect, same two frames, line numbers moved 210→230 and 667→676 between releases. Reproduction should be: point olm at a server that will 401 (or is unreachable), then send SIGINT/SIGTERM before the tunnel establishes.
Fix is presumably a nil check in the deferred func, or assigning the client before the defer is registered.
2. SendMessageInterval's ticker goroutine outlives the client — most frequent
Count: 9 (3× v1.8.0, 6× v1.8.2). This is the one that actually costs us uptime.
The keepalive goroutine spawned at client.go:303 keeps firing after the client it closes over is gone:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x51 pc=0x100e9ebbc]
goroutine 69 [running]:
github.com/fosrl/olm/websocket.(*Client).SendMessageInterval.func1.1(...)
github.com/fosrl/olm@v1.8.2/websocket/client.go:307
github.com/fosrl/olm/websocket.(*Client).SendMessageInterval.func1()
github.com/fosrl/olm@v1.8.2/websocket/client.go:317 +0x5c
created by github.com/fosrl/olm/websocket.(*Client).SendMessageInterval in goroutine 67
github.com/fosrl/olm@v1.8.2/websocket/client.go:303 +0x134
addr=0x51 rather than 0x28 suggests a nil struct pointer being field-accessed at an offset, not a nil receiver. The ticker looks like it needs to select on a done/ctx channel and return, rather than relying on the client still being valid each tick.
3. PeerManager.RemoveAlias panics on a peer-remove message from the read pump
Count: 8 (v1.8.0). Triggered by an inbound wg peer remove while the read pump is running — i.e. by normal server-side churn, not by anything the client did.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x10 pc=0x104716140]
goroutine 4656 [running]:
github.com/fosrl/olm/peers.(*PeerManager).RemoveAlias(0x140003660b0, 0x2, {0x140005900d0, 0xe})
github.com/fosrl/olm@v1.8.0/peers/manager.go:802 +0x390
github.com/fosrl/olm/olm.(*Olm).handleWgPeerRemoveData(0x140002f2708, {{0x14000522000, 0x17}, {0x1049acf60, 0x1400075a060}, 0x36})
github.com/fosrl/olm@v1.8.0/olm/data.go:102 +0x320
github.com/fosrl/olm/websocket.(*Client).readPumpWithDisconnectDetection(0x1400050a380)
github.com/fosrl/olm@v1.8.0/websocket/client.go:865 +0x328
created by github.com/fosrl/olm/websocket.(*Client).establishConnection in goroutine 4639
github.com/fosrl/olm@v1.8.0/websocket/client.go:589 +0x6dc
addr=0x10 — looks like a lookup that can miss (removing an alias for a peer already removed) whose result is used without a presence check. Note this one panics inside the read pump, so it takes the whole client down on a message the server is entitled to send.
4. PeerMonitor.sendRelay during performRapidInitialTest, straight off AddPeer
Count: 1 (v1.8.2). Rarest, but it is on the add path, so it is a startup-race rather than a teardown race:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x28 pc=0x102cede3c]
goroutine 193 [running]:
github.com/fosrl/olm/peers/monitor.(*PeerMonitor).sendRelay(0x102db4576?, 0x1?)
github.com/fosrl/olm@v1.8.2/peers/monitor/monitor.go:616 +0x1c
github.com/fosrl/olm/peers/monitor.(*PeerMonitor).RequestRelay(...)
github.com/fosrl/olm@v1.8.2/peers/monitor/monitor.go:637
github.com/fosrl/olm/peers.(*PeerManager).performRapidInitialTest(0x140002ae480, 0x2, {0x140000bfdb0, 0x10}, {0x0, 0x0, 0x0})
github.com/fosrl/olm@v1.8.2/peers/manager.go:935 +0x230
created by github.com/fosrl/olm/peers.(*PeerManager).AddPeer in goroutine 59
github.com/fosrl/olm@v1.8.2/peers/manager.go:221 +0x930
sendRelay(0x102db4576?, 0x1?) with a nil-ish receiver suggests performRapidInitialTest starts before the monitor is fully constructed.
Common thread
All four are the same shape: a goroutine or deferred call reaching an object whose lifetime it does not own. (1) and (4) are "not constructed yet", (2) and (3) are "already gone". If there is a single ownership/shutdown model change that would cover them, that seems more valuable than four separate nil checks — though (1) in particular has survived unchanged from v1.7.0 to v1.8.2 and would be a one-line stop-gap.
What would help me help you
Happy to do any of:
- run
--log-level debug and attach the full log around a panic;
- run a patched build against our environment, which reproduces (2) and (3) within a day or so;
- open a PR if you tell me which direction you want (nil checks vs. a done-channel on the ticker vs. a broader ownership change).
Client: olm v1.7.0 → v1.8.0 → v1.8.2, darwin/arm64 (macOS, Apple silicon), run headless under a launchd job that restarts the client on exit.
Server: Pangolin (hosted, current).
Log window: 2026-07-15 → 2026-08-09, ~26 days of continuous operation.
Over that window the client took 28 nil-pointer panics across four distinct crash sites. All four are lifecycle bugs: something dereferences the websocket client, a peer, or a monitor after it has gone away or before it was ever assigned. They are reproducible in the sense that they recur on their own; none needs unusual input.
I run olm under a supervisor, so each panic is "just" a reconnect — but each reconnect also re-installs ~158 host routes, so the crashes are visible as periodic estate-wide route churn. Happy to run a debug build or a patched binary against any of these.
Everything below is copied verbatim from the supervisor log; I have only replaced our hostname/org/client-id with placeholders.
1.
websocket.(*Client).Closecalled on a nil receiver, fromStartTunnel's deferred cleanupCount: 19 (18× v1.7.0, 1× v1.8.2). The oldest of the four and still present.
This one is the easiest to characterise: it always follows a connection that never came up — an HTTP 401 on the websocket, or the server being unreachable — and then a shutdown signal.
StartTunnel's deferred cleanup (func6) callsClose()on a client that was never assigned, so the receiver isnil.v1.8.2:
The identical panic in v1.7.0, immediately after
Authentication error: 401 WebSocket connection unauthorized:Same defect, same two frames, line numbers moved 210→230 and 667→676 between releases. Reproduction should be: point olm at a server that will 401 (or is unreachable), then send SIGINT/SIGTERM before the tunnel establishes.
Fix is presumably a nil check in the deferred func, or assigning the client before the defer is registered.
2.
SendMessageInterval's ticker goroutine outlives the client — most frequentCount: 9 (3× v1.8.0, 6× v1.8.2). This is the one that actually costs us uptime.
The keepalive goroutine spawned at
client.go:303keeps firing after the client it closes over is gone:addr=0x51rather than0x28suggests a nil struct pointer being field-accessed at an offset, not a nil receiver. The ticker looks like it needs to select on a done/ctx channel and return, rather than relying on the client still being valid each tick.3.
PeerManager.RemoveAliaspanics on a peer-remove message from the read pumpCount: 8 (v1.8.0). Triggered by an inbound
wg peer removewhile the read pump is running — i.e. by normal server-side churn, not by anything the client did.addr=0x10— looks like a lookup that can miss (removing an alias for a peer already removed) whose result is used without a presence check. Note this one panics inside the read pump, so it takes the whole client down on a message the server is entitled to send.4.
PeerMonitor.sendRelayduringperformRapidInitialTest, straight offAddPeerCount: 1 (v1.8.2). Rarest, but it is on the add path, so it is a startup-race rather than a teardown race:
sendRelay(0x102db4576?, 0x1?)with a nil-ish receiver suggestsperformRapidInitialTeststarts before the monitor is fully constructed.Common thread
All four are the same shape: a goroutine or deferred call reaching an object whose lifetime it does not own. (1) and (4) are "not constructed yet", (2) and (3) are "already gone". If there is a single ownership/shutdown model change that would cover them, that seems more valuable than four separate nil checks — though (1) in particular has survived unchanged from v1.7.0 to v1.8.2 and would be a one-line stop-gap.
What would help me help you
Happy to do any of:
--log-level debugand attach the full log around a panic;