Skip to content

topics: read particle.retainPath under the particle lock in scanMessages (fixes a data race with RetainMessage) - #539

Open
ChristophNetsch wants to merge 1 commit into
mochi-mqtt:mainfrom
alpamayo-solutions:fix/retained-scan-race-upstream
Open

ChristophNetsch wants to merge 1 commit into
mochi-mqtt:mainfrom
alpamayo-solutions:fix/retained-scan-race-upstream

Conversation

@ChristophNetsch

Copy link
Copy Markdown

What races

TopicsIndex.RetainMessage writes particle.retainPath while holding the root lock and the particle's own lock (topics.go:461, and the clearing write at :471). TopicsIndex.scanMessages — the retained-message scan a SUBSCRIBE triggers through Messages()/publishRetainedToClient — reads adjacent.retainPath and particle.retainPath with no lock at all (topics.go:554, :573). A retained publish that coincides with a wildcard subscribe on an overlapping filter is therefore a data race. Under -race:

WARNING: DATA RACE
Write at 0x... by goroutine N:
  github.com/mochi-mqtt/server/v2.(*TopicsIndex).RetainMessage()  topics.go:461
  github.com/mochi-mqtt/server/v2.(*Server).retainMessage()        server.go:979
  github.com/mochi-mqtt/server/v2.(*Server).processPublish()       server.go:929
  ...
Previous read at 0x... by goroutine M:
  github.com/mochi-mqtt/server/v2.(*TopicsIndex).scanMessages()    topics.go:554
  github.com/mochi-mqtt/server/v2.(*TopicsIndex).Messages()        topics.go:526
  github.com/mochi-mqtt/server/v2.(*Server).publishRetainedToClient()
  github.com/mochi-mqtt/server/v2.(*Server).processSubscribe()
  ...

This is the race #200 describes; it was closed as not reproducible. The test added here reproduces it deterministically: one goroutine retains and clears messages under a/b/+ while the test thread scans a/#, a/b/+ and an exact filter. On the current main it fails under -race with the report above on every run; with the fix it passes.

The fix

Read retainPath under the particle's lock at both sites in scanMessages, copying the string out before use. The scan already takes no root lock, and it takes only one particle lock at a time and briefly, so there is no lock-order interaction with RetainMessage (root → particle) and no new contention beyond the two short critical sections per visited particle. Behaviour is unchanged: the scan sees either the previous or the new retainPath, which was already the intended semantics; it just no longer sees a torn read.

Cost

Two uncontended mutex acquisitions per scanned particle. No allocation, no API change.

Test

TestRetainMessageWhileScanningDoesNotRace in topics_test.go. Run with go test -race -run TestRetainMessageWhileScanningDoesNotRace ./ — fails on main, passes with this change.

…Messages

RetainMessage writes particle.retainPath under the root and particle locks;
scanMessages (a subscriber's retained-message scan, via Messages) read it
with no lock at all. A retained publish racing a wildcard SUBSCRIBE is a
data race the race detector reports as:

  Write at ... by (*TopicsIndex).RetainMessage  topics.go:461
  Previous read at ... by (*TopicsIndex).scanMessages  topics.go:554

This is mochi-mqtt#200, closed as not reproducible; the added test reproduces it
deterministically under -race and passes with the two reads locked.

(cherry picked from commit 8b986f0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant