On QUEUE_CONNECTION=database, a publish that runs longer than 90 seconds is handed to a second worker while the first is still running, and the same post goes out to the network twice.
The mismatch
app/Jobs/PublishToSocialPlatform.php declares its contract explicitly:
public int $tries = 20;
/** Download/upload + Pinterest poll headroom; keep Horizon/Redis timeouts above this. */
public int $timeout = 900;
public int $uniqueFor = 960;
config/queue.php honours that on Redis and not on the database driver:
| Connection |
retry_after |
vs. the job's 900s timeout |
redis |
REDIS_QUEUE_RETRY_AFTER = 960 |
correct — 60s of headroom, matching uniqueFor |
database |
DB_QUEUE_RETRY_AFTER = 90 |
810 seconds short |
retry_after is how long the driver waits before assuming a reserved job died and releasing it back to the queue. When it is shorter than the job's own timeout, the driver releases a job that is still running.
Worth noting config/queue.php:18 defaults to 'database', so this is not only
reached by deliberately choosing that driver — an install with QUEUE_CONNECTION
unset lands there too.
Reproduction
MySQL 8.0.46, QUEUE_CONNECTION=database, two queue:work processes, and a job mirroring PublishToSocialPlatform's profile ($tries = 20, $timeout = 900) that runs for 120 seconds — a plausible video upload:
retry_after = 90 (shipped default)
START pid=1270804 at=03:14:21
START pid=1270805 at=03:15:51 <- +90s, exactly retry_after
END pid=1270804 at=03:16:21
END pid=1270805 at=03:17:11
-> 2 executions, overlapping by 30s
retry_after = 960
START pid=1300870 at=03:17:12
END pid=1300870 at=03:19:12
-> 1 execution
Two notes on why the existing safeguards don't cover this:
ShouldBeUnique holds its lock at dispatch. retry_after expiry pops a row that is already queued rather than dispatching a new one, so the unique lock is never consulted.- A worker's
--timeout flag cannot lower the ceiling either: Worker::timeoutForJob() is $job->timeout() ?? $options->timeout, so the job's own 900 wins.
The duplicate needs more than one consumer on the queue to occur — with a single worker on the social-* queues there is no second worker to hand the job to.
Scope
Hosted TryPost is unaffected: .env.example ships QUEUE_CONNECTION=redis, where 960 already satisfies the invariant. This reaches self-hosted installs on the database driver, and the failure mode is a post published twice to a live network — public and not undoable.
Suggested fix
Raise the database connection's default to match redis, since the same jobs run on both:
'retry_after' => (int) env('DB_QUEUE_RETRY_AFTER', 960),The invariant worth stating somewhere durable is retry_after > the largest job $timeout on that connection. Happy to open a PR for the config change, a .env.example note, or both — whichever you prefer.
One note on timing: you mentioned on #307 that the self-hosting docs need updating now that both engines are supported. If those docs are going to describe running on MySQL, DB_QUEUE_RETRY_AFTER=960 probably belongs next to QUEUE_CONNECTION=database
in them — the two lines are only correct read together, and a self-hoster following the docs today would get the broken pairing.
On
QUEUE_CONNECTION=database, a publish that runs longer than 90 seconds is handed to a second worker while the first is still running, and the same post goes out to the network twice.The mismatch
app/Jobs/PublishToSocialPlatform.phpdeclares its contract explicitly:config/queue.phphonours that on Redis and not on the database driver:retry_afterredisREDIS_QUEUE_RETRY_AFTER= 960uniqueFordatabaseDB_QUEUE_RETRY_AFTER= 90retry_afteris how long the driver waits before assuming a reserved job died and releasing it back to the queue. When it is shorter than the job's own timeout, the driver releases a job that is still running.Worth noting
config/queue.php:18defaults to'database', so this is not onlyreached by deliberately choosing that driver — an install with
QUEUE_CONNECTIONunset lands there too.
Reproduction
MySQL 8.0.46,
QUEUE_CONNECTION=database, twoqueue:workprocesses, and a job mirroringPublishToSocialPlatform's profile ($tries = 20,$timeout = 900) that runs for 120 seconds — a plausible video upload:Two notes on why the existing safeguards don't cover this:
ShouldBeUniqueholds its lock at dispatch.retry_afterexpiry pops a row that is already queued rather than dispatching a new one, so the unique lock is never consulted.--timeoutflag cannot lower the ceiling either:Worker::timeoutForJob()is$job->timeout() ?? $options->timeout, so the job's own 900 wins.The duplicate needs more than one consumer on the queue to occur — with a single worker on the
social-*queues there is no second worker to hand the job to.Scope
Hosted TryPost is unaffected:
.env.exampleshipsQUEUE_CONNECTION=redis, where 960 already satisfies the invariant. This reaches self-hosted installs on the database driver, and the failure mode is a post published twice to a live network — public and not undoable.Suggested fix
Raise the
databaseconnection's default to matchredis, since the same jobs run on both:The invariant worth stating somewhere durable is
retry_after> the largest job$timeouton that connection. Happy to open a PR for the config change, a.env.examplenote, or both — whichever you prefer.One note on timing: you mentioned on #307 that the self-hosting docs need updating now that both engines are supported. If those docs are going to describe running on MySQL,
DB_QUEUE_RETRY_AFTER=960probably belongs next toQUEUE_CONNECTION=databasein them — the two lines are only correct read together, and a self-hoster following the docs today would get the broken pairing.