feat(webhooks): add dispatch queue publisher (NAN-5339) 1/5 by agusayerza · Pull Request #5955 · NangoHQ/nango

agusayerza · 2026-04-27T13:57:22Z

WebhookDispatchMessage shared type in @nangohq/types
DispatchQueuePublisher in server package: batched SendMessageBatch, parallel batches, one inline retry for failed entries, partial-failure emits metric + log
New WEBHOOK_DISPATCH_* metric types in @nangohq/utils
Adds @aws-sdk/client-sqs depedency to server

No call sites yet, wired up in NAN-5340.

linear · 2026-04-27T13:57:25Z

NAN-5339 Build dispatch-queue publisher abstraction

cubic-dev-ai

No issues found across 9 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 74bcc1255f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

agusayerza · 2026-04-27T14:53:53Z

+            const failedIds = (response.Failed ?? []).map((f) => f.Id).filter((id): id is string => typeof id === 'string');
+            return { failedIds };
+        } catch (err) {
+            logger.error(`SendMessageBatchCommand threw`, { error: err });


Intetional log here as we don't have per-batch tracing

TBonnin · 2026-04-27T17:53:14Z

+    return {
+        Id: indexToEntryId(index),
+        MessageBody: JSON.stringify(message),
+        MessageGroupId: messageGroupId


no MessageDeduplicationId

We are using a standard SQS queue with fair-queue behavior via MessageGroupId, not a FIFO queue. Unfortunately the caveat is that fair-queues do not support MessageDeduplicationId. We will be using the task name to dedupe on the orchestrator. I'm open to using a FIFO queue with MessageDeduplicationId but we lose the fairness across tenants. Even then orchestrator would still need deduping transport retries. (FIFO guarantees at-least-once delivery)

TBonnin · 2026-04-27T17:54:21Z

+    /**
+     * Publish a list of dispatch messages in batches. Batches are fired in parallel up to
+     * `publishConcurrency`; failed entries within a batch are retried once inline. Any
+     * entries still failing after the retry are counted as `failed`. Never throws — the


Never throws is contradicted by L112

Yeah, fixing it. I meant never throws on regular SQS operations.

TBonnin · 2026-04-27T17:55:48Z

+
+                if (enqueued > 0) {
+                    metrics.increment(metrics.Types.WEBHOOK_DISPATCH_MESSAGES_ENQUEUED, enqueued, { provider });
+                    metrics.increment(metrics.Types.WEBHOOK_DISPATCH_PUBLISH_SUCCESS, enqueued, { provider });


Do we need both? We are counting the same things twice

Agreed, enqueued ended up being redundant here and also reads more like queue depth, so I removed it and kept publish.success / publish.failure

TBonnin · 2026-04-27T18:02:05Z

+        tracerMocks.dogstatsd.decrement.mockClear();
+        tracerMocks.dogstatsd.gauge.mockClear();
+        tracerMocks.dogstatsd.histogram.mockClear();
+        tracerMocks.dogstatsd.distribution.mockClear();


isn't restoreAllMocks already cleaning up the spies?

Most of these are hoisted vi.fn() mocks, not vi.spyOns, and restoreAllMocks() does not clear or reset the history of vi.fn()

marcindobry · 2026-04-29T11:08:02Z

+        }
+        this.batchSize = Math.min(configuredBatchSize, SQS_BATCH_MAX_ENTRIES);
+
+        const configuredPublishConcurrency = props.publishConcurrency ?? SQS_BATCH_MAX_ENTRIES;


Why are we using a constant for batching as a default for concurrency?

Missed this, originally I had set it to the same value and forgot to split into new const

TBonnin · 2026-04-29T12:58:59Z

@@ -67,6 +67,16 @@ export enum Types {
    WEBHOOK_ASYNC_ACTION_FAILED = 'nango.webhook.async_action.failed',
    WEBHOOK_INCOMING_PAYLOAD_SIZE_BYTES = 'nango.webhook.incoming.payloadSizeBytes',

+    WEBHOOK_DISPATCH_PUBLISH_SUCCESS = 'nango.webhook.dispatch_queue.publish.success',
+    WEBHOOK_DISPATCH_PUBLISH_FAILURE = 'nango.webhook.dispatch_queue.publish.failure',


I imagine the following metrics are declared for future use?

Introduces a DispatchQueuePublisher abstraction for publishing webhook execution messages to SQS. Uses activity log ID as the message identifier. Adds tracing and a shared webhook concurrency helper.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ec71242da9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-04-30T14:49:50Z

+    return {
+        Id: indexToEntryId(index),
+        MessageBody: JSON.stringify(message),
+        MessageGroupId: messageGroupId


Validate FIFO requirements before setting SQS group fields

toEntry always sends MessageGroupId but never sets MessageDeduplicationId, which only works for a FIFO queue that already has content-based dedup enabled. In environments where the queue is standard, or FIFO without content-based deduplication, SendMessageBatch will reject every entry and this publisher will return all messages as failed, causing webhook dispatches to be dropped instead of queued. Please either enforce/validate the queue mode at construction time or include a deterministic deduplication ID (for example from taskName) when publishing.

Useful? React with 👍 / 👎.

- WebhookDispatchMessage shared type in `@nangohq/types` - DispatchQueuePublisher in server package: batched SendMessageBatch, parallel batches, one inline retry for failed entries, partial-failure emits metric + log - New WEBHOOK_DISPATCH_* metric types in `@nangohq/utils` - Adds `@aws-sdk/client-sqs` depedency to server No call sites yet, wired up in NAN-5340.

cubic-dev-ai Bot reviewed Apr 27, 2026

View reviewed changes

agusayerza requested a review from a team April 27, 2026 14:40

agusayerza changed the title ~~feat(webhooks): add dispatch queue publisher abstraction (NAN-5339)~~ feat(webhooks): add dispatch queue publisher (NAN-5339) Apr 27, 2026

agusayerza marked this pull request as ready for review April 27, 2026 14:46

agusayerza commented Apr 27, 2026

View reviewed changes

Comment thread packages/server/lib/webhook/dispatch-queue/publisher.ts Outdated

chatgpt-codex-connector Bot reviewed Apr 27, 2026

View reviewed changes

Comment thread packages/types/lib/webhooks/dispatch.ts

agusayerza commented Apr 27, 2026

View reviewed changes

agusayerza force-pushed the agus/NAN-5339/dispatch-queue-publisher branch from 74bcc12 to 269273c Compare April 27, 2026 16:16

TBonnin reviewed Apr 27, 2026

View reviewed changes

agusayerza requested review from a team and TBonnin April 27, 2026 20:02

agusayerza force-pushed the agus/NAN-5339/dispatch-queue-publisher branch 3 times, most recently from 907fa9e to 5fb9cd8 Compare April 28, 2026 02:22

agusayerza changed the base branch from agus/NAN-5338/ingress-request-id to master April 28, 2026 12:32

marcindobry approved these changes Apr 29, 2026

View reviewed changes

TBonnin reviewed Apr 29, 2026

View reviewed changes

TBonnin approved these changes Apr 29, 2026

View reviewed changes

agusayerza changed the title ~~feat(webhooks): add dispatch queue publisher (NAN-5339)~~ feat(webhooks): add dispatch queue publisher (NAN-5339) 1/4 Apr 29, 2026

agusayerza force-pushed the agus/NAN-5339/dispatch-queue-publisher branch from 2e1ff55 to b4f5512 Compare April 29, 2026 18:28

agusayerza changed the title ~~feat(webhooks): add dispatch queue publisher (NAN-5339) 1/4~~ feat(webhooks): add dispatch queue publisher (NAN-5339) 1/5 Apr 29, 2026

feat(webhooks): add dispatch queue publisher abstraction (NAN-5339)

ec71242

Introduces a DispatchQueuePublisher abstraction for publishing webhook execution messages to SQS. Uses activity log ID as the message identifier. Adds tracing and a shared webhook concurrency helper.

agusayerza force-pushed the agus/NAN-5339/dispatch-queue-publisher branch from b4f5512 to ec71242 Compare April 30, 2026 14:45

chatgpt-codex-connector Bot reviewed Apr 30, 2026

View reviewed changes

agusayerza added this pull request to the merge queue Apr 30, 2026

Merged via the queue into master with commit 5e2da9b Apr 30, 2026
23 checks passed

agusayerza deleted the agus/NAN-5339/dispatch-queue-publisher branch April 30, 2026 16:00

Conversation

agusayerza commented Apr 27, 2026

Uh oh!

linear Bot commented Apr 27, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants