add batch messaging support #44
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation / Background
Google Cloud Pub Sub client supports batch messaging, but the previous Watermill Publish implementation was synchronous. By publishing messages serially and waiting for acknowledgement each time. This PR tries to be as backwards compatible as possible
Details
The process is split into three phases:
Preparation: Marshals all messages. Validates messages before network requests, aborting early on failure.
Dispatch (Async): Calls topic.Publish() for each message without waiting. This allows the client to fill internal buffers and trigger batch uploads based on configured settings.
Collection (Sync): Waits for acknowledgements using .Get(ctx), updates metadata with server IDs, and accumulates errors instead of failing fast. Preserves ordering key resumption logic.
Alternative approaches considered (if applicable)
An errgroup could be used instead.