A Go library that publishes background jobs to GCP Pub/Sub. It provides a pluggable Driver interface so that different job-sourcing strategies (Postgres polling, WAL, message queues, etc.) can be implemented independently.

Each job carries a queue name; the courier publishes it to that queue's configured topic. The payload is published verbatim as the message body — its shape is the producer↔consumer contract, not the courier's concern. The message carries shadow (decoded from the job's control header, so workers can ack shadow jobs without executing them), plus job_id, trace_id and producer_id attributes so consumers can dedup, trace and attribute jobs without decoding the payload. Attributes with empty values are omitted.

┌──────────────────────────────────────────────────────────┐
│  courier (this library)                                  │
│                                                          │
│   ┌────────┐    submit([]Job)    ┌───────────────────┐   │
│   │ Driver │ ──────────────────► │ per-queue Pub/Sub │   │
│   │  .Run()│ ◄────────────────── │ publishers        │   │
│   └────────┘   []SubmitResult    └───────────────────┘   │
│        ▲                                  │              │
│        │ ctx cancel                       │              │
│        │ (SIGINT/SIGTERM)                 ▼              │
│   ┌──────────────┐              ┌──────────────────┐     │
│   │ Health HTTP  │              │  Pub/Sub topics  │     │
│   │ GET /health  │              │  (one per queue) │     │
│   └──────────────┘              └──────────────────┘     │
└──────────────────────────────────────────────────────────┘

Each queue gets its own Pub/Sub client so one queue's backpressure or connection trouble cannot affect another's.

A Driver is the only thing you need to implement to create a new courier service. The interface is intentionally minimal:

Drivers are registered globally before calling Main:

func main() {
    courier.RegisterDriver(myDriver)
    courier.Main()
}

Required:

• JACK_COURIER_PUBSUB_PROJECT — GCP project ID of the Pub/Sub topics.
• JACK_COURIER_PUBSUB_TOPICS — comma-separated queue:topic pairs mapping each queue to the topic it publishes to, e.g. high:clerk_jobs_high,low:clerk_jobs_low. A job whose queue has no mapping fails its whole batch before anything is published.

Optional:

• JACK_COURIER_SUBMIT_TIMEOUT — deadline applied to each submit call's publishes, parsed as a Go duration (e.g. 30s, 1m). Defaults to 30s. The driver's lifecycle context still controls overall shutdown; this only bounds an individual submit so a stalled Pub/Sub cannot hang the courier. It also caps the client's own publish attempts, so an abandoned publish stops retrying in the background.
• JACK_COURIER_SHUTDOWN_TIMEOUT — graceful shutdown timeout, parsed as a Go duration. Defaults to 10s. On SIGINT/SIGTERM this bounds total termination time. If the driver does not return within it, the driver is abandoned and the process exits anyway, so the courier never takes longer than the timeout to shut down.
• JACK_COURIER_SINK_NOOP — if true or 1, the courier never publishes to Pub/Sub: submitted jobs are acknowledged as accepted and dropped, and the JACK_COURIER_PUBSUB_* variables are not required. Used to validate drivers end-to-end in production (partitioning, batching, cursor advancement) before the real sink is in place.
• PUBSUB_EMULATOR_HOST — standard Pub/Sub emulator override for local development.

Errors are logged and counted in Datadog metrics, and optionally Sentry.

Submit failures are reported at most once per queue per minute, since drivers retry failing batches on a tight loop; logs and metrics reflect every occurrence.

Shutdown cancellation is counted as jack.courier.submit.count{status:canceled} but never reported, and the final flush shares the shutdown timeout budget.

With WithDatadogTracing(service, environment, version) the courier starts the Datadog APM tracer for the process and emits one courier.submit span per submit call, covering the enqueue and the wait for every publish to resolve. The span's resource is the queue, with jobs.count/jobs.failed tags; batch-level failures mark the span as an error, except shutdown cancellation, which would otherwise put every deploy on the APM error rate. An empty service disables tracing entirely, so the option can be passed unconditionally. The courier owns the process-global tracer; drivers that create their own spans (e.g. the polling driver's database queries) report through it. Agent connectivity comes from the standard DD_* environment variables.

submit returns per-job results. A failure is permanent (Reason set to payload_too_large or validation_error) only when retrying can never succeed; drivers should dead-letter those. All other failures are retryable: the driver retries them on its own schedule. When every job in a batch fails retryably (e.g. a transport outage), submit returns a single batch error instead, so drivers retry the whole batch with backoff.