queue

queue is a consolidated worker queue with root-module implementations for in-memory, Redis Pub/Sub, Redis Streams, Valkey Streams, NATS, and NSQ. Independently released target-oriented adapters provide RabbitMQ and service lifecycle integration. The RabbitMQ adapter delegates AMQP policy to go-rabbitmq-queues.

The package has a stable v1 API. Production code is held to meaningful 100% coverage; durable delivery claims require backend-specific integration evidence.

• Go 1.27.0 or later
• a supported broker for non-memory backends

go get github.com/faustbrian/go-queue

Root backend packages, including adapters/redisstream, ship in the same module and are imported explicitly. Install the independently versioned github.com/faustbrian/go-queue/adapters/rabbitmq or github.com/faustbrian/go-queue/adapters/service module when needed. Former v1 paths remain deprecated compatibility facades.

worker, err := redisdb.NewWorkerE(
    redisdb.WithAddr("127.0.0.1:6379"),
    redisdb.WithChannel("jobs"),
    redisdb.WithRunFunc(func(ctx context.Context, task core.TaskMessage) error {
        return handle(ctx, task.Payload())
    }),
)
if err != nil {
    return err
}

q, err := queue.NewQueue(queue.WithWorker(worker), queue.WithWorkerCount(8))
if err != nil {
    return err
}
q.Start()
defer q.Release()

Redis Pub/Sub is low-latency and non-durable. Use Redis Streams or Valkey Streams when work must remain pending until settlement. They are independent native backends; adopting Valkey does not require removing Redis. Read delivery semantics before selecting a backend. Scheduler and API processes that only submit Valkey work should use valkeystream.NewPublisherE; it appends jobs without joining a consumer group or starting worker loops.

Services should compose concrete producers and workers through adapters/service. The adapter keeps concrete queue APIs visible, closes queue admission during service drain, drains accepted publishers before closing an owned transport, uses the existing correlation queue boundary for every message and delivery attempt, and optionally propagates bounded W3C trace context through an explicit caller-owned OpenTelemetry propagator.

External control planes should depend on the backend-neutral contracts in management. Incompatible workers remain visible, but management capabilities are enabled only when both peers report support. The managementhttp package makes those contracts remotely callable without exposing backend clients. managementhttp.NewFleetClient can resolve a changing set of worker endpoints for multi-replica deployments, aggregate their status, route worker-specific commands, and fan queue or worker-group lifecycle commands to every current replica.

• explicit retry, acknowledgement, redelivery, cancellation, and shutdown behavior
• safe failure classification and codes that preserve errors.Is while redacting arbitrary handler, panic, and settlement text
• handler backoff limited to retryable failures so terminal and uncertain classifications reach backend settlement without repeated side effects
• optional one-time decoded-delivery validation before handler retry execution
• durable Redis Streams, Valkey Streams, and NSQ paths with explicit settlement
• an independently versioned RabbitMQ adapter with confirmed publish-before-ack settlement
• observable lifecycle events, metrics, and backend identity
• stable management-protocol version and capability negotiation for external control planes
• bounded worker and queue status contracts that distinguish unsupported backend measurements from measured zero values, with paginated readers
• bounded authenticated HTTP transport for remote status, records, and control commands
• bounded dynamic management fleets with fail-closed discovery, complete status aggregation, worker routing, and explicit partial fan-out outcomes
• backend-neutral command enforcement contracts with explicit confirmation, bounded bulk retry, acknowledgement, timeout, partial, and unknown outcomes
• revisioned desired-state reconciliation with monotonic per-target application, retry-safe failures, and caller-owned scheduling
• queue-owned pause, resume, drain, and terminate enforcement that stops admission at safe boundaries and reports in-flight work honestly
• bounded failed-job and dead-letter inspection with payloads hidden by default and privileged content capped at one mebibyte
• one root release unit for maintained root backends, with explicit nested release units for optional integrations
• backend-specific guarantees documented without abstraction leakage

Start with the documentation index, quickstart, adoption guide, and API reference. Review the backend matrix, failure model, service integration, and integration evidence before production use. Valkey adopters should use the Valkey 9 Streams guide and runnable example. RabbitMQ migrations should use the target-oriented adapter guide. Release history is maintained in CHANGELOG.md.

For ecosystem-wide selection and ownership guidance, see the versioned Golib ecosystem index and Persistence and durability family guidance.

Run make check before submitting a change. Backend changes must also pass make integration with the services documented in CONTRIBUTING.md.

Read CONTRIBUTING.md and follow the code of conduct. Every backend change must document its delivery and settlement impact.

Report vulnerabilities privately according to SECURITY.md. Review docs/security.md before processing untrusted jobs.

queue is available under the MIT License. Fork provenance and third-party attribution are recorded in NOTICE and THIRD_PARTY_NOTICES.md.