feat(payment-gateway): add provider-neutral checkout, signed webhook inbox, retries, and reconciliation #877

Parent epic: #868
Depends on: #869, #876

Context

SaaS subscription billing needs payment collection, but provider calls and webhook handling must remain isolated behind adapters. Payment status must never be trusted from browser redirects, and provider outages must not hold or roll back source database transactions.

Objective

Implement a provider-neutral payment_gateway capability for hosted checkout/session requests, signed inbound webhooks, normalized payment events, refunds/cancellations where supported, retry/DLQ, provider health, and reconciliation.

Scope

• Add admitted module/extension descriptor, permissions, provider registry, jobs, events, lifecycle descriptors, and admin navigation.
• Define provider-neutral contracts for:
  • create checkout/payment session;
  • query transaction status;
  • cancel/expire session where supported;
  • request refund where supported;
  • normalize provider events/status/error classes;
  • reconcile remote versus local state.
• Reuse integration-hub/domain-event/outbox patterns for outbound provider work and inbound webhook processing.
• Add schema for payment intents/sessions, provider references, webhook inbox, normalized events, processing attempts, allocations to subscription invoices, refund requests/results, and reconciliation evidence.
• Verify webhook signature, timestamp/freshness, provider/account binding, payload size, replay identity, and event ordering.
• Store raw payload only when explicitly required, encrypted/restricted, retention-bound, and redacted for logs.
• Implement idempotent event handling and safe out-of-order status transitions.
• Add provider adapters as optional configuration; include a fake/sandbox adapter for tests and documentation, not a hardcoded production dependency.
• Add timeout, bounded retry/backoff, circuit breaker, DLQ/manual retry, health/readiness, and reconciliation workers.
• Add OpenAPI/AsyncAPI and admin UI for sessions, webhook attempts, reconciliation, refund requests, and provider health.
• Notify feat(subscription-billing): add subscription, invoice, credit, renewal, and dunning state machines #876 only through validated normalized outcomes/events.

Out of scope

• General ledger, bank reconciliation, merchant settlement accounting, tax, payment allocation across tenant business invoices, or PCI card-data handling.
• Collecting raw card credentials.
• Treating browser return URL as payment confirmation.
• Mandatory internet/provider dependency for LAN/offline/manual-payment mode.

Security and integrity requirements

• Secrets come only from environment/secrets manager references and are never returned or logged.
• Provider endpoints and callback URLs are allow-listed; SSRF/open-redirect protections apply.
• Webhook replay, duplicate delivery, out-of-order events, signature failure, and oversized payload are handled fail-closed.
• Provider call occurs outside DB transaction; local intent/outbox is committed first.
• Payment status transitions are monotonic or explicitly compensating; invalid regressions are rejected/reconciled.
• Refund/cancel actions require dedicated permissions, mandatory reason, idempotency, audit, and SoD/step-up guidance.
• Tenant/provider/account binding prevents cross-tenant event substitution.
• Logs/audit use masked provider references and safe error classes.

Acceptance criteria

• A sandbox/fake adapter can create a hosted checkout session asynchronously.
• Valid signed webhook updates a payment exactly once.
• Invalid signature, replay, wrong tenant/account, stale timestamp, and oversized payload are rejected.
• Out-of-order events produce deterministic safe state and reconciliation evidence.
• Provider timeout/outage yields retry/DLQ without holding source transactions.
• Browser redirect alone never marks an invoice paid.
• Reconciliation detects and resolves/flags local-provider mismatch idempotently.
• LAN/offline/manual-payment mode runs without provider configuration or outbound calls.
• Provider secret/token/raw sensitive payload never appears in log, audit, response, or generated docs.
• OpenAPI/AsyncAPI/UI/metrics/runbooks are synchronized.

Testing

• Unit tests for normalized status transitions, signature verification, replay, ordering, error classification, and adapter contract.
• PostgreSQL integration tests for RLS, inbox idempotency race, event same-commit handling, allocations, and least-privilege roles.
• Fake-provider integration tests for success, timeout, duplicate webhook, reordered events, refund, and outage.
• SSRF/open-redirect/payload-limit/redaction security tests.
• Worker restart, retry, DLQ, circuit-breaker, and reconciliation tests.
• E2E invoice → checkout → webhook → paid status.
• Mutation tests for signature bypass, browser-return trust, and missing tenant binding.

Documentation

• Provider adapter guide, ERD/data dictionary, OpenAPI/AsyncAPI, secrets/config reference, webhook and reconciliation SOP, provider outage/incident runbook, refund controls, privacy/retention matrix, module README, and changeset.

Dependencies

Feeds platform operator UI #878, security controls #879, operations/reporting #880, and pilot #881.