test(saas-control-plane): prove derived-app onboarding, sandbox billing, deployment, restore, and upgrade end-to-end #881

Parent epic: #868
Depends on: #870, #871, #872, #873, #874, #875, #876, #877, #878, #879, #880

Context

The control plane is only credible when a derived application can consume it without editing base registries or rebuilding core systems. The final proof must cover composition compatibility, tenant onboarding, entitlements, usage, billing, sandbox payment, lifecycle, security, operations, deployment, restore, and base-version upgrade.

Objective

Build a minimal external derived-application pilot/fixture and produce reproducible evidence that the SaaS control-plane contracts work end-to-end while preserving AWCMS-Mini boundaries and offline/LAN behavior.

Pilot scope

Use a deliberately small generic derived application with:

• one application module contributed through application-registry.ts;
• one feature key, one quota key, and one usage meter from feat(saas-contracts): add static feature, quota, meter, and commercial-event registries with conformance gates #874;
• one published plan version;
• one tenant provisioning plan with a derived step;
• one subscription and recurring invoice;
• one usage-based invoice line;
• fake/sandbox hosted checkout and signed webhook;
• one upgrade or downgrade request;
• one grace/suspension/restore lifecycle scenario;
• platform-operator and tenant self-service journeys.

The pilot must not introduce Platform Masjid, Jualanku.info, ERP, POS, marketplace, or vertical business logic into the base.

Scope

• Create or use an approved separate derived repository/fixture according to docs(saas-control-plane): decide module admission, control-plane boundary, trust model, and lifecycle contracts #869.
• Consume AWCMS-Mini through the official composition and compatibility manifest/test kit.
• Prove no edits to base src/modules/index.ts, applied migrations, or internal module tables.
• Add derived migration namespace/ownership and cross-tenant RLS tests.
• Execute the full journey:
  1. publish plan;
  2. request/resume tenant provisioning;
  3. activate tenant and entitlements;
  4. enforce feature/quota;
  5. emit/deduplicate/aggregate usage;
  6. create subscription and invoice;
  7. create sandbox checkout;
  8. process signed webhook exactly once;
  9. reconcile payment and invoice;
  10. run downgrade/grace/suspension/restore;
  11. verify dashboards, audit, events, metrics, and reports.
• Add Docker development/LAN setup, Dockerfile.production, Coolify deployment guide, environment reference, internal DB network, least-privilege roles, and no public PostgreSQL exposure.
• Run backup/restore and provider-outage/worker-restart drills.
• Rehearse one AWCMS-Mini base upgrade with migration/compatibility checks and no tenant data loss.
• Produce an evidence report mapping every epic acceptance criterion to tests, commands, logs/artifacts, and documentation.

Out of scope

• Production provider credentials or live financial transactions.
• Full billing/tax/accounting system.
• Kubernetes/microservices/sharding/distributed cache.
• Marketing website or vertical product features.

Security and operational requirements

• Runtime DB roles are least privilege, non-owner, non-superuser, and non-BYPASSRLS.
• All tenant data is RLS ENABLE + FORCE; cross-tenant negative tests are mandatory.
• Provider secrets remain sandbox/environment references and are absent from repository/log/audit.
• Webhook signature/replay, idempotency, concurrency, SoD, step-up, support access, masking, and safe errors are exercised adversarially.
• Provider calls occur outside source DB transactions.
• LAN/offline profile starts and operates without payment/network configuration; manual payment path remains testable.
• Backup is encrypted/integrity-checked as supported, and restore evidence includes RTO/RPO measurement.

Acceptance criteria

• Derived module composes and passes compatibility checks without editing base registry files.
• Provisioning is idempotent, resumable after injected failure, and produces one tenant.
• Effective entitlements and quota enforcement are deterministic and do not bypass RBAC/ABAC/RLS.
• Duplicate usage and webhook events are processed once.
• Invoice/credit/payment evidence uses exact amounts and immutable issued records.
• Browser return URL alone cannot mark payment successful.
• Downgrade/suspension preserves tenant data and restrictions apply across API, SSR, public routes, and workers.
• Platform/tenant role separation, SoD, step-up, and expiring support access are proven.
• Reporting projections rebuild/reconcile and expose freshness.
• Provider outage, worker restart, DB saturation, and stale projection have controlled degraded behavior.
• Docker/Coolify deployment, backup/restore, and one base upgrade rehearsal succeed.
• bun run check, PostgreSQL integration, contract, security, E2E, performance, build, readiness, and non-production preflight are green.
• Evidence report and all operator/user/security/deployment documentation are complete.

Testing evidence

• Unit, PostgreSQL integration, contract, adversarial security, browser E2E, accessibility, performance/load/soak, resilience, backup/restore, and upgrade rehearsal.
• Record files changed, commands, test counts/results, query-plan evidence, capacity assumptions, risks, limitations, and residual follow-ups.
• Any generic foundation gap found during the pilot must be filed as a separate atomic issue; do not silently expand this issue or move derived logic into base.

Documentation

• Pilot PRD/SRS-lite, architecture/ADR, ERD, OpenAPI/AsyncAPI, deployment/env/security/privacy/backup/incident/operator/tenant guides, evidence matrix, lessons learned, and updates to the AWCMS-Mini derived-application guide.