Skip to content
This repository was archived by the owner on Aug 11, 2026. It is now read-only.
This repository was archived by the owner on Aug 11, 2026. It is now read-only.

test(resilience): add failure-injection and disaster-recovery verification #699

Description

@ahliweb

Parent epic: #679
Depends conceptually on: #684, #691, #697

Objective

Verify documented recovery behavior under controlled failures and produce reproducible RTO/RPO evidence.

Scope

  • Add non-production failure scenarios for PostgreSQL disconnect/restart, pool saturation, worker interruption, provider timeout/outage, and partial R2/email/SSO failures.
  • Verify migration rehearsal, backup restore, and rollback/recovery paths in isolated infrastructure.
  • Assert critical local transactions remain independent of optional provider outages.
  • Produce machine-readable scenario results plus a concise DR report.
  • Add safety interlocks so chaos tests cannot target production.

Guardrails

  • Never run destructive tests against production or shared tenant data.
  • Use synthetic fixtures and isolated credentials.
  • Do not log secrets or raw personal data.

Acceptance criteria

  • Production-like target detection blocks execution by default.
  • Scenarios have deterministic setup, timeout, cleanup, and pass/fail criteria.
  • RTO/RPO measurements are recorded for database restore and representative services.
  • Retry/idempotency behavior avoids duplicate side effects.
  • Runbook discrepancies become tracked follow-up issues.
  • CI runs a safe subset; full drill cadence is documented.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:deploymentDeployment profile, systemd, backuparea:securitySecurity and 2FA workpriority:p2Prioritas sedang - opsional/provider-dependenttype:testPerubahan/penambahan test sesuai doc 06

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions