feat(data-exchange): add staged import, validation, preview, idempotent commit, export, and reconciliation #752

Parent epic: #738
Depends on: #739, #742
Admission requirement: apply docs/awcms-mini/21_module_admission_governance.md before implementation.

Context

Derived applications repeatedly need safe CSV/JSON import and export. Implementing this separately in every module risks long-running HTTP requests, partial writes, cross-tenant mixing, formula injection, unbounded file parsing, inconsistent validation, and irreconcilable exports.

The base needs a generic staged exchange framework while each owning module supplies its own schema, validation, mapping, and commit adapter.

Objective

Add a provider-neutral data-exchange capability for staged upload/intake, schema validation, preview/diff, asynchronous idempotent commit, resumable partial failure, export manifests/checksums, and reconciliation.

Scope

• Complete module-admission decision/ADR before scaffold.
• Define static module-contributed exchange descriptors containing:
  • import/export key and owner module;
  • accepted formats and versioned schema;
  • max file/row/field limits;
  • validation/mapping capability;
  • commit adapter and idempotency semantics;
  • required permission/business scope;
  • sensitive-field/redaction policy.
• Support CSV and JSON as the baseline; add other formats only through later admission/evidence.
• Add staged intake using approved managed-file/object references, checksum, media-type verification, and safe filename metadata.
• Add asynchronous parse/validate/preview jobs with bounded batches, progress, pause/resume/cancel, error artifacts, and expiry.
• Provide preview with totals, valid/invalid/warning rows, normalized values, proposed create/update/skip/conflict actions, and safe sample limits.
• Add explicit idempotent commit separate from preview; no mutation occurs during validation/preview.
• Support resumable partial failure according to module-declared atomicity strategy; never imply all-domain universal transaction semantics.
• Add export jobs with snapshot/as-of metadata, schema/version, filter/scope, row count, checksum, file manifest, and expiry.
• Add reconciliation comparing source/commit/export counts and checksums or module-provided control totals.
• Add accessible admin UI for jobs, progress, preview, errors, commit, export, download, retry, cancel, and history.
• Add lifecycle integration with feat(data-lifecycle): add retention, partitioning, archival, legal-hold, and safe purge foundations #745 and domain events through feat(domain-events): add transactional event outbox, idempotent consumers, retries, ordering, and dead-letter handling #742.
• Add metrics for rows/bytes processed, validation outcome, queue/processing duration, retry/failure, and reconciliation mismatch using low-cardinality labels.

Security and safety requirements

• Large imports never execute as one long HTTP request or one unbounded database transaction.
• Tenant and business scope are captured server-side and revalidated at every job phase; staged data from tenants cannot mix.
• Parsers are size/row/field/time bounded and resistant to CSV formula injection, zip/decompression bombs if archives are later supported, path traversal, unsafe encodings, and malicious content metadata.
• Spreadsheet-compatible exports neutralize formula prefixes where applicable.
• Preview/error artifacts minimize and mask PII; raw invalid values require explicit permission and retention limits.
• Commit is permission-gated, reason/audit enabled, idempotent, and safe on worker restart.
• Module adapters cannot directly write another module's tables.

Out of scope

• Generic ETL scripting language, arbitrary SQL mapping, tenant-uploaded code, or runtime plugins.
• Real-time data synchronization; use events/integration contracts for that.
• XLSX/PDF as baseline formats unless separately admitted.

Acceptance criteria

• Admission decision/ADR confirms module category, ownership, dependencies, and offline support.
• CSV and JSON descriptors are schema-versioned and contributed by modules through static contracts.
• Intake enforces checksum, media type, size/row/field/time bounds, and safe filename handling.
• Preview performs no domain mutation and shows deterministic proposed actions/errors.
• Commit is explicit, idempotent, resumable, audited, and revalidates tenant/scope/permission.
• A worker interruption and retry do not duplicate committed rows.
• Cross-tenant staging/commit/download is rejected by RLS/ABAC negative tests.
• Export manifest includes schema/version, filters/scope, row count, checksum, and creation metadata; reconciliation can detect a deliberate mismatch.
• CSV formula injection and malicious/oversized input fixtures are neutralized/rejected.
• At least three module fixtures demonstrate create/update/conflict, partial failure/resume, and export/reconciliation.
• OpenAPI/AsyncAPI, ERD/data dictionary, threat model, file/lifecycle/job runbooks, admin/user guide, repository inventory, and changeset are updated.
• bun run api:spec:check, bun run security:readiness, unit/PostgreSQL/worker/UI/E2E tests, build, and bun run check pass.