Skip to content
This repository was archived by the owner on Aug 11, 2026. It is now read-only.
This repository was archived by the owner on Aug 11, 2026. It is now read-only.

chore(visitor-analytics): add rollup, retention purge, readiness checks, and docs #624

Description

@ahliweb

Context

Visitor analytics can become high-volume quickly. After schema, collection, API, UI, and optional online enrichment are available, the module needs operational safeguards: rollup jobs, retention purge, config validation, readiness checks, and documentation.

This issue closes the operational and compliance loop for the Visitor Analytics epic.

Objective

Add visitor analytics rollup, retention purge, readiness/config checks, and documentation for online/offline operation, privacy, and security.

Scope

Add scripts:

scripts/visitor-analytics-rollup.ts
scripts/visitor-analytics-purge.ts

Add package scripts:

{
  "analytics:rollup": "bun scripts/visitor-analytics-rollup.ts",
  "analytics:purge": "bun scripts/visitor-analytics-purge.ts"
}

Rollup behavior:

  • Aggregate awcms_mini_visit_events into awcms_mini_visitor_daily_rollups.
  • Support idempotent reruns for a date or date range.
  • Roll up per tenant and per area.
  • Populate human visitors, human pageviews, bot pageviews, authenticated unique users, public unique visitors, admin unique users, top paths, top browsers, top devices, and top countries where available.

Purge behavior:

  • Delete or anonymize raw/old detail according to retention config.
  • Preserve rollups within rollup retention.
  • Prefer small batches to avoid long locks.
  • Record purge summary to audit log.

Config/readiness checks:

  • VISITOR_ANALYTICS_RAW_IP_ENABLED=true without safe retention should fail readiness.
  • VISITOR_ANALYTICS_RAW_USER_AGENT_ENABLED=true without safe retention should warn/fail according to severity.
  • VISITOR_ANALYTICS_GEO_ENABLED=true without trusted source should fail.
  • VISITOR_ANALYTICS_EVENT_RETENTION_DAYS must be valid positive integer.
  • VISITOR_ANALYTICS_RAW_DETAIL_RETENTION_DAYS must be valid and should not exceed event retention unless explicitly justified.
  • VISITOR_ANALYTICS_ROLLUP_RETENTION_DAYS should not be shorter than event retention.
  • VISITOR_ANALYTICS_HASH_SALT should be required or strongly warned when stable hashing is enabled.

Docs to update/add:

  • docs/awcms-mini/visitor-analytics.md
  • docs/awcms-mini/18_configuration_env_reference.md
  • docs/awcms-mini/deployment-profiles.md
  • docs/awcms-mini/20_threat_model_security_architecture.md
  • OpenAPI docs if affected by final API shape
  • README/module docs if needed

Out of scope

Acceptance criteria

  • bun run analytics:rollup exists and runs with Bun only.
  • Rollup can be rerun idempotently for the same date without double-counting.
  • bun run analytics:purge exists and runs with Bun only.
  • Purge removes/anonymizes data according to configured retention.
  • Purge keeps daily rollups while they are still inside rollup retention.
  • Purge records an audit event with safe summary metadata only.
  • Config validation covers visitor analytics env vars.
  • Security readiness reports unsafe raw detail/geolocation configurations.
  • Documentation explains offline/LAN mode, full online mode, trusted proxy/Cloudflare mode, privacy-first defaults, and retention policy.
  • Documentation maps controls to UU PDP, PP PSTE, ISO/IEC 27001/27002/27005/27701, OWASP ASVS, and OWASP Logging Cheat Sheet at a practical level.
  • Tests cover rollup idempotency, purge retention, invalid config, and readiness warnings/failures.
  • bun run config:validate passes.
  • bun run security:readiness passes or reports expected warnings for safe defaults.
  • bun run check passes.

Security and privacy notes

  • Visitor analytics data can identify or profile users when combined with username, IP, user-agent, and location; default retention must be conservative.
  • Raw IP and raw user-agent should be short-lived and disabled by default.
  • Rollups should avoid storing personally identifiable detail.
  • Purge jobs must not silently delete audit trail entries except according to the existing audit retention mechanism.
  • Avoid long-running transactions and table locks during purge.

Dependencies

Depends on #617, #618, #620, and #621. Complements #622 and #623.

Repository audit addendum — privacy defaults and consent

Platform hardening epic: #679

The 2026-07-11 repository audit found two privacy-default gaps that belong in this existing operational/privacy issue rather than a duplicate issue:

  • .env.example enables visitor analytics by default.
  • The anonymous visitor-key cookie lifetime is approximately two years.

Extend this issue's scope and acceptance criteria as follows:

  • Make visitor analytics explicitly opt-in/default-off for new installations and document the upgrade behavior for existing deployments.
  • Define a documented lawful purpose/configuration decision before collection is enabled; do not present the software setting itself as legal consent.
  • Minimize anonymous persistent identification: use a substantially shorter configurable lifetime, rotation, and revocation/deletion behavior.
  • Do not set the visitor-key cookie or persist events when analytics is disabled.
  • Keep raw IP and raw user-agent disabled by default; retain privacy-preserving hash/rollup behavior and conservative retention.
  • Document data-subject deletion/anonymization behavior and tenant-admin controls under UU PDP and ISO/IEC 27701:2025.
  • Add tests proving default-off behavior, no-cookie/no-write when disabled, rotation/expiry, and safe upgrade compatibility.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions