Skip to content

feat(analytics): credit paid purchases to the source product in Overwatch - #4434

Merged
TRAP-RCG merged 3 commits into
mainfrom
feat/credit-purchases-to-source-product
Oct 12, 2026
Merged

feat(analytics): credit paid purchases to the source product in Overwatch#4434
TRAP-RCG merged 3 commits into
mainfrom
feat/credit-purchases-to-source-product

Conversation

@TRAP-RCG

@TRAP-RCG TRAP-RCG commented Oct 12, 2026 •
edited by onoya
Loading

Copy link
Copy Markdown
Contributor

Summary

Signups are already credited in Overwatch to the product a user came through (the first/last touch utm.source that names a product with an ingest key, marked attribution: 'self-reported'). This adds directional, self-reported raw purchase events for individual subscription invoices and credit packs. It does not add a cross-product subscribe funnel stage or a payout-grade ledger.

Owner decision (2026-10-13): approved for directional raw purchase telemetry from individual accounts. This includes purchases credited from stored signup touches after later cookie-consent withdrawal; the limitation is documented below. These self-reported events must not feed payouts, contractual counts, external reports, or an externally accessible funnel. Cross-product subscribe and payout-grade attribution remain deferred until #4118 provides the host-owned, Stripe-reconciled ledger.

What is sent

One purchase event per source product, per payment:

field value
event purchase
eventId stableEventId('purchase', productId, <Stripe invoice id or payment intent id>)
userId the payer, pseudonymized like every other emit
utm the stored touch that named the product
metadata.touch first / last / both
metadata.attribution self-reported
metadata.kind subscription / credits
metadata.value amount collected, major units (29.99), the GA4 convention the client pixel uses
metadata.currency ISO 4217, upper case
metadata.billingReason invoices only: subscription_create, subscription_cycle, subscription_update

Not named subscribe: Overwatch's signup stage matches both signup and subscribe, so a renewal sent as subscribe would be counted as a new signup.

Where money is collected

  • Subscription invoices (individual user only): apps/workers/src/events/stripe/invoicePaymentSucceeded.ts, after the subscription/credit work and the stage guard. Every invoice that collected money counts (first charge, renewal, seat increase).
  • Credit packs: payment_intent.succeeded in pages/api/stripe/webhook.ts, after grantCreditPurchase commits (the only caller; start-payment.ts is the only place pack intents are created).

Excluded: organization invoices (the checkout user is not the organization's acquisition source), zero-amount invoices (trials, fully discounted or balance-paid), refunds and disputes, synthetic users, foreign-stage events, and intents without credits.

Where the credit is read from, and why

From user.acquisition (the consented touches stored at signup), not from cookies or checkout metadata:

  • webhooks have no browser, so there are no cookies to read;
  • credit-pack payment intents carry no attribution metadata at all, so a single source for both kinds of purchase has to be the user record;
  • it keeps a product's purchases on the same product its signup was credited to, so signup -> purchase lines up per product.

A user with no stored acquisition (declined consent, or signed up before touches were stored) credits nothing.

Safety

  • Idempotent per Stripe invoice / payment intent, so webhook and handler retries send the same eventId and the receiver keeps one event (two source products get two distinct ids).
  • Never throws and never rejects: the helper fails open (emitter rejections, a throwing key lookup, a failed payer lookup), and the emit is awaited with the emitter's own 2s timeout. Failed purchase delivery logs a warning without the ingest key or payload. A delivery whose credits landed cannot be failed or retried by analytics.

Docs

docs-site/docs/security/telemetry-data-classification.md described the subscription webhook as emitting nothing. It now describes the purchase stream: what is sent, where from, that Overwatch stores it as raw events only (its funnel has no purchase stage), and that a later consent withdrawal does not stop stored touches crediting purchases.

Test plan

  • signupEvents.test.ts: first/last/both touches, host and keyless sources excluded (with the host keyed), no acquisition / no user, synthetic skipped, zero/negative/missing amount, missing payment id or currency, stable eventId per payment, value and currency, never throws when the emit rejects or the key lookup throws
  • invoicePaymentSucceeded.test.ts: paid create/cycle/update individual invoices credited with the invoice-keyed eventId (stored touches used, checkout metadata ignored), organization invoice excluded, $0 invoice, synthetic and no-acquisition payers, foreign stage, handler still resolves when the payer lookup or emit fails. Replaces the test that held the deferral.
  • webhook.test.ts: credit pack credited and keyed on the payment intent, same eventId on redelivery, nothing for no-touch or synthetic buyers, other environments or no credits, nothing when the grant rolls back, 200 when the emit rejects
  • Mutation-checked: synthetic guard, amount guard, host exclusion, eventId key, value units, currency, never-throws, payer-lookup guard, webhook payment id and kind each break at least one test
  • Purchase transport tests: server and network failures warn without exposing ingest keys or failing the caller
  • Typecheck: no new errors in client or workers

🤖 Generated with Claude Code

@TRAP-RCG
TRAP-RCG requested a review from a team as a code owner October 12, 2026 15:24
@TRAP-RCG
TRAP-RCG marked this pull request as draft October 12, 2026 15:47
@TRAP-RCG

TRAP-RCG commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

Self-review

The fixes are in 2257dba5f. The PR is now a draft, because the blocker below is a product decision, not code.

Blocking - no recorded decision to credit purchases

The description said "the product decision is in". The only decision on record says the opposite. @onoya on #3465, recording Keep for signups: "Cross-product subscribe stays deferred until [#4118] exists". #4118, the host-owned, Stripe-reconciled payout ledger, is still open with no comments, and I found no later decision. The description now says this PR is blocked on that decision. @onoya, this needs your go or no-go recorded here before it can come out of draft.

Fixed

  • The published telemetry doc contradicted the code. telemetry-data-classification.md still said "The subscription webhook emits nothing; that path stays deferred." It now covers:
    • the purchase stream: subscription invoices and credit packs;
    • where the credited product comes from: the touches stored at signup;
    • what each event carries, and the exclusions;
    • idempotency per invoice or payment intent;
    • the two points below.
  • No purchase stage in Overwatch. The emitter docblock called this "the stage after signup in that product's funnel". Overwatch's funnel is visits → signups (SIGNUP_EVENTS = ['signup', 'subscribe']) → new subscribers from product-stats snapshots, with nothing reading purchase. These events are stored as raw events only. The docblock and the doc now say so.
  • Consent withdrawal is not honoured for purchases. Touches are stored with consent at signup, and every later purchase is credited from them. The webhooks have no browser, and the server keeps no per-user consent state, so a visitor who later withdraws consent keeps having purchases credited to the source product. This can't be fixed in this PR without server-side consent state. It is now stated plainly in the docblock and in the doc rather than left implicit, and it belongs in the decision above.
  • Module header: it claimed purchase crediting "was a deliberate product decision". That's replaced with a must-stay-in-sync pointer to the doc.

Verified, no change needed

  • Invoice worker has the Overwatch config: the invoice Lambda links allSecrets, which includes OVERWATCH_INGEST_URL, the ingest keys and the pseudonym salt. The emit won't silently no-op in deployed stages.
  • Payer lookups: findById and findByStripeCustomerId both return toJSON() with acquisition and isSynthetic intact. Neither is select: false, and the transform doesn't strip them.
  • Schema: Overwatch accepts any event name up to 128 characters. The metadata is flat scalars, well under the 1KB cap.
  • value = amountMinor / 100: exact. Checkout is USD-only and enforced (USD_CHECKOUT_SESSION_PARAMS, and start-payment.ts pins usd).
  • eventId: keyed on the Stripe invoice or payment intent id, not the user id, so it can't leak the raw user id the way the signup id did before feat(analytics): send signup to the product a new user came through #3465's fix.
  • Never throws: both call sites sit after the credit grant, the emitter is fail-open, and the payer lookup is caught, so a webhook can't be failed or retried over analytics.

Checks

  • signupEvents.test.ts + webhook.test.ts: 56 pass.
  • invoicePaymentSucceeded.test.ts: 10 pass.
  • Help content validation passes.
  • The diff is ASCII only.

🤖 Generated with Claude Code

@TRAP-RCG
TRAP-RCG marked this pull request as ready for review October 12, 2026 16:05
@onoya onoya added review:ongoing Review in progress and removed awaiting review labels Oct 12, 2026
TRAP-RCG and others added 3 commits October 13, 2026 00:47
@TRAP-RCG @claude @onoya
feat(analytics): credit paid purchases to the source product in Overw…
6f56689 
…atch

Signups were already credited to the product a user came through (their
first/last touch utm.source, marked self-reported). Paid purchases now are
too, as a `purchase` event under each such product:

- subscription invoices, from the invoice.payment_succeeded worker, for
  every invoice that collected money (first charge, renewal, seat increase);
- credit packs, from payment_intent.succeeded after the credits are granted.

The touches are read from user.acquisition (stored with consent at signup),
since webhooks have no cookies and credit-pack intents carry no attribution
metadata. Zero-amount invoices (trials), refunds and synthetic users send
nothing. The eventId is keyed on the Stripe invoice or payment intent id, so
redeliveries are de-duplicated by the receiver. The emit never throws and
cannot fail or retry a webhook.

Not named `subscribe`: Overwatch counts `subscribe` as a signup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@TRAP-RCG @claude @onoya
docs(analytics): describe the purchase stream accurately
4956890 
- Update the telemetry classification doc, which still said the
  subscription webhook emits nothing: it now describes the purchase
  events, what they carry, that Overwatch stores them as raw events only
  (its funnel has no purchase stage), and that a later consent withdrawal
  does not stop stored signup touches crediting purchases.
- Correct the emitter docblocks to match: no funnel stage, the consent
  limitation, and a pointer to the doc instead of a decision claim.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@onoya
fix(analytics): exclude org purchases and log ingest failures
78be284
@onoya
onoya force-pushed the feat/credit-purchases-to-source-product branch from 2257dba to 78be284 Compare October 12, 2026 16:54
onoya
onoya approved these changes Oct 12, 2026
View reviewed changes

@onoya onoya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I pushed 78be284dd: organization invoices no longer inherit the checkout user's signup source, including malformed organization metadata, and failed purchase emits now leave a diagnostic without failing billing. The 103 affected tests and both package typechecks pass.

As owner, I approve directional raw purchase telemetry for individual subscriptions and credit packs, including use of stored signup touches after later cookie-consent withdrawal. These self-reported events remain excluded from payouts, contractual counts, external reports, and externally accessible funnels. Cross-product subscribe stays deferred until #4118.

@onoya onoya added review:done Review complete and removed review:ongoing Review in progress labels Oct 12, 2026
@TRAP-RCG
TRAP-RCG added this pull request to the merge queue Oct 12, 2026
Merged via the queue into main with commit 4455ba8 Oct 12, 2026
40 checks passed
@TRAP-RCG
TRAP-RCG deleted the feat/credit-purchases-to-source-product branch October 12, 2026 18:02
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 12, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@onoya onoya onoya approved these changes

Assignees

@TRAP-RCG TRAP-RCG

Labels

review:done Review complete

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@TRAP-RCG @onoya @julsanchez