Competitive visibility for D2C brands — powered by custom Bright Data Scraper Studio collectors that self-heal when sites redesign.
| Status | Shipped · stable for the Lumen D2C beauty cohort |
| Repo | https://github.com/priyanshuchawda/brandradar |
| License | MIT |
BrandRadar watches a fixed cohort of rivals on their own public sites (not marketplaces), normalizes shelf and changelog data into a weekly snapshot, scores how clearly you can still “see” the market, and pushes a Monday brief to Discord. When a layout change empties an extract, the same Studio collector id (c_*) is healed in place — no redeploy, no new scraper identity.
What ships in this release
- Custom Bright Data Scraper Studio collectors per rival (own-domain serums + blogs)
- Gemini-assisted deep pass (sublink ranking + evidence-bound brief)
- Discord delivery into the channels the team already uses
- Monday Diff respond / watch / hold plays
- Heal Lab proof — break → zero → heal on the same
c_*
Founders, growth leads, and category managers at regional D2C brands who already know their own numbers and do not have a research team.
They need three answers every week:
- What changed on rival shelves and update pages?
- Where are we expensive, thin, or poorly rated?
- What should we do on Monday?
Spreadsheets go stale. Homegrown browser automation breaks on the first class rename. Marketplace library scrapers target the wrong competitive set. BrandRadar’s wedge is custom Studio collectors + self-heal + decision-ready plays.
| Surface | Path | Role |
|---|---|---|
| Control Tower | /control-tower |
Ops board for the POV brand Lumen — serum-shelf ₹ table, mission runs, Gemini-assisted plans |
| Monday Diff | /monday-diff |
Week-over-week rival update diffs, visibility score, plays |
| Heal Lab | /heal-lab/before · /after |
Owned changelog pages used to prove break → heal → recover on the same c_* |
| Discord | BrandRadar · Lumen server | Per-rival channels, #monday-diff, #heal-alerts, slash commands |
POV brand: Lumen (clean beauty D2C, India).
Watchlist: Mamaearth, Plum Goodness, Minimalist, Foxtale — each rival’s own serum collection + blog/news URLs.
Rival public URLs (own domains only)
│
▼
Bright Data Scraper Studio collectors (c_*)
Discovery listings · optional PDP · intel updates · heal-lab
│
▼
Normalize → Brand / Intel snapshot (Zod schemas)
│
├──► Diff + visibility + plays
├──► Control Tower ₹ shelf
├──► Discord embeds + slash commands
└──► Week cache (local disk / optional Vercel Blob)
Self-heal path (same collector id forever):
QA flags empty / null_rate / row_collapse
→ bdata scraper heal <same c_*> "…" --url <page>
→ approve / auto-approve
→ re-run → rows back
Detail: docs/architecture.md · Bright Data internals: docs/brightdata.md
- Create a custom Studio collector (CLI or Studio UI) for one URL shape — e.g. Plum serums listing.
- Run via CLI (
bdata scraper run) or app (POST /dca/triggerthroughlib/brightdata.ts). - Normalize raw Studio JSON (
lib/studio-normalize.ts) into typed rows (name, ₹ price, URL). - Persist intel weeks under
data/intel/<ISO-week>/(or Blob in production). - Diff vs prior week → visibility score → deterministic plays (
lib/plays.ts). - Deliver Control Tower UI + Discord (
lib/discord-embeds.ts). - Heal on QA failure with the same
c_*(lib/heal-engine.ts+lib/studio.ts).
Collector ids are the production API handle. Healing never mints a new id.
git clone https://github.com/priyanshuchawda/brandradar.git
cd brandradar
cp .env.example .env.local
npm install
npm run devOpen http://localhost:3000 (or http://127.0.0.1:3000).
npm test # unit tests
npm run lint
npm run discord:bootstrap # create Discord layout + pins (needs bot token)
npm run discord:tidy # prune junk channels + post example intelSecrets stay in .env.local — never commit tokens.
BrandRadar uses custom Scraper Studio collectors, not pre-built marketplace library scrapers.
# login once
npx -p @brightdata/cli bdata login
# create a Discovery collector for a listing page
npx -p @brightdata/cli bdata scraper create \
"https://plumgoodness.com/collections/serums" \
"Extract serum products: product_name, price, list_price, currency, product_url. Listing page only." \
--name brandradar-plum-discovery --pretty
# run
npx -p @brightdata/cli bdata scraper run c_YOUR_ID \
"https://plumgoodness.com/collections/serums" --pretty
# heal in place (same id)
npx -p @brightdata/cli bdata scraper heal c_YOUR_ID \
"Extract serum products: product_name, price, list_price, currency, product_url." \
--url "https://plumgoodness.com/collections/serums" --pretty
npx -p @brightdata/cli bdata scraper approve c_YOUR_ID \
--url "https://plumgoodness.com/collections/serums" --prettyIn the app, runs go through lib/brightdata.ts: SDK realtime first, then REST fallback POST /dca/trigger → poll GET /dca/dataset.
Full collector guide: docs/collectors.md
Pinned in config/rivals.json and/or .env.local:
| Rival | Surface | Collector env / id |
|---|---|---|
| Mamaearth | Face serum collection | COLLECTOR_MAMAEARTH_DISCOVERY · c_msxk5ul22mk8gvsdgy |
| Plum | Serums collection | COLLECTOR_PLUM_DISCOVERY · c_mt5dogddx8lyp1q7f |
| Minimalist | Face serum collection | COLLECTOR_MINIMALIST_DISCOVERY · c_mt5gcpsg1hqf5xpv81 |
| Foxtale | Serums collection | COLLECTOR_FOXTALE_DISCOVERY · c_mt5h0alj2rhv1aozp0 |
| Heal Lab | Owned before/after pages | COLLECTOR_HEAL_LAB · c_mt3ekwjs2lzsn3dwl7 |
| Intel updates | Rival blogs/news | COLLECTOR_INTEL_UPDATES |
Sample Plum extract (sanitized): data/demo/plum-serums-sample.json
{
"product_name": "10% Niacinamide & Rice Water Brightening Face Serum…",
"price": 307,
"list_price": 349,
"currency": "INR",
"product_url": "https://plumgoodness.com/products/…"
}| Variable | Purpose |
|---|---|
BRIGHT_DATA_API_TOKEN |
Studio + SDK (also set BRIGHTDATA_API_KEY to the same value for CLI) |
COLLECTOR_*_DISCOVERY / COLLECTOR_*_PDP |
Per-rival / vertical collectors |
COLLECTOR_INTEL_UPDATES |
Monday Diff update-page pulls |
COLLECTOR_HEAL_LAB |
Heal Lab owned-site collector |
USE_MOCK=false |
Prefer live Studio when ids are set |
| Variable | Purpose |
|---|---|
DISCORD_BOT_TOKEN |
Bot token |
DISCORD_GUILD_ID |
Server id |
DISCORD_APPLICATION_ID |
Application / client id |
DISCORD_PUBLIC_KEY |
Slash-command signature verify |
DISCORD_CHANNEL_ID |
#monday-diff |
DISCORD_HEAL_CHANNEL_ID |
#heal-alerts |
DISCORD_RIVAL_*_CHANNEL_ID |
Per-rival channels (optional) |
Setup: docs/discord.md
| Variable | Purpose |
|---|---|
GEMINI_API_KEY |
Optional heal-prompt drafting + play copy rewrite |
CRON_SECRET |
Auth for /api/cron/monday-diff |
BLOB_READ_WRITE_TOKEN |
Week snapshots on Vercel |
BRANDRADAR_API_KEY |
Lock mutating routes when the API is exposed |
Sites redesign. Extracts go empty. BrandRadar’s contract:
| Step | Behavior |
|---|---|
| Detect | Listing QA (lib/extract-qa.ts) — empty, high null rate, row collapse |
| Repair | bdata scraper heal on the same c_* with plain-language field intent |
| Verify | Re-run Collection API; proof is row counts returning (e.g. 5 → 0 → 5) |
| Notify | Discord #heal-alerts broken / recovered embeds |
Shared loop: lib/heal-engine.ts (≤2 heal passes locally; tighter budget on Vercel). Gemini is opt-in for prompt text only — it does not invent prices or URLs.
Owned proof pages (local after npm run dev):
- http://localhost:3000/heal-lab/before — healthy layout
- http://localhost:3000/heal-lab/after — redesigned DOM (same content, broken selectors)
Guide: docs/heal-lab.md
Weekly competitive intel on public update pages (blogs / guides / changelogs):
- Load cohort from
config/rivals.json - Pull listing rows per rival (Studio or cached week)
- Diff vs prior week →
added/removed/modified - Visibility score 0–100 + per-rival health
- Emit plays (
attack/watch/fill) — rule-based, not LLM-invented - Optional cron → Discord
#monday-diff
Guide: docs/monday-diff.md · Example: examples/intel-snapshot.json
| Method | Path | Role |
|---|---|---|
POST |
/api/intel |
Pull / cache cohort intel (forceMock, persist, refresh) |
POST |
/api/intel/heal |
Heal COLLECTOR_INTEL_UPDATES |
GET/POST |
/api/heal-lab |
Fixtures or live heal loop on owned pages |
POST |
/api/scan |
Catalog arena scan |
POST |
/api/discord/setup |
Bootstrap server + slash commands |
POST |
/api/discord/interactions |
/intel, /rivals, /schema, /help |
POST |
/api/cron/monday-diff |
Weekly pull → Discord (CRON_SECRET) |
All Bright Data and Gemini calls are server-side only. The browser talks to /api/*.
Listing / serum row (Studio Discovery):
{
"product_name": "…",
"price": 307,
"list_price": 349,
"currency": "INR",
"product_url": "https://…"
}Intel listing row (update pages):
{
"title": "…",
"url": "https://…",
"published_at": "2026-08-01",
"summary": "…"
}Intel snapshot (data/intel/<week>/snapshot.json): rivals[], diff[], visibility, plays[], health.collector_ids[].
Schemas: lib/intel-schema.ts · lib/schema.ts
| Component | AI? | Notes |
|---|---|---|
| Bright Data Scraper Studio create / heal | Yes | Studio AI Flow generates / repairs collector logic |
| Monday Diff plays + visibility | No | Deterministic rules from diff + row health |
| Discord embeds | No | Templates from snapshot JSON |
| Gemini Flash | Optional | Heal prompt drafting when QA flags break (useGemini) |
| Gemini Flash-Lite | Optional | Arena rival pick + play copy rewrite — numbers stay on extracted rows |
We do not invent prices, URLs, or rival posts. Mock paths are labeled mode: fixture / mode: mock.
- Next.js App Router + TypeScript + Zod
- Bright Data Scraper Studio CLI (
@brightdata/cli) + SDK (@brightdata/sdk) - Discord HTTP Interactions (Ed25519) — not a Gateway bot for slash commands
- Optional Google Gemini for copy / heal prompts
- Tests Vitest (
npm test)
See docs/stack.md.
- Public HTTPS pages only — no logins, paywalls, or personal data
- Each rival’s own domain — not aggregators or marketplace scrapers as the product extractor
- Mask secrets in recordings; rotate any leaked tokens
Security notes: docs/security.md
| Doc | Contents |
|---|---|
| docs/operations.md | Local runbook, Studio + Discord verify |
| docs/architecture.md | System diagram, pipelines, modules |
| docs/product.md | Problem, users, plays, verticals |
| docs/collectors.md | Studio create / run / heal |
| docs/brightdata.md | Studio types, /dca/trigger, CLI |
| docs/heal-lab.md | Self-heal proof on owned pages |
| docs/monday-diff.md | Cohort intel, visibility, cron |
| docs/discord.md | Server layout, embeds, slash commands |
| docs/integrations.md | Bright Data + Gemini boundaries |
| docs/security.md | Auth, rate limits, URL policy |
| docs/stack.md | Language and runtime choices |
| examples/ | Sample snapshot + PDP JSON |
- More verticals (edtech, food) with the same Discovery → PDP → heal contract
- Stronger week memory and alerting without a heavy warehouse
- Deeper Discord ops (threaded shelf diffs, scheduled missions)
- Hardening production auth and multi-tenant tenancy (out of scope today)
See CONTRIBUTING.md. Issues and PRs welcome.
npm test && npm run lint