ETF Radar — Cross-Market Theme Linkage Analysis

Tracks US theme ETF strength and momentum, auto-maps them to China A-share ETFs, and identifies resonance /传导 / divergence signals. Also provides a market temperature gauge (MA20/MA5 standing rate), constituent stock analysis, and portfolio monitoring to help A-share retail investors spot cross-market opportunities.

中文文档

• Primary: https://im47.cn/etf-radar/ (custom domain)
• Fallback: https://im47cn.github.io/etf-radar/ (GitHub Pages default)

1. GitHub Actions triggers the Python pipeline on a schedule (backend/src/pipeline.py)
2. Fetches ETF OHLC data from yfinance (US) and AkShare (China A-shares)
3. Computes multi-period log returns → dual-track strength scoring (percentile × sigmoid momentum) → 60-day correlation mapping score → multi-period voting signals
4. Writes to data/latest/{themes,etfs,signals,meta}.json and commits back to the repo
5. Archives daily to data/snapshots/<YYYY-MM-DD>/
6. Frontend (React + Vite + zod) builds statically and deploys to GitHub Pages; client fetches JSON for rendering

cd backend
uv venv && uv sync --extra dev
uv run pytest                              # run tests (see CI for counts)
uv run python -m src.pipeline --mode=full --data-root=../data --config-dir=../config

cd frontend
npm ci
npm run dev      # http://localhost:5173/etf-radar/
npm run build
npm test -- --run  # run tests (see CI for counts)

The /portfolio and /watchlist routes require Supabase credentials. Other pages work fine without them, but these pages will show an "unconfigured" notice.

cp frontend/.env.local.example frontend/.env.local
# Edit .env.local with your Supabase Project URL and anon key
# Ask the project maintainer for credentials, or create your own Supabase project

npm run dev  # http://localhost:5173/etf-radar/#/trading?tab=holdings

Magic Link login: emails may land in the spam folder of Chinese mailboxes (QQ/163) — please check; or use Google OAuth for one-click login.

Database Schema: see backend/migrations/001_user_holdings.sql and subsequent migration files; execute them sequentially in the Supabase SQL Editor.

cd backend && uv run python ../scripts/bootstrap_data.py
git add data/latest/ && git commit -m "data: initial seed" && git push

cd backend && uv run python ../scripts/calibrate_algo.py

Outputs strength distributions at different K values; deviation ≤15% is marked ✓. Adjust config/algo.yml accordingly.

A daily cron archives data/latest/ to data/snapshots/<BJT-date>/. The data/latest/snapshots-index.json is maintained by the archive/backfill scripts; the frontend uses it to discover available dates.

First-time backfill (one-off): if snapshot history is insufficient, run the backfill script to generate historical data:

cd backend
uv run --all-extras python -m scripts.backfill_snapshots --start 2026-01-02 --end 2026-06-13

Backfilled meta.json carries a backfilled: true flag to distinguish from automated archives. --skip-existing is on by default to protect real archived data.

backend/scripts/backfill_snapshots.py can rebuild snapshots for the past N trading days in one pass without modifying any scoring logic, used to drive the /rotation timeline animation. How it works: compute_outputs() accepts an asof_bjt anchor, slices the in-memory OHLC DataFrame by day, and reuses the same scoring functions.

cd backend && uv run --all-extras python -m scripts.backfill_snapshots \
  --start 2026-01-02 --end 2026-06-13 \
  --data-root ../data --config-dir ../config

• --skip-existing (on by default) protects existing archives; will not overwrite real cron output
• --force forces rewrite
• Output: data/snapshots/<YYYY-MM-DD>/{themes,signals,etfs,meta}.json + data/latest/snapshots-index.json
• ~120 trading days take 1-2 minutes (one network fetch + in-memory slicing)
• akshare occasional rate limiting: failed CN ETFs are recorded in meta.providers.cn.failed_symbols and overall status degrades to degraded; other themes are unaffected

python scripts/archive_cleanup.py

• US market: yfinance (Yahoo Finance, ~15 min delay)
• A-shares: AkShare (Eastmoney data, ~15 min delay)
• Market temperature: dapanyuntu.com (market cloud map, 2nd-level industry MA20 standing rate)
• L1 soft resilience: on failure, retains the last successful snapshot; UI shows a yellow banner warning "data is XX minutes stale"
• mypy strict + ruff + jsonschema: multi-layer data contract validation (Pydantic on write + JSON Schema validation + frontend zod runtime parse)
• Health monitor + self-heal: hourly checks of data freshness and critical workflow run status; auto-dispatches compensating workflows on anomalies

• GitHub Pages Source = GitHub Actions (uses actions/deploy-pages; not Jekyll / gh-pages branch)
• deploy-frontend.yml trigger: push to main with changes in frontend/** or data/latest/**
• Data refresh workflows (cn/us-refresh) explicitly call gh workflow run deploy-frontend.yml at the end to trigger deployment
  • Reason: commits pushed by GITHUB_TOKEN do not trigger downstream workflows (anti-loop safety policy)
• Custom domain (optional): currently set to im47.cn; change in Settings → Pages → Custom domain

• Design & implementation process docs: session-scoped artifacts, not tracked in-repo (see docs/CONVENTIONS.md)
• Original product doc: docs/htsc-us-cn-linkage-product-doc.md
• Original requirements: docs/htsc-us-cn-linkage-requirements.md

MIT