Know what's really behind your API endpoint.

Detect model swaps · Monitor endpoint drift · Redact sensitive data

A local transparent proxy that sits between your app and any OpenAI-compatible API, giving you visibility and local control that are hard to get from hosted relays alone.

Quick Start · Why This Matters · Features · How It Works · Bundled Fingerprints · Configuration · API Reference · Citation

---

You're paying for a premium model. But is that what you're actually getting?

Third-party API relays are everywhere — they offer cheaper prices, regional availability, and unified billing. But they also create a trust gap:

RelayRadar makes the invisible visible. It's not a testing tool you run once — it's continuous monitoring that catches problems as they happen.

# Clone and install
git clone https://github.com/AI45Lab/RelayRadar.git
cd RelayRadar && pnpm install

# Start everything
pnpm dev

Open http://localhost:5173 for the Console, then create your endpoint in Endpoint Manage. RelayRadar stores endpoint settings and fingerprint baselines in SQLite (data/relayradar.sqlite).

After that, point your SDK at http://localhost:8080/v1:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="your-key")
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)
# Everything works exactly the same — but now you have visibility.

Intercepts every request and response, redacting sensitive data before it leaves your network.

• Auto-detection: Emails, phone numbers, ID numbers, API keys (sk-*, ghp_*), JWTs, database URIs, private keys
• Reversible redaction: Replaces sensitive data with opaque [[RR_...]] tokens, restores them in responses
• Custom rules: Define literal strings to always redact (project names, internal IDs)
• Prompt protection: [[LOCAL_ONLY:...]] markers strip fragments before upstream forwarding
• Response blocking: Optionally block responses flagged as high-risk
• ML Privacy Filter: Optionally run openai/privacy-filter (ONNX, runs fully locally) on every request to catch PII that pattern rules miss — names, addresses, account numbers, and more. Enabled via privacyFilterEnabled in policy; threshold tunable. Falls back to pattern redaction if the model fails.

Shield targets identity-linking and credential data — not functional inputs to your task. The rule of thumb:

Pattern rules and the ML filter are both calibrated to target Incidental and Credential PII. Functional inputs (numbers, dates, measurements used as task parameters) are passed through unchanged.

Probes every endpoint on a schedule, tracking behavioral stability over time.

• 28 built-in challenge prompts covering JSON formatting, code generation, multilingual output, constraint following, and more
• Dice coefficient comparison between consecutive responses — detects when outputs start changing
• Designed to reduce false positives on stable endpoints: Pure text-similarity based, no format-constraint noise
• Automatic escalation: Stable → Watch → Drifted → High Risk based on divergence thresholds
• Toggle from UI: Enable/disable Sentinel per-policy directly from the Console

Answers the question: "Is this endpoint really running the model it claims?"

• AB3IT fingerprinting: Adaptive three-phase discovery — broad scan → variant exploration → reference sampling — selects the most model-discriminative prompts automatically
• Statistical rigor: Total Variation Distance + permutation testing for p-values
• Universal compatibility: No logprobs required — works with OpenAI-compatible relays and gateways, including reasoning models
• Baseline catalog: Ships with 19 bundled AB3IT baselines and lets you build, store, and compare your own
• Model-identity checks: Compares live endpoint behavior against a known baseline and reports match, mismatch, or inconclusive evidence

A real-time web UI for monitoring and management.

• Overview: All endpoints at a glance with status badges, drift scores, request volume
• Endpoint Detail: Metric cards, 4-panel sparkline trends (latency, errors, drift, fingerprint), risk event timeline
• Shield Center: Redaction stats, policy editor, manual redaction string management
• Fingerprint Lab: Model definitions, baseline sampling with progress tracking, audit history

┌─────────────┐     ┌──────────────────────────────┐     ┌─────────────────┐
│  Your App   │────▶│     RelayRadar Proxy (:8080)  │────▶│  Upstream API   │
│             │◀────│                                │◀────│  (relay/direct) │
└─────────────┘     │  ┌─────────┐  ┌────────────┐  │     └─────────────────┘
                    │  │ Shield  │  │  Request    │  │
                    │  │ redact  │  │  Logger     │  │
                    │  │ restore │  │  (SQLite)   │  │
                    │  └─────────┘  └────────────┘  │
                    └──────────┬───────────────────┘
                               │
                    ┌──────────┴───────────────────┐
                    │  Background Services          │
                    │  ┌──────────┐ ┌────────────┐  │
                    │  │ Sentinel │ │ Fingerprint │  │
                    │  │ (drift)  │ │ (identity)  │  │
                    │  └──────────┘ └────────────┘  │
                    └──────────────────────────────┘
                               │
                    ┌──────────┴───────────────────┐
                    │  Console UI (:5173)           │
                    │  React + Vite                 │
                    └──────────────────────────────┘

Every 30 minutes by default (configurable), Sentinel probes each endpoint:

1. Send — Challenge prompt from a pool of 28 probes (JSON, code, bilingual, constraint-following, etc.)
2. Compare — Compute Dice coefficient against the previous response signature
3. Score — Divergence = 1 − similarity (0 = identical, 1 = completely different)
4. Escalate — Record drift events when divergence exceeds thresholds

Drift tracks change over time for a single endpoint. It doesn't need a reference — just history.

AB3IT (Adaptive Border-Based Behavioral Identity Testing) is RelayRadar's model fingerprinting method. Instead of a fixed probe set, it discovers the highest-discriminating prompts for each model automatically via three phases:

1. Phase 1 — Broad Scan: Probe a pool of 52 candidate prompts (single-token openers, factual completions, code fragments, multilingual triggers, instruction-style) — 3 samples each — and rank them by output entropy
2. Phase 2 — Variant Exploration: Generate 3 variants (shortened, instruction-wrapped, reframed) for the top 8 candidates; score each variant the same way
3. Select: Combine Phase 1 + Phase 2 results and pick up to 8 prompts with the highest border scores (most diverse outputs)
4. Phase 3 — Reference Sampling: Collect 20 samples per selected prompt under temperature=0 to build the model's token-distribution fingerprint
5. Compare: Compute Total Variation Distance between observed and baseline distributions
6. Test: Permutation test for statistical significance
7. Conclude: model_match / model_mismatch / inconclusive with confidence score

Fingerprint answers "is this the right model?" by comparing against a known baseline. It's fully independent from drift and requires no logprob access.

RelayRadar includes a built-in fingerprint library for common relay-facing models:

You can use these baselines out of the box, or build your own fingerprints for private models and relay-specific aliases.

{
  "name": "Production Relay",
  "baseUrl": "https://relay.example.com/v1",
  "declaredModel": "gpt-4o",
  "providerTag": "openai",
  "apiKeyEnv": "OPENAI_API_KEY",
  "passthroughAuth": false,
  "enabled": true
}

Create/update endpoints via Console UI or /rr/endpoints APIs. Endpoint state is persisted in SQLite.

{
  "requiredRedactionFields": ["email", "phone", "api_key", "token", "id_number"],
  "manualRedactionStrings": [],
  "blockOnHighRiskResponse": true,
  "privacyFilterEnabled": false,
  "privacyFilterThreshold": 0.85,
  "sentinelEnabled": true,
  "sentinelIntervalMinutes": 30,
  "sentinelPromptsPerCycle": 3,
  "fingerprintAuditEnabled": true,
  "fingerprintAuditDriftThreshold": 0.72,
  "fingerprintAuditCooldownMinutes": 5
}

POST /v1/chat/completions    # Chat (streaming & non-streaming)
POST /v1/responses           # Responses API
GET  /v1/models              # List models

GET  /rr/health              # Health check
GET  /rr/overview            # All endpoints with status and metrics
POST /rr/models/list         # Discover upstream OpenAI-compatible models

GET    /rr/endpoints              # List endpoints
POST   /rr/endpoints              # Create endpoint
PUT    /rr/endpoints/:id          # Update endpoint
DELETE /rr/endpoints/:id          # Delete endpoint
POST   /rr/endpoints/:id/default  # Set default
POST   /rr/endpoints/:id/fingerprint-baseline # Set comparison baseline
GET    /rr/endpoints/:id          # Detail with metrics

GET  /rr/fingerprint/:id           # Portrait + audit history
POST /rr/fingerprint/:id/audit     # Trigger audit
GET  /rr/fingerprint/baselines     # List baselines
POST /rr/fingerprint/baselines/run # Build new baseline
POST /rr/fingerprint/baselines/:id/preferred # Mark preferred baseline
DELETE /rr/fingerprint/baselines/:id # Delete baseline
GET  /rr/fingerprint/models        # List model definitions
POST /rr/fingerprint/models        # Create model definition
PUT  /rr/fingerprint/models/:id    # Update model definition
DELETE /rr/fingerprint/models/:id  # Delete model definition
POST /rr/fingerprint/catalog/build # Batch catalog build

GET  /rr/shield          # Shield statistics
GET  /rr/events          # Risk event log (?endpointId=&limit=)
GET  /rr/policy          # Current policy
PUT  /rr/policy          # Update policy
POST /rr/reload          # Reload policy config from disk
GET  /rr/sentinel/:id    # Endpoint sentinel history
POST /rr/sentinel/run    # Manual sentinel cycle
POST /rr/sentinel/:id/run # Manual sentinel run for one endpoint

relayradar/
├── assets/                # README images and brand assets
├── apps/
│   ├── proxy/             # TypeScript + Fastify + SQLite
│   │   └── src/
│   │       ├── proxy/     # OpenAI-compatible request handling
│   │       ├── shield/    # Redaction engine
│   │       ├── sentinel/  # Drift probes
│   │       ├── fingerprint/ # AB3IT model identity
│   │       └── admin/     # Management API
│   └── console/           # React 19 + Vite 7
│       └── src/
│           ├── pages/     # 5 page components
│           └── components/  # Reusable UI
├── packages/
│   └── shared/            # TypeScript types
├── config/                # endpoints.json, policy.json
└── data/                  # Built-in fingerprint catalog and local runtime data

No extra observability infrastructure required. No Prometheus, no Redis, no cloud database. You only need the upstream LLM endpoint and credentials you want RelayRadar to monitor.

If you use RelayRadar in research or public reports, please cite:

@software{relayradar2026,
  title = {RelayRadar: Towards Transparent and Trustworthy Black-Box LLM Service Access},
  author = {{Shanghai Artificial Intelligence Laboratory}},
  year = {2026},
  url = {https://github.com/AI45Lab/RelayRadar}
}

BUSL-1.1 — source-available for non-production use. See LICENSE for details.