ASPE — Attack Surface Prioritization Engine

Ingests raw recon (Nmap XML, httpx JSON), enriches it, scores each host on a multi-dimensional "interestingness" formula, clusters hosts semantically, and uses DeepSeek to rank the Top 50 targets with attack-chain reasoning.

For authorized security engagements only.

            ┌─────────┐   ┌──────────┐   ┌──────────────┐
 Recon ───▶ │ FastAPI │──▶│  Celery  │──▶│ Postgres +   │
 files      │  (API)  │   │ (worker) │   │ pgvector     │
            └────┬────┘   └────┬─────┘   └──────────────┘
                 │             │  ▲
                 ▼             ▼  │ cache
            ┌─────────┐   ┌────────┐   ┌────────────┐
            │ React + │   │ Redis  │   │ DeepSeek   │
            │ D3 (UI) │   │ broker │   │ /v1 (LLM)  │
            └─────────┘   └────────┘   └────────────┘

1. Ingestion (app/ingestors) — NmapIngestor, HttpxIngestor, unified Host schema, bulk COPY into Postgres.
2. Enrichment (app/enrichers) — Wappalyzer fingerprinting, async probing of 30+ sensitive paths, MaxMind GeoLite2 geo (CSV fallback).
3. Scoring (app/scorers) — Priority = 0.35·Anomaly + 0.30·Exposure + 0.20·Business + 0.15·CVE. IsolationForest anomaly, path-weighted exposure, hostname keyword business value, NVD CVE lookup.
4. Clustering (app/clustering) — sentence-transformer embeddings in pgvector, HDBSCAN clusters, UMAP 2D projection.
5. AI Orchestrator (app/orchestrators) — DeepSeekOrchestrator for Top-50 ranking, cluster naming, executive summary. Redis-cached; falls back to pure algorithmic scores if the API fails.
6. API + Frontend (app/api, frontend) — REST + WebSocket, React SPA with priority list, D3 cluster scatterplot, executive summary box.

cp .env.example .env          # add your DEEPSEEK_API_KEY
docker compose up --build

• Frontend: http://localhost:3000
• API docs: http://localhost:8000/docs

In the UI, paste your API key (default demo-secret-key-change-me), upload data/sample_httpx.json, and hit Launch Pipeline.

All require the X-API-Key header (WebSocket uses ?api_key=).

A few spec details were adjusted because they would otherwise break at runtime:

• Embedding dimension is 384, not 768. all-MiniLM-L6-v2 natively outputs 384-dim vectors; the pgvector column and config match that. Using 768 would raise a dimension-mismatch on insert. To truly get 768, swap to a model like all-mpnet-base-v2 and set EMBEDDING_DIM=768.
• HDBSCAN is imported from the hdbscan package, not sklearn.cluster (scikit-learn ships no HDBSCAN under that path in the pinned version's API used here; hdbscan is the canonical library and is in requirements).
• UMAP (umap-learn) is added for the 2D projection the scatterplot needs.
• Wappalyzer/GeoLite/NVD all degrade gracefully when unavailable so the pipeline never hard-fails on missing external resources.
• Auth is a single shared header key, intentionally minimal for the demo phase only — replace before any real exposure.