The unified search engine for AI — connect it, run
xerj autoindex, and it works: any folder
becomes typed, queryable indices — zero configuration.
Give agents long-term memory, semantic search
& RAG, hybrid BM25 + vector
retrieval, auto-embedding on ingest, and
anomaly detection — no bolt-on vector database, no
glue code. One binary that speaks the Elasticsearch wire protocol,
released under Apache‑2.0: no lock-in, no SSPL,
no per-feature gates. Yours to run, fork, and build on.
Five LLM models tracked in real time. P95 latency, 60-second window. Ribbon depth encodes model generation. This is one XERJ query — not five dashboards stitched together.
The fastest way to make data useful to an AI agent is to not write a
pipeline at all. xerj autoindex <folder>
— a subcommand of the same binary — walks the tree, sniffs every file's
format by content (extensions are never trusted) across
13 format families — JSONL, JSON, dialect-sniffed CSV, logs, SQL dumps,
SQLite, PDF, DOCX, HTML, XML, YAML, plain text, gzip — infers field
types and date encodings from the data itself, writes explicit mappings,
and streams everything in with idempotent IDs. Junk files are recorded,
never fatal. It ends by writing a catalog index — the data map — so the
agent's first question, "what is even in here?", is answered by
the engine. Real captured run:
$ xerj autoindex ./sample autoindex: 4 files (0 MB) under /tmp/xerj-pubpass/sample phase A: sniffing + sampling 4 files… phase A: 3 datasets inferred, 1 junk/skipped files phase B: indexing 3 files with 8 workers → http://localhost:9280 done in 0.2s — 3 datasets, 5801 records live, 0 junk records, 1 junk/skipped files ax-logs 5000 docs ax-exports 800 docs ax-docs 1 docs next: `xerj autoindex map --url http://localhost:9280` for the data map; search via GET /ax-*/_search
Honesty first: in a controlled 10-question exam on that 518 MB corpus, a XERJ-backed agent scored 9 correct + 1 partial vs a fair grep/python baseline's 10/10 — a tie on accuracy, not a rout. What XERJ wins is structural: a full corpus inventory in 4 API calls, sub-second aggregations over millions of rows, and uniform access to SQLite, DOCX, gzip, and decimal-comma CSV through one API — the advantages that matter at scale, over remote/API-only access, and under repeated querying. The pipeline is streaming and resumable, verified on multi-GB corpora.
Every operation an agent needs is a plain HTTP call on :9200 — no SDK, no signup, no external embedding key. Store what an agent learns, recall it later by meaning (the built-in embedder is lexical hash-based — honest hybrid lexical+vector retrieval, not neural understanding). Each namespace is physically isolated, so agents never read each other's memories. Real run against an empty XERJ:
$ curl -sXPOST localhost:9200/_memory/agent-demo \
-H 'content-type: application/json' \
-d '{"text":"The user prefers metric units and a dark UI theme.","metadata":{"kind":"preference"}}'
{"created":true,"id":"77eff57b-e432-431c-8b49-8a16b33ab551","namespace":"agent-demo"}
$ curl -sXPOST localhost:9200/_memory/agent-demo/_recall \
-H 'content-type: application/json' \
-d '{"query":"what display settings does the user like?","semantic":true,"k":1}'
{"hits":[{"id":"77eff57b-e432-431c-8b49-8a16b33ab551",
"score":0.655571460723877,
"text":"The user prefers metric units and a dark UI theme."}], // metadata trimmed
"namespace":"agent-demo"}
xerj autoindex <folder> — sniffed formats, inferred mappings, a self-describing data map, resumable idempotent ingest.
The /_memory REST API — store, recall by meaning / keyword / vector, filter, forget, per-agent isolation.
First-class documentation, each validated end-to-end against a live XERJ — RAG, hybrid search, log analytics, anomaly detection, ES migration.
Every AI application writes three kinds of data. Logs for debugging. Vectors for retrieval. Memory for context. Today, teams glue three systems together — Elasticsearch for text, Pinecone for vectors, a custom store for agent state — each with its own operator, billing line, and failure mode. XERJ collapses them into one engine because under the surface they're the same problem: find the few records that matter, fast, cheap, and now.
830K real queries clustered by intent in XERJ's embedding space. RAG retrieval dominates. Code-assist forms a tight cluster. Classify + extract-json separate cleanly — all queryable from a single index.
Hybrid search in XERJ isn't an orchestration layer stitching two systems together. BM25 and vector kNN share one query tree, one cost model, one execution pass — with proper RRF score fusion built in, not bolted on. The service graph shows real traffic: api-gateway fans into five indexes without the double-call to ES + Pinecone.
Spend shouldn't be a monthly surprise. XERJ encodes cost directly alongside traffic — a 13:00 weekday spike in model spend is visible the moment it happens, not in the next finance review. Heatmap: weekday × 2-hour windows, opacity = $/hour.
Leave a work email and the live dashboards open immediately — explore the query surface, vector search, aggregations, and agent memory on real data. Stored only in your browser: no funnel, no sequenced drip, no "journey."
The engine numbers back the use cases up — they are not the product. kNN recall was a 1.00 tie (both exact on this setup). FULL BENCHMARK METHODOLOGY + REPRODUCTION →
XERJ is written in Rust and ships as a single static binary. No JVM to tune, no sidecars to deploy, no cluster coordinator to install. The query planner, the vector index, the ingestion pipeline, the embedded Raft consensus, and the HTTP server all live in the same process.