docs(example): hybrid retrieval for RAG chatbots (drop-in for pgvector + BM25) - #62
Merged
Conversation
…r + BM25) Tested-live example for the common RAG shape (docs -> chunks -> embeddings -> pure cosine, often Supabase/pgvector) showing where dense-only retrieval buries exact terms and how XERJ's BM25+vector RRF hybrid fixes the retrieval half. Real result over 18 doc chunks (rank of the correct chunk, lower better): E4304: cosine #13 BM25 #1 hybrid #3 createCheckoutSession cosine #14 BM25 #1 hybrid #1 WEBHOOK_SIGNING_SECRET cosine #4 BM25 #1 hybrid #1 At #13/#14 the correct chunk never enters a RAG top-k window -> the LLM never sees it; asked "E4304" dense-only returns "E5012 gateway timeout" (adjacent but wrong). BM25 nails exact identifiers; hybrid keeps that AND concept-matching. Demonstrates the RAG payoff: retrieval decides the grounded answer (wrong chunk -> refusal/wrong; right chunk -> correct). Candid scope: XERJ owns the RETRIEVAL half (cosine+BM25+hybrid+filters); chunking, embedding-model choice, and LLM/prompt wiring are the app's job and XERJ doesn't touch them. Drop-in migration: same chunks + embeddings, swap the pgvector cosine query for an XERJ hybrid (rrf) query. Runnable scripts + honest limitations (equal-weight RRF put E4304 at #3, tune weights / add an identifier router). Co-Authored-By: Xerj Squad A <noreply@xerj.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A tested-live example for the common RAG-chatbot retrieval shape — documents → chunks → embeddings → pure cosine (often Supabase/
pgvector) — showing where dense-only retrieval fails and how XERJ fixes the retrieval half by adding BM25 and fusing with vectors (RRF).The result (real, 18 doc chunks, EmbeddingGemma)
Rank of the correct chunk (lower is better):
Pure cosine buries the chunk that defines
E4304at #13 — outside any RAG top-k window, so the LLM never sees it; asked "E4304" it returnsE5012 — gateway timeout(semantically adjacent, factually wrong). BM25 nails exact identifiers; hybrid (RRF) keeps that exactness and vector concept-matching. And it decides the grounded answer: wrong chunk → refusal/wrong; right chunk → correct.Candid scope
XERJ owns the retrieval half (cosine + BM25 + hybrid + keyword filters). Chunking, embedding-model choice, and LLM/prompt wiring are the app's job and XERJ doesn't touch them. Drop-in: keep your chunker/embedder/LLM, swap the pgvector cosine query for an XERJ
hybrid(fusion: rrf) query — same vectors.Contents
01_corpus.py,02_embed_index.py,03_retrieval_compare.py— runnable.README.md— the demo, the RRF query shape, the pgvector→XERJ migration, and honest limitations (equal-weight RRF put E4304 at Fix/autoindex text family split #3; tune weights / add an identifier router).Docs/example only — no engine code.
🤖 Generated with Claude Code