Skip to content

BhumikaSaini/PolicyPilot

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PolicyPilot

Check out a quick demo video here

RAG-powered health insurance policy comparator

Upload your insurance plan PDFs, describe what matters to you in plain English, and get a cited, priority-weighted comparison table with an AI recommendation.


How it works

  1. Upload policy PDFs (or try pre-loaded sample plans)
  2. Describe your needs in plain English (For example, "I need low overall costs and prescription medication coverage. I also travel frequently so I would like my insurance coverage to be available across states.")
  3. Review extracted criteria aligned to standard insurance terminology and assign priorities
  4. Review the plan comparison table showing Covered / Partial / Not Covered / Unknown per plan per criterion, each with a cited source passage
  5. Ask any follow-up questions in a chat interface grounded in your plan documents and comparison results

Engineering Highlights

System Architecture

System Architecture

Evaluation

  • Ragas Core Metrics — Faithfulness, Answer Relevancy, Context Precision, Context Recall.
  • Synthetic Testset Generation — Ragas TestsetGenerator creates eval dataset from policy documents (no manual labeling).
  • Quantitative Before/After Story — Metrics compared with reranking on vs. off to demonstrate measurable improvement. See eval/eval_report.ipynb.
  • W&B Weave Tracing — All eval LLM calls logged with prompts, responses, and latency. View eval dashboard →

Tech Stack

Layer Technology
LLM Groq API — llama-3.3-70b-versatile
Embeddings BAAI/bge-small-en-v1.5 (CPU, via sentence-transformers)
Reranker cross-encoder/ms-marco-MiniLM-L-6-v2 (CPU)
RAG Framework LlamaIndex
Vector Store FAISS
Backend FastAPI (Python)
Frontend Next.js + TypeScript
Evaluation Ragas + W&B Weave

Local Development

Prerequisites

Backend

cd backend
python -m venv .venv
source .venv/bin/activate
pip install -r requirements-core.txt
pip install -r requirements-eval.txt

cp .env.example .env
# Add your API keys to .env

uvicorn app.main:app --reload

Frontend

cd frontend
npm install
npm run dev

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors