A production-ready Retrieval-Augmented Generation (RAG) application that lets you chat with your PDF documents using PostgreSQL as a vector database. Built with Streamlit for an intuitive web interface and powered by OpenAI's embeddings and chat models.

🎥 Watch the complete tutorial on YouTube - Learn how to build this application from scratch!

• 🔄 Auto-Configuration - Loads API keys and database credentials from .env files
• ✅ Real-time Validation - Tests OpenAI API and PostgreSQL connections before processing
• 📄 Smart PDF Processing - Intelligent document chunking and embedding generation
• 💬 Interactive Chat Interface - Natural conversation with your documents
• 🔍 Database Collection Manager - View, explore, and manage stored embeddings
• 📚 Source Attribution - Shows exact document sections that answer your questions
• 🛡️ Production Ready - Error handling, validation, and security best practices

graph TD
    A[PDF Upload] --> B[Text Extraction]
    B --> C[Document Chunking]
    C --> D[Generate Embeddings]
    D --> E[Store in PostgreSQL]
    E --> F[Vector Search]
    F --> G[RAG Pipeline]
    G --> H[Chat Response]
    
    I[Database Manager] --> E
    J[Connection Validator] --> K[OpenAI API]
    J --> L[PostgreSQL + pgvector]

• Python 3.8 or higher
• PostgreSQL 13+ with pgvector extension
• OpenAI API key

git clone https://github.com/ancur4u/postgres_rag.git
cd postgres_rag

pip install -r requirements.txt

-- Connect to your PostgreSQL database
CREATE EXTENSION IF NOT EXISTS vector;

-- Verify installation
SELECT * FROM pg_extension WHERE extname = 'vector';

Create a .env file in the project root:

# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key_here

# PostgreSQL Configuration
DB_HOST=localhost
DB_PORT=5432
DB_NAME=your_database_name
DB_USER=your_username
DB_PASS=your_password

streamlit run app.py

The application will be available at http://localhost:8501

1. Open the application in your browser
2. The app will auto-validate your configuration from the .env file
3. If validation fails, use the sidebar to manually enter credentials

1. Click "Upload a PDF" in the main area
2. Select your document (supports multi-page PDFs)
3. Wait for processing and embedding generation

1. Use the chat interface to ask questions
2. View responses with source citations
3. Explore conversation history

1. Use the sidebar "Database Collections" section
2. View stored embeddings and metadata
3. Delete collections when no longer needed

# In rag_utils.py - Adjust these parameters:

CHUNK_SIZE = 1000          # Text chunk size for processing
CHUNK_OVERLAP = 200        # Overlap between chunks
EMBEDDING_MODEL = "text-embedding-ada-002"  # OpenAI embedding model
CHAT_MODEL = "gpt-3.5-turbo"               # OpenAI chat model

postgres_rag/
│
├── app.py                 # Main Streamlit application
├── rag_utils.py          # RAG processing utilities
├── requirements.txt      # Python dependencies
├── .env.example         # Environment variables template
├── README.md            # This file
│
├── docs/                # Documentation
│   ├── deployment.md    # Deployment guide
│   └── troubleshooting.md # Common issues and solutions
│
└── examples/            # Example documents and notebooks
    ├── sample.pdf       # Test document
    └── demo.ipynb      # Jupyter notebook examples

# Build the image
docker build -t postgres-rag .

# Run with environment variables
docker run -p 8501:8501 --env-file .env postgres-rag

• Streamlit Cloud: Direct deployment from GitHub
• Heroku: Using the included Procfile
• AWS/GCP: Container deployment with managed PostgreSQL

See docs/deployment.md for detailed deployment instructions.

The application creates the following table structure:

-- LangChain's default pgvector table
CREATE TABLE langchain_pg_embedding (
    uuid UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    document TEXT,
    cmetadata JSONB,
    custom_id TEXT,
    embedding vector(1536)  -- OpenAI embedding dimension
);

-- Index for fast similarity search
CREATE INDEX ON langchain_pg_embedding 
USING ivfflat (embedding vector_cosine_ops);

Run the test suite:

# Install test dependencies
pip install pytest pytest-asyncio

# Run tests
pytest tests/

# Run with coverage
pytest --cov=. tests/

1. pgvector Extension Not Found

-- Install pgvector extension
CREATE EXTENSION vector;

2. OpenAI API Rate Limits

• Upgrade your OpenAI plan
• Implement rate limiting in the application

3. Memory Issues with Large PDFs

• Adjust CHUNK_SIZE parameter
• Process documents in smaller batches

See docs/troubleshooting.md for more solutions.

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

This project is licensed under the MIT License - see the LICENSE file for details.

• LangChain for the RAG framework
• pgvector for PostgreSQL vector extensions
• Streamlit for the amazing web framework
• OpenAI for powerful AI models

• 🎥 YouTube Tutorial - Complete walkthrough

🚀 If this project helped you, please give it a ⭐ star!

Made with ❤️ by Ankur Parashar