Purpose: A small knowledge base MPC backend that extracts content from websites and files, embeds text, and stores searchable chunks for retrieval.

Ingestion: Downloads websites, extracts titles, partitions content into markdown elements, cleans and groups paragraphs, and chunks content by title for meaningful units.

Embeddings: Uses fastembed.TextEmbedding to produce vector embeddings for each chunk.

Storage: Persists Document and DocumentChunk records to a backing Postgres database utilizing the PGVector extension (via the project's DB layer).

Tech stack: Python, httpx, unstructured (partitioning/cleaning), fastembed (embeddings), Postgres, Docker for local deployment.

Deployment / quick run: Make sure the folder postgres_data exists. Start the stack with Docker Compose:

docker compose up

Who it's for: Useful as a lightweight knowledge‑base ingestion pipeline for building vector search or RAG systems.

See ENDPOINTS.md