Tequio — a production-shaped chatbot starter kit: LangGraph, FastAPI, LiteLLM, Redis, pgvector, Langfuse

Clone it, add an API key, ingest your FAQs, and ship a tenant-scoped RAG assistant behind FastAPI. The runtime gives you Redis-backed caching, three-layer memory, declarative tools, guardrails, streaming, evaluations, and optional Langfuse traces.

Tequio is the Mexican tradition of communal work: everyone contributes labor so the whole town benefits. This repository is that idea applied to production chatbot infrastructure — built in the open, MIT-licensed, meant to be taken and extended.

See architecture for the boundaries and extension plan.

Prerequisites: Docker Desktop with Compose and an OpenAI-compatible key that supports chat plus embeddings. This starts the complete local stack; no host Python installation is required for the five commands below.

1. Copy the environment file and set OPENAI_API_KEY to your key.

   cp .env.example .env

2. Build and start PostgreSQL/pgvector, Redis Stack, and FastAPI.

   make up

   To run the order-lookup command in the demo flow too, replace that command with:

   make demo

3. Create the vector extension, table, and HNSW index.

   make migrate

4. Embed and index the bundled knowledge/demo corpus.

   make ingest TENANT=demo

5. Ask a question. Use the JSON form for standard clients, or the SSE form for event-stream clients.

   curl -i http://localhost:8000/chat \
     -H 'Content-Type: application/json' \
     --data '{"tenant_id":"demo","message":"What are Acme Corp business hours?"}'

   curl -N -i http://localhost:8000/chat \
     -H 'Accept: text/event-stream' \
     -H 'Content-Type: application/json' \
     --data '{"tenant_id":"demo","message":"What are Acme Corp business hours?","stream":true}'

The API is ready when curl http://localhost:8000/healthz returns {"status":"ok",...}. If Compose reports a startup error, inspect docker compose logs api first.

Run the quickstart first. For the order lookup in step 3, start the local demo service with make demo. The following flow is copy-pasteable against the implemented API. Before step 1, set Langfuse keys in .env and restart with the same Compose command if you want the final request to appear in Langfuse.

1. Ask a knowledge-base FAQ in a named session. The JSON response has "route":"rag", a source citation in response, and the supplied session identifier. This first request is eligible for the response cache.

   SESSION_ID=00000000-0000-4000-8000-000000000001
   curl -i http://localhost:8000/chat \
     -H 'Content-Type: application/json' \
     --data "{\"tenant_id\":\"demo\",\"session_id\":\"$SESSION_ID\",\"message\":\"How long do I have to return an item?\"}"

2. Continue the session. The model receives the visible prior turn, so it can resolve that policy without restating the return-policy subject. This context-aware response deliberately reports X-Cache: miss.

   curl -i http://localhost:8000/chat \
     -H 'Content-Type: application/json' \
     --data "{\"tenant_id\":\"demo\",\"session_id\":\"$SESSION_ID\",\"message\":\"Does that policy apply to sale items too?\"}"

   Inspect the visible transcript and any summary block with:

   curl "http://localhost:8000/sessions/$SESSION_ID/history?tenant_id=demo"

3. Ask for an order. With the demo Compose profile running, the agent selects the get_order declaration in config/tools.yaml, calls the local mock API, and returns its result. Order tools are marked sensitive, so this response is never stored in the shared response cache.

   curl -i http://localhost:8000/chat \
     -H 'Content-Type: application/json' \
     --data '{"tenant_id":"demo","message":"Where is my order 12345?"}'

4. Repeat the first FAQ without session context. The response headers include X-Cache: hit-exact after the cacheable completion from step 1.

   curl -i http://localhost:8000/chat \
     -H 'Content-Type: application/json' \
     --data '{"tenant_id":"demo","message":"How long do I have to return an item?"}'

5. When LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY were set before the flow, open the Langfuse project selected by those keys. Each non-cache-hit chat request creates a chat trace with guardrails, router, retrieve, tool execution, and generate spans as applicable. See deployment.

flowchart LR
    client[Client] --> api[FastAPI /chat]
    api --> guard["Input guardrails<br/>PII redaction + injection inspection"]
    guard --> cache["Redis cache service<br/>exact → semantic"]
    cache -->|miss| route["LangGraph agent<br/>router → retrieve → tools → generate"]
    cache -->|hit| response[JSON or SSE response]
    route --> llm[LiteLLM]
    llm --> providers[Model providers]
    route --> pg[("PostgreSQL + pgvector<br/>documents + LangGraph session memory")]
    route --> redis[("Redis Stack<br/>response + retrieval cache")]
    route -. optional traces .-> langfuse[Langfuse]
    response --> client

The diagram shows the implemented LangGraph request path: a deterministic LLM router selects direct generation, RAG retrieval, or a reviewed declarative tool before generation. Session state is checkpointed in PostgreSQL, while Redis caches only context-free, cache-safe requests. Read the implementation-level architecture and rationale.

• FastAPI keeps the serving contract small: one /chat endpoint supports JSON and SSE while domain errors stay mapped to safe HTTP responses.
• LangGraph orchestrates cache misses: a deterministic router directs small talk around retrieval and knowledge questions through it.
• LiteLLM isolates providers: switch the primary or fallback model through environment variables.
• pgvector is the default retrieval store: transactional version swaps keep one active tenant corpus.
• Caching lives in the application: it can include prompt, model, tools, knowledge version, and tenant in its decision.
• Tenant IDs exist from day one: every stored document and cache key is scoped, even while demo is the default tenant.
• Evals are in V1: deterministic offline checks make safety and grounding measurable before production traffic.

V1.1: Qdrant backend, reranking, API-key auth, admin scripts.

V2: full multi-tenancy, prebuilt connectors, human handoff, simple panel, and policy engine.

• The trust model is single-tenant: tenant_id is client-supplied until the V1.1 auth layer.
• There is no rate limiting yet; it is planned as part of the roadmap.

• Expanded quickstart
• FAQ ingestion and reindexing
• Cache behavior
• Declarative tools and MCP
• Three-layer memory
• Deployment
• Contributing