Geotechnical Engineering RAG Assistant
A locally hosted AI assistant that reads your entire engineering document library and answers questions grounded in your own files. PDFs, Word docs, spreadsheets, presentations, images, CAD files; all indexed, all searchable, all private. Everything runs on your machine. No cloud. No API keys. No data leaves your network.
- What You Need
- Step 1: Set Up llama-server
- Step 2: Start GeoRAG
- Step 3: First-Time Walkthrough
- Using the Chat
- Managing Your Documents
- Scheduled Downtime
- Backup and Restore
- Stopping the Server
- File Types That Work
- Built-In Topic Tags
- Common Issues & Fixes
- System Architecture
- Data Flow
- Processing Pipeline Deep Dive
- Chat & Retrieval Internals
- Scheduler & Downtime
- llama-server Supervisor
- Concurrency Model
- Tech Stack
- Project Structure
- API Reference
- Configuration Reference
- Database Schema
- Frontend Architecture
- Scripts Reference
- License
GeoRAG is a local AI assistant for geotechnical engineers. It reads your engineering documents (reports, calculations, specifications, test results, drawings) and lets you ask questions in natural language. The AI answers using the actual content of your files, citing exactly which documents and pages it drew from.
It's like having a colleague who has read every document in your library and can instantly recall any detail.
Key principle: Your data never leaves your computer. The AI models run locally through llama.cpp's llama-server (open source, free). There are no subscriptions, no cloud uploads, and no API costs.
| Capability | What It Means |
|---|---|
| Read your documents | Scans your Engineering/ folder and extracts text from PDFs, Word, Excel, PowerPoint, images, and more |
| OCR scanned PDFs | Automatically OCR scanned/image-based PDFs that have no embedded text, making them permanently searchable |
| Organize by topic | Automatically classifies documents into geotechnical categories (piling, tunneling, ground improvement, etc.) |
| Answer questions | Ask anything about your documents and get answers with source citations |
| Stream responses | See the AI's answer appear word by word in real time |
| Remember conversations | All chat history is saved and can be revisited later |
| Semantic document search | On the Documents page, search file contents by meaning (embedding similarity), not just filename |
| Work across tabs | Open multiple browser tabs and they stay in sync |
| Open source files | Click any cited document to open it directly on your machine |
| Schedule downtime | Define a daily window in which the local LLMs shut down to free memory, then restart automatically when the window ends |
| Back up and restore | Export your RAG data to a portable .georag archive and import it on another machine |
Your Engineering/ folder
↓
GeoRAG scans and reads every supported file
↓
Each document is split into small pieces and converted to numbers (embeddings)
↓
These are stored in a searchable database on your machine
↓
When you ask a question, GeoRAG finds the most relevant pieces
↓
The AI reads those pieces and writes an answer, citing its sources
Two pieces of software make this work:
- GeoRAG (this project), the web app that processes documents and runs the interface
llama-server(from llama.cpp), two instances run locally, one for chat (port 8001) and one for embeddings (port 8002)
| Requirement | Minimum | Recommended |
|---|---|---|
| Computer | 16 GB RAM | 32 GB+ RAM |
| Disk space | 8 GB free | 20 GB+ |
| Operating system | macOS 14+, Windows 10+, or Ubuntu 20.04+ | macOS with Apple Silicon (M1/M2/M3/M4) |
| Python | 3.10 | 3.11 to 3.13 |
| Tesseract OCR | 4.x+ | Latest (brew install tesseract on macOS) |
| llama-server | Recent build of llama.cpp, on your PATH |
Apple Silicon Metal build or CUDA build matching your GPU |
| GPU / VRAM | 8 GB VRAM, or any 16 GB Apple Silicon Mac (unified memory). Runs the Quick and Optimal Retrieval Depth levels with the default Qwen 3.5 9B model | 16 GB VRAM for Deep and Deeper, 24 GB for Ludicrous (see Retrieval Depth Presets) |
| Optional larger model | Not needed | Qwen 3.8 27B (MLX): an Apple Silicon Mac with 32 GB for Quick and Optimal, 36 GB for Deep and Deeper, 48 GB for Ludicrous (see Optional: the MLX chat model) |
Memory footprint (default model). The default chat model is Qwen 3.5 9B (a 4-bit
Q4_K_Mquant), and the three GGUF model files are only around 7 GB on disk. What decides GPU memory beyond the weights is the context window the chat model is loaded with, because it has to hold the retrieved chunks, the conversation history and the reply. GeoRAG therefore loads a different context size per VRAM tier, chosen by the Retrieval Depth level in use:
Assumed VRAM Retrieval Depth levels Context window Chat model Total with vision projector + embedding model 8 GB Quick, Optimal 32,768 tokens ~5.9 GB ~7.7 GB 16 GB Deep, Deeper 131,072 tokens ~7.5 GB ~9.3 GB 24 GB Ludicrous 262,144 tokens (the model's maximum) ~10.0 GB ~11.8 GB The memory figures are llama.cpp's own projections for this model with a q8_0 KV cache, measured on Apple Silicon; the vision projector adds about 1.1 GB and the embedding model about 0.7 GB. These are GPU-memory figures; each server also holds roughly 0.5 to 1 GB of ordinary RAM for CPU-side buffers, so the "frees about" total in the pause dialog, which counts everything actually allocated, is higher (13.6 GB with Ludicrous loaded). The 8 GB tier is a tight fit, while the 16 GB and 24 GB tiers leave generous headroom for the desktop, other applications and llama.cpp's 1 GB safety margin. The tier is an assumption, not a gate: when you pick a level, GeoRAG reloads the model with that context and asks llama.cpp whether it actually fits. If it does not, the level is refused with an error before any chat starts, and the previous context is restored. The base tier is the exception: it always runs, and on machines smaller than the minimum llama.cpp (
--fit on) moves some layers to the CPU, which is slower. Memory footprint (Qwen 3.8 27B, MLX). The optional larger model behaves differently. Its weights alone take 15 GB, nothing is preallocated, and the MLX runtime's context cache costs about 0.25 GB per 1,000 tokens (roughly 16 times llama.cpp's quantized cache for the 9B). Its tiers therefore use much smaller context budgets:
Assumed Mac (unified memory) Retrieval Depth levels Context budget Estimated need with a full context 32 GB Quick, Optimal 24,576 tokens ~22.6 GB 36 GB Deep, Deeper 40,960 tokens ~26.8 GB 48 GB Ludicrous 73,728 tokens ~35.0 GB Measured on a 36 GB M4 Max with mlx-lm 0.31: 15.0 GB loaded, 18.0 GB with a 12,000-token prompt, 21.0 GB with 24,000 tokens. macOS lets the GPU use about 28 GB on that machine, so Quick through Deeper run and Ludicrous is refused. The need is an estimate checked before a chat starts (there is no load-time verdict to read as with llama.cpp); typical use stays well below it because the context is rarely full.
GeoRAG launches and manages two llama-server processes (from llama.cpp) itself; you don't start them by hand:
| Role | Port | Default model | Alias |
|---|---|---|---|
| Chat (with vision) | 8001 | Qwen 3.5 9B + mmproj sidecar (or, when selected, Qwen 3.8 27B served by mlx_lm.server) |
qwen3.5-9B |
| Embeddings | 8002 | Nomic embed text v1.5 | nomic-embed-text-v1.5 |
The FastAPI backend (backend/services/llama_supervisor.py) spawns both processes on startup, tracks their PIDs, and can stop/restart them, whether for a manual pause from the UI or automatically during a scheduled downtime window. All you need to do is install llama-server and point GeoRAG at your model files.
Build or install from the upstream project: https://github.com/ggml-org/llama.cpp. On macOS the simplest route is brew install llama.cpp. After install, llama-server --version should print a version, and llama-server must be on your PATH (the app shells out to the llama-server binary by name).
Place GGUFs anywhere; the examples below assume ~/LLMs/. You need three files:
~/LLMs/lmstudio-community/Qwen3.5-9B-GGUF/Qwen3.5-9B-Q4_K_M.gguf
~/LLMs/lmstudio-community/Qwen3.5-9B-GGUF/mmproj-Qwen3.5-9B-BF16.gguf
~/LLMs/second-state/Nomic-embed-text-v1.5-Embedding-GGUF/nomic-embed-text-v1.5-Q8_0.gguf
The mmproj-*.gguf sidecar is the multimodal projector for the chat model. Without it, llama-server cannot caption images and backend/services/extractors/image_extractor.py will fail on PNG/JPG documents.
Both Qwen GGUFs ship together in lmstudio-community/Qwen3.5-9B-GGUF; the embedding GGUF lives in second-state/Nomic-embed-text-v1.5-Embedding-GGUF.
Point GeoRAG at your model file paths by editing LLAMA_CHAT_MODEL, LLAMA_CHAT_MMPROJ, and LLAMA_EMBED_MODEL in backend/config.py.
You don't need to run these yourself; this is what llama_supervisor.py launches automatically, shown here for reference (e.g. if you want to reproduce the exact behavior outside the app, or tune parameters in backend/config.py).
Chat server (port 8001)
llama-server \
--model ~/LLMs/lmstudio-community/Qwen3.5-9B-GGUF/Qwen3.5-9B-Q4_K_M.gguf \
--mmproj ~/LLMs/lmstudio-community/Qwen3.5-9B-GGUF/mmproj-Qwen3.5-9B-BF16.gguf \
--port 8001 \
--alias qwen3.5-9B \
-c 32768 \
-n -1 \
--no-context-shift \
--temp 0.6 \
--top-p 0.95 \
--top-k 20 \
--repeat-penalty 1.00 \
--presence-penalty 0.00 \
--fit on \
-fa on \
-ctk q8_0 \
-ctv q8_0 \
--chat-template-kwargs '{"preserve_thinking": true}'Qwen can reason ("think") before it answers, but GeoRAG turns that off for chat (CHAT_THINKING = False in backend/config.py, sent with every chat request as chat_template_kwargs: {"enable_thinking": false}): the answer is grounded in the retrieved text, and the reasoning pass mostly added waiting. With CHAT_THINKING = True the reasoning is streamed separately from the answer (reasoning_content), forwarded wrapped in <think>...</think>, and shown in the chat as a block that auto-collapses once the model finishes thinking, with a click-to-expand toggle. The reply is not limited either way: -n -1 lets the model generate until it stops by itself or the context window is full. -c is shown here for the base tier; the app sets it to 32768, 131072 or 262144 according to the Retrieval Depth level in use (see the note above).
Embedding server (port 8002)
llama-server \
--model ~/LLMs/second-state/Nomic-embed-text-v1.5-Embedding-GGUF/nomic-embed-text-v1.5-Q8_0.gguf \
--port 8002 \
--alias nomic-embed-text-v1.5 \
--embeddings \
--pooling mean \
-c 8192 \
-b 8192 \
-ub 8192 \
--rope-scaling yarn \
--rope-freq-scale 0.75 \
-fa on \
-ngl 99The --alias values must match the chat model's request_model in CHAT_MODELS and EMBEDDING_MODEL in backend/config.py. If you change one, change the other.
GeoRAG starts both servers automatically when the app launches. Watch the status pill in the top bar: it reads "Starting LLMs" while the models load, "LLMs paused" if manually paused (click to resume), and shows a warning if a server fails to start; check
data/logs/llama-chat.log/data/logs/llama-embed.logfor details. Both processes are also stopped and restarted automatically during any scheduled downtime window you configure.
On an Apple Silicon Mac, GeoRAG can serve a second, larger chat model, Qwen 3.8 27B (MLX 4-bit), in place of the 9B. You choose between them in the Model and Retrieval Depth dialog (the gear icon in the chat), and the choice is remembered across restarts. Only one chat model is loaded at a time: switching stops one server and starts the other on the same port.
MLX models cannot be loaded by llama.cpp, so this model is served by mlx_lm.server from the mlx-lm package. That runtime lives in its own virtualenv, venv-mlx/, because it needs a newer tokenizers than the chromadb pinned in requirements.txt allows.
Setup:
- Put the model folder at
~/LLMs/lmstudio-community/Qwen3.8-27B-MLX-4bit(about 15 GB), or pointMLX_CHAT_MODELat it in.env. - Run
./start.sh. When it finds the model folder on an Apple Silicon Mac it createsvenv-mlx/and installsrequirements-mlx.txt(one time, about 430 MB). To do it by hand:
python3 -m venv venv-mlx
venv-mlx/bin/pip install -r requirements-mlx.txt- Open the dialog and pick Qwen 3.8 27B. If the model folder or the runtime is missing, or the Mac does not have the memory, the card says why and the 9B stays active.
What to expect compared with the 9B (measured on a 36 GB M4 Max):
| Qwen 3.5 9B (llama.cpp) | Qwen 3.8 27B (MLX) | |
|---|---|---|
| Reading the retrieved context | about 465 tokens/s | about 130 to 175 tokens/s (a 24,000-token prompt takes 3 minutes before the first word) |
| Writing the answer | about 35 tokens/s | 21 tokens/s on a short prompt, 12 to 14 at 24,000 tokens |
| Memory | 8 to 14 GB depending on depth | 15 GB plus about 0.25 GB per 1,000 tokens in use |
| Image description during library processing | Yes | No: mlx_lm.server is text only, so images indexed while it is selected get no description |
| Reply length | Unlimited: the model answers until it stops or its context window is full | The same, except for a crash guard at 8,192 tokens (the MLX server fails on much longer replies) |
| Memory check | llama.cpp's own fit report at load | Estimate against the Mac's GPU working set, before each chat |
Process your library with the 9B selected if it contains images, then switch to the 27B for chat if you want its answers.
If the MLX server stops producing output part-way through a reply (it does not report its own failures; the request would otherwise hang), GeoRAG gives up after 180 seconds of silence, shows an error in the chat and restarts the model.
# Navigate to the project
cd /path/to/Engineering/_0RAG
# Make the scripts runnable (first time only)
chmod +x start.sh kill.sh
# Start everything
./start.shThat's it. The script will:
- Create a Python virtual environment (first run only)
- Install all dependencies (first run only)
- Set up data directories
- Fail fast if the
llama-serverbinary or any configured model file is missing - Start the web server, which launches and manages both llama-server instances itself
You'll see:
===============================
GeoRAG - Geotechnical RAG
===============================
Using Python 3.13.x
Virtual environment found.
Starting GeoRAG server (llama-servers launched by the app)...
URL: http://localhost:3000
Open http://localhost:3000 in your browser.
If you prefer to control each step:
# Install Tesseract (required for OCR of scanned PDFs)
brew install tesseract # macOS
# or: sudo apt install tesseract-ocr # Ubuntu/Debian
# Create and activate a Python environment
python3.11 -m venv venv
source venv/bin/activate # macOS/Linux
# or: venv\Scripts\activate # Windows
# Install dependencies
pip install --upgrade pip
pip install -r requirements.txt
# Create data folders
mkdir -p data/{chroma,cache,logs}
# Start the server
uvicorn backend.app:app --host 0.0.0.0 --port 3000 --reloadWhen you first open GeoRAG, the Library Panel on the right side walks you through setup:
-
Scan: Click the scan button. GeoRAG discovers all supported files in your
Engineering/folder. You'll see a count of files found, broken down by type. -
Process: Click "Start Processing". GeoRAG will:
- Extract text from every document
- Auto-classify documents into topic categories
- Split text into searchable chunks
- Generate embeddings for similarity search
- This takes time; the progress bar shows where you are, and you can see which file is being processed at any moment.
-
OCR (if needed): After processing, some scanned/image-based PDFs may fail because they have no embedded text. Click "OCR Failed Files (N)" in the Library Panel. GeoRAG will use Tesseract to add a text layer to each PDF, overwriting the original so it becomes permanently searchable. After OCR completes, click "Process New Files" to run the normal pipeline on the now-readable PDFs.
-
Chat: Once processing finishes, go to the Chat page, select one or more topic tags, and start asking questions.
Processing speed depends on your hardware and the number of files. The embedding llama-server (port 8002) is the bottleneck. You can stop and resume processing at any time; already processed files won't be redone.
- Select topics first: On the welcome screen, click one or more topic tags (e.g., "Piling", "Deep Excavation"). This tells GeoRAG which document collections to search.
- Ask anything: Type your question and press Send. The AI will find relevant document sections and write an answer.
- Check the sources: Below each answer, you'll see which documents were cited, with page numbers and relevance scores. Click a source to open the original file.
- Choose the model and depth: Click the gear icon next to the tag bar. The dialog has two parts. Chat model switches between Qwen 3.5 9B and, on an Apple Silicon Mac with it installed, Qwen 3.8 27B (see Optional: the MLX chat model). Retrieval depth controls how many document chunks the AI considers:
| Depth Level | Speed | Detail | Good For | Assumes (9B) | Assumes (27B MLX) |
|---|---|---|---|---|---|
| Quick and less demanding | Fastest | Brief answers | Simple factual lookups | 8 GB VRAM | 32 GB Mac |
| Optimal and balanced | Balanced | Good detail | Most questions (default) | 8 GB VRAM | 32 GB Mac |
| Deep and demanding | Slower | Thorough | Multi-document analysis | 16 GB VRAM | 36 GB Mac |
| Deeper and very demanding | Slow | Very detailed | Complex technical questions | 16 GB VRAM | 36 GB Mac |
| Ludicrous | Slowest | Maximum context | When you need everything | 24 GB VRAM | 48 GB Mac |
Deeper levels need a larger context window and so more memory. Picking a model, or a level in a higher tier, reloads the chat model if needed (a few seconds) and checks that it fits; if your machine does not have the memory, the dialog shows an error and stays on the previous choice. When you switch model, your depth level is kept if the new model can run it here; otherwise the dialog falls back to Optimal and tells you why. The exact limits per level are listed under Retrieval Depth Presets.
- Conversations are saved: Use the sidebar on the left to switch between past conversations.
- Trash and restore: Delete a conversation and it goes to trash. You can restore it or permanently delete it.
Go to the Documents page (/documents) to:
- Search: Find files by name, or tick Semantic Search and press Enter to search file contents by meaning (the query is embedded and matched against the vector store), so you find relevant documents even when the filename doesn't mention the term.
- Filter: Narrow by file type (PDF, Word, Excel, etc.), processing status, or topic tag
- Reprocess or skip: Select files and click Mark as New to queue them for reprocessing, or Mark as Skipped to exclude them from processing.
- Tag files: Click a file to manage its tags, or select multiple files for batch tagging. The stats bar and tag filter counts update live after every tag change.
- Manage tags: Click "Manage Tags" to create, edit, or delete topic tags. You can change a tag's display name, color, and description. The internal slug is immutable once created. Deleting a tag removes it from all files and deletes its vector collection. Tag creation and deletion are disabled while processing is running.
- Open files: Click the filename to open it directly in your system's default application. Use Open Folder from the Manage File dialog to reveal it in your file manager.
- See stats: The top bar shows total file counts and processing status, refreshed automatically when tags are assigned or removed
The local LLM models hold GPU/unified memory while the app runs. If you want your machine free during working hours, define a daily downtime window in the system settings (the gear/status area in the top bar).
- During the window, both llama-servers are shut down to free GPU/unified memory.
- When the window ends, the servers are restarted automatically.
- If you restart the app in the middle of the window, the servers stay down until the window closes.
- Click End downtime now to override the current window immediately (the saved schedule still applies tomorrow).
- The window can wrap past midnight (e.g.
22:00to06:00).
The status pill in the top bar reflects all of this: Starting LLMs, LLMs idle, LLMs paused, Scheduled pause, and so on. You can also manually pause the LLMs (to free memory) by clicking the pill; GeoRAG refuses if any LLM work is currently active. The pause confirmation lists what is loaded (the chat model with its current context window, and the embedding model) and how much memory pausing will free, read from llama.cpp's load logs.
From the system menu you can export your data to a single portable .georag archive and import it on another machine.
- Export: Exports include the RAG data (document metadata, tags, vector collections). The export runs in the background; download the archive when it completes.
- Import: Upload a
.georagarchive. It is validated first, then imported in the background. Older archives that contain wiki data are still accepted; the wiki parts are ignored. - Export and import are mutually exclusive with each other and with document processing; GeoRAG returns a busy error rather than corrupting state.
# Option 1: Press Ctrl+C in the terminal where start.sh is running
# Option 2: Run the kill script
./kill.sh| Type | Formats | What Gets Extracted |
|---|---|---|
| Documents | Full text with page boundaries. Scanned/image-based PDFs can be OCR'd via the Library Panel to add a searchable text layer. | |
| Word (.doc, .docx) | Paragraphs and tables | |
| Excel (.xls, .xlsx, .xlsm) | All sheets with sheet names | |
| PowerPoint (.ppt, .pptx) | All slides with text and tables | |
| Images | PNG, JPG, GIF, BMP, TIFF | AI generated description of image content (max 10 MB) |
| Text files | TXT, Markdown, CSV, HTML, RTF | Direct text content |
| CAD | DWG, DXF | Indexed by filename only (no content extraction yet) |
| Video | MP4, AVI, MOV, MKV | Discovered but not content extracted |
GeoRAG seeds 34 built-in geotechnical categories on first launch (defined in DEFAULT_TAGS in backend/config.py). Documents are automatically classified into these during processing. The core categories include:
| Tag | Focus |
|---|---|
| Piling | Pile foundations, driven/bored piles, load tests |
| Diaphragm Wall | Diaphragm/slurry walls, barrettes |
| Ground Improvement | Compaction, grouting, soil mixing |
| Ground Anchors | Anchors, soil nails, rock bolts, tiebacks |
| Deep Excavation | Shoring, braced cuts, sheet piling, cofferdams |
| Tunneling | Tunnel design, TBM, NATM, cut and cover |
| Earthquake Engineering | Seismic design, liquefaction, dynamic analysis |
| Lab Testing | Triaxial, consolidation, direct shear |
| In-Situ Testing | SPT, CPT, pressuremeter, plate load |
| FEA / Numerical | PLAXIS, finite element / numerical modelling |
...plus categories such as Grouting, Soil Nails, Slope Stability, Shallow Foundations, Soil Mechanics, Geosynthetics, Energy Piles, Transportation Geotechnics, Project Documents, Conference Proceedings, DWG Drawing Templates, and more. See DEFAULT_TAGS for the full list.
You can also create your own custom tags through the Documents page (Manage Tags) or the API, and use Explore New Tags to have the LLM discover new categories in your library.
- Make sure Tesseract is installed:
brew install tesseract(macOS) orapt install tesseract-ocr(Linux) - Make sure
ocrmypdfis installed:pip install ocrmypdf - The button only appears when there are failed PDF files. Process your library first; scanned PDFs that fail text extraction will show up as candidates for OCR.
- Some PDFs may be corrupted or password-protected; OCR will skip those and continue to the next file
- Check the error list in the Library Panel for details on individual failures
- For non-English documents, add the language code to
OCR_LANGUAGESinbackend/config.py(e.g.,["eng", "tur"]for English + Turkish). You may also need to install the Tesseract language pack (e.g.,brew install tesseract-lang).
GeoRAG auto-launches both llama-server instances on startup; this means one or both failed to come up. Check:
llama-serveris installed and on yourPATH(llama-server --version)- The model files exist at the paths configured in
backend/config.py(LLAMA_CHAT_MODEL,LLAMA_CHAT_MMPROJ,LLAMA_EMBED_MODEL) data/logs/llama-chat.loganddata/logs/llama-embed.logfor the actual startup error
Once running, verify with:
curl http://127.0.0.1:8001/v1/models # chat
curl http://127.0.0.1:8002/v1/models # embeddings- Check
data/logs/llama-chat.logfor errors from the chat llama-server (port 8001) - Make sure
--alias qwen3.5-9BmatchesCHAT_MODELinbackend/config.py; if you changed the model, update bothLLAMA_CHAT_MODEL/--aliasandCHAT_MODELto match
- Make sure the embedding llama-server (port 8002) is running with
--embeddingsand--alias nomic-embed-text-v1.5 - The chat server alone is not sufficient; both processes must be up
- The chat llama-server must be started with
--mmproj <path-to-mmproj.gguf>. Without it, llama-server cannot accept images andimage_extractor.pywill mark image files as failed. - Image description is not available while the MLX chat model (Qwen 3.8 27B) is selected:
mlx_lm.serveris text only. Select Qwen 3.5 9B before processing a library that contains images.
The local llama-servers are doing all the AI work. To speed things up:
- Pass
-ngl 99(or as many layers as fit) to push the model onto your GPU - Use a smaller/faster chat GGUF (e.g. a smaller Qwen quantization)
- Close other apps competing for GPU/RAM
- Files must be inside the
Engineering/parent directory (not inside_0RAG/itself) - The file extension must be supported (see the table above)
- Folders named
.git,__pycache__,node_modules, andvenvare skipped
lsof -i :3000 # See what's using it
./kill.sh # Or just kill all GeoRAG processesIf you reinstall or upgrade Python, the existing venv/ will still reference the old Python binary and you'll see errors like cannot execute: required file not found. Delete the venv and let start.sh recreate it:
rm -rf venv/
./start.shrm -rf data/ # Deletes all processed data (your source documents are safe)
./start.sh # Re-creates everything from scratch+------------------------------------------------------------+
| Browser |
| |
| +-----------+ +-------------+ +----------------+ |
| | Chat Page | | Documents | | Library Panel | |
| | (chat.js) | | (docs.js) | | (panel.js) | |
| +-----+-----+ +------+------+ +-------+--------+ |
| | | | |
| | BroadcastChannel (cross-tab) | |
+--------+----------------+-----------------+----------------+
| | |
| SSE/HTTP | HTTP | HTTP (poll)
v v v
+------------------------------------------------------------+
| FastAPI Backend (:3000) |
| |
| +--------+ +---------+ +----------+ +--------+ |
| | Chat | | Docs | |Processing| | Tags | |
| | Router | | Router | | Router | | Router | |
| +---+----+ +----+----+ +----+-----+ +---+----+ |
| | | | | |
| +----------------------------------------------+ |
| | Services Layer | |
| | | |
| | +----------+ +---------+ +--------------+ | |
| | | LLM | |Embedding| | Document | | |
| | | Client | | Client | | Processor | | |
| | | (chat + | | | |(orchestrator)| | |
| | | vision) | | | +------+-------+ | |
| | +----+-----+ +----+----+ +-----+------+ | |
| | | | | Scanner | | |
| | | | | Chunker | | |
| | | | | Tagger | | |
| | | | | Extractors | | |
| | | | +------------+ | |
| +----------------------------------------------+ |
| | | |
+----------+-------------+-----------------------------------+
| |
v v
+--------------------------------------+ +---------------------------+
| llama-server (chat, :8001) | | Data Storage |
| +--------------------------------+ | | |
| | /v1/chat/completions | | | +---------------------+ |
| | (qwen3.5-9B + mmproj vision) | | | | ChromaDB | |
| +--------------------------------+ | | | (vectors) | |
+--------------------------------------+ | +---------------------+ |
+--------------------------------------+ | | SQLite | |
| llama-server (embeddings, :8002) | | | (metadata) | |
| +--------------------------------+ | | +---------------------+ |
| | /v1/embeddings | | +---------------------------+
| | (nomic-embed-text-v1.5) | |
| +--------------------------------+ |
+--------------------------------------+
Three tier layout:
- Browser: Vanilla JS frontend served as static files. Chat page, documents page, library panel, and a system status pill. Tabs sync via BroadcastChannel API.
- FastAPI Backend (port 3000): Python async server handling API requests, orchestrating the RAG pipeline, and streaming responses via SSE.
- Two
llama-serverinstances (ports 8001 / 8002): Local LLM runtimes exposing OpenAI-compatible APIs. The chat server runs the chat model plus its multimodal projector for image captioning; the embedding server runs the embedding model.
Storage:
- ChromaDB: On disk vector database. One collection per topic tag, cosine distance metric, 768 dim vectors.
- SQLite: File metadata, tag associations, conversations, messages, source citations, and the single-row app settings (downtime schedule). Runs in WAL mode.
Engineering/ directory
→ Scanner: walks filesystem, registers files in SQLite (status=new)
→ Extractors: format specific text extraction (10 files in parallel)
→ Tagger: heuristic keyword matching, then LLM fallback for unmatched files
→ Chunker: RecursiveCharacterTextSplitter (1000 tokens, 200 overlap)
→ Embedding Client: batch embed via embedding llama-server /v1/embeddings
→ Vector Store: upsert into ChromaDB per tag collections
→ SQLite: update status=processed, save chunk count + text preview
User message + selected tags
→ Embed query via embedding llama-server /v1/embeddings
→ Query each selected tag's ChromaDB collection (top K per tag)
→ Deduplicate across tags, rank by cosine similarity
→ Assemble system prompt with the top document chunks
→ Append last 4 conversation turns (8 messages)
→ Stream completion from chat llama-server /v1/chat/completions
→ Deliver tokens to browser via SSE
→ Save message + source citations to SQLite
When you click "Start Processing", the system runs a six phase pipeline with managed concurrency:
Walks Engineering/ recursively. Skips _0RAG/, .git/, __pycache__/, node_modules/, venv/. Registers each discovered file in SQLite with status=new. Detects files removed from disk and cleans them from both SQLite and ChromaDB.
10 files extracted in parallel. Each format has a dedicated extractor:
| Format | Library | Strategy |
|---|---|---|
| pdfplumber | Per page extraction with [Page N] markers |
|
| DOCX | python-docx | Paragraph text + table cell contents |
| XLSX | openpyxl | All sheets with [Sheet: name] markers |
| PPTX | python-pptx | Per slide text with [Slide N] markers + tables |
| Images | llama-server vision (--mmproj) |
Base64 encoded image sent to chat model for description (max 10 MB) |
| Text/HTML/RTF/CSV/MD | Built in | Direct read with HTML tag stripping; UTF 8 to latin 1 fallback |
| DWG/DXF | N/A | Filename indexed only, no content extraction |
Failed extractions mark the file as status=failed and log the error. For PDFs, this typically means the file is a scanned image with no embedded text layer. These can be recovered via the OCR pipeline (see below).
An optional step triggered from the Library Panel after initial processing. Targets only PDFs with status=failed:
- Queries all files where
scan_status=failedandextension=pdf - For each file, runs
ocrmypdf(Tesseract) in a background thread viaasyncio.to_thread() - OCR writes to a temporary file, then
shutil.move()replaces the original (no corruption on failure) - Uses
skip_text=Trueto handle mixed PDFs (some pages already have text) - On success: resets
scan_status=new, clearsextracted_text_previewandchunk_count - On failure: file stays
failed, error logged, pipeline continues to next file
After OCR completes, the user clicks "Process New Files" to run the normal pipeline on the now-readable PDFs.
Mutual exclusion: OCR, processing, and tag exploration all block each other. Only one can run at a time.
Two pass classification:
-
Heuristic pass: 94 keyword patterns matched against folder paths and filenames. Maps patterns like "pile", "bored pile", "diaphragm", "grouting", "plaxis" to tag categories. Confidence: 0.7. Instant, no LLM call.
-
LLM fallback: Files with no heuristic match are batched (up to 10) and sent to the chat model with the first 1500 characters of text. The LLM returns a JSON array of tag classifications. Confidence: 0.8. Falls back to individual calls if batch parsing fails.
Tags are saved to the FileTag junction table with source=auto or source=folder_hint.
Text is split using RecursiveCharacterTextSplitter from langchain:
- Chunk size: 1000 tokens
- Chunk overlap: 200 tokens
- Separator priority:
\n\nthen\nthen.thenthen""
Page/slide/sheet metadata is extracted via regex from markers embedded during extraction and attached to each chunk's metadata.
Chunks are batched (up to 128 per request) and sent to the embedding llama-server's /v1/embeddings endpoint using the nomic-embed-text-v1.5 model. Three concurrent embedding requests are allowed (semaphore controlled). Each chunk becomes a 768 dimensional float vector.
Chunks are upserted into ChromaDB:
- One collection per tag, named
tag_{tag_name}, cosine distance metric - Chunk IDs:
file_path::chunk_N(enables idempotent reprocessing via upsert) - Stored per chunk: embedding vector, chunk text, metadata (
file_path,filename,file_type,parent_dir,page,chunk_index) - Batch size: 5000 chunks per upsert call
- Files with multiple tags are stored in multiple collections
When a user sends a message via POST /api/chat:
- The request's Retrieval Depth level is resolved for the chat model in use and the chat server is checked against it (
llama_supervisor.ensure_chat_ctx): with llama.cpp the model is reloaded first if the level needs a larger context, with MLX the level's memory estimate is checked. If it does not fit in memory the request ends with anerrorevent before anything else happens - The user's query is embedded using the same Nomic model
- For each selected tag, the corresponding ChromaDB collection is queried for the level's
top_kmost similar chunks - Results from all tags are merged and deduplicated by chunk ID
- The level's top
max_chunksare selected by relevance score - A system prompt is built containing the context chunks with source metadata
- The last
CONVERSATION_HISTORY_TURNSturns (default: 4 turns = 8 messages) are appended, newest first, for as long as they fit: half of the level's context window is kept free for the reply, and older messages that would push the prompt past the other half are dropped - The full prompt is streamed to the chat llama-server via
/v1/chat/completionswithstream=true, thinking switched off (unlessCHAT_THINKINGis on) and nomax_tokens, so the model answers for as long as it wants (the MLX server is the exception: it is given what is left of the level's context budget, capped by its crash guard) - Tokens are forwarded to the client as SSE events (
event: token,data: {"token": "..."}). WithCHAT_THINKINGon, reasoning tokens arrive first, wrapped in<think>...</think>, followed by the answer - If the reply ends because the context window filled up, or on MLX at the crash guard (
finish_reason: "length"), a visible "response cut off" note is appended so a truncated reply is never mistaken for a complete one - On completion, a
doneevent sends{conversation_id, sources}and the message is persisted
Replies stored while thinking was on keep their <think> block for display, but it is stripped from the conversation history sent back to the model on later turns.
Stream mirroring: Other browser tabs can connect to GET /api/chat/stream-mirror to receive the same token stream in real time via async queues.
Cancellation: POST /api/chat/stop sets a cancellation flag that the streaming loop checks between tokens.
The levels are defined in backend/config.py as CHAT_DEPTH_LEVELS, each naming a memory tier (base, mid, top). What a tier means, its context window and the memory it assumes, is defined per chat model in CHAT_MODELS. The chat UI loads the result for the active model from GET /api/system/chat-depths.
Retrieval is the same for both models:
| Level | Tier | Top K Per Tag | Max Context Chunks |
|---|---|---|---|
| Quick and less demanding | base | 5 | 8 |
| Optimal and balanced | base | 10 | 16 |
| Deep and demanding | mid | 20 | 32 |
| Deeper and very demanding | mid | 50 | 80 |
| Ludicrous | top | 100 | 200 |
| Level | Context Window, Qwen 3.5 9B | Assumed VRAM | Context Budget, Qwen 3.8 27B (MLX) | Assumed Mac |
|---|---|---|---|---|
| Quick and less demanding | 32,768 | 8 GB | 24,576 | 32 GB |
| Optimal and balanced | 32,768 | 8 GB | 24,576 | 32 GB |
| Deep and demanding | 131,072 | 16 GB | 40,960 | 36 GB |
| Deeper and very demanding | 131,072 | 16 GB | 40,960 | 36 GB |
| Ludicrous | 262,144 | 24 GB | 73,728 | 48 GB |
- No thinking, no reply limit. The model answers directly (
CHAT_THINKING = False) and for as long as it wants. A reply ends when the model stops by itself, when you press Stop, or when the context window is full, in which case it ends with a visible "response cut off" note and you can send "continue". The context window is shared by the prompt and the reply, so half of it is kept free for the reply when the prompt is assembled (earlier messages are dropped first). - Why thinking is off. Measured on a 36 GB M4 Max: for a two-sentence question the 9B reasoned for 3,537 tokens (76 seconds) before answering, against 0.6 seconds to the first word with thinking off; on a 50-source question the 27B reasoned for about 7,700 tokens (over 9 minutes) and reached its crash guard without writing an answer. Set
CHAT_THINKING = Trueto bring reasoning back; it is then unlimited. - One exception on MLX.
mlx_lm.server0.31 fails with a Metal "Resource limit exceeded" error roughly 10,000 tokens into a reply, losing the reply and leaving the request hanging. The 27B therefore has a crash guard, not a budget: a reply stops at 8,192 tokens (reply_guard_tokensinCHAT_MODELS; set it toNoneto remove it). If the server does fail, GeoRAG reports it after 180 seconds of silence and restarts the model. - Context Window / Budget is what costs memory; see What You Need for the measured figures. For the 9B it is the
-cllama-server is loaded with. For the 27B nothing is preallocated, so it is a budget: the prompt and reply together are kept inside it, and the memory it would take when full is what gets checked. The 27B's budgets are far smaller because its context cache costs about 16 times more memory per token.
Switching levels (llama.cpp). Selecting a level calls POST /api/system/chat-depth. If the loaded context is already large enough nothing is restarted. Otherwise the chat server is relaunched with the level's context (a few seconds) and llama.cpp's --fit report is read from data/logs/llama-chat.log: llama.cpp never refuses an oversized context outright, it logs that it cannot meet its memory target and moves model layers to the CPU, and that log line is what GeoRAG treats as "does not fit". The level is then refused with llama.cpp's numbers, and the server is put back on its previous context. A process that fails to load at all is handled the same way. The context only grows while the app is running (going back to a lower level does not reload the model), and the server is never restarted while other LLM work (library processing, OCR, tag exploration, a streaming chat) is in progress.
Switching levels (MLX). mlx_lm.server allocates on demand, so there is nothing to reload and no load-time verdict to read: a context that is too big would only show up mid-answer, as swapping or an out-of-memory failure. Instead the level's need is estimated (llama_supervisor.mlx_memory_estimate: the weights on disk, plus kv_bytes_per_token for every token of the context budget, plus overhead_bytes) and compared with the memory macOS lets the GPU use (max_recommended_working_set_size, as reported by MLX, less what the embedding server holds). A level that does not fit is refused with those numbers.
Switching models. POST /api/system/chat-model stops the chat server and starts the other model's backend at its base tier. If the new model cannot be served on this machine (not an Apple Silicon Mac, runtime or model folder missing, not enough memory, or it fails to load) the previous model is restored and the reason returned. The depth level sent with the request is carried over when the new model can run it; otherwise the response falls back to the default level and says why.
The same checks run at the start of every chat request, so a level remembered by the browser but no longer runnable is caught before the chat starts. The model in use is stored in data/chat-model and the context of the level last selected in data/llama-chat.ctx; both are used the next time the chat server starts.
backend/services/scheduler.py runs a periodic reconcile loop (every 30s) that drives the daily downtime window. Settings live in a single-row app_settings table (id=1).
- Downtime math: a window is
downtime_start/downtime_end(localHH:MM, default 06:30 to 09:30), enabled byschedule_enabled. Windows may wrap past midnight. - Entering downtime: both llama-servers are stopped (including a sweep for orphaned servers left by a
uvicorn --reload). - Exiting downtime: when the window ends, both llama-servers are restarted.
- Overrides:
POST /api/system/end-downtimeforces uptime until the current window's end. A manual pause latches a user-paused flag that the scheduler honours so it won't undo the pause on the next tick. - The reconcile loop is idempotent, so it self-heals if any single transition didn't fully take.
backend/services/llama_supervisor.py owns the two model-server subprocesses so the app (and scheduler) can stop/start them: the embedding llama-server, and the chat server, which is llama-server or mlx_lm.server depending on the chat model selected. The llama-chat.* pid, log and context files belong to whichever chat backend is active.
- On app startup (
lifespaninbackend/app.py) both servers are launched unless the app started inside a downtime window. They are not stopped on app shutdown, so a fresh instance (or a--reload) can adopt them via their pid files and keep the warm KV cache. - Startup is idempotent: if a port already answers, the running process is adopted (pid discovered via the pid file or
lsof) rather than launching a duplicate. - Stops send
SIGTERM(escalating toSIGKILL), detect exit viaPopen.poll()so a stop finishes in ~1s instead of waiting on a zombie, and reap orphaned servers still listening on the port. - Lifecycle state (
running,starting,paused,down) feeds the top-bar status pill;GET /api/system/statuscomposes it with the current busy operation into a single human-readable label + severity.
| Resource | Limit | Mechanism | Rationale |
|---|---|---|---|
| File extraction | 10 parallel | asyncio.gather batch |
I/O bound, benefits from parallelism |
| LLM tagging | 1 | asyncio.Semaphore(1) |
llama-server single-slot serialization |
| Embedding requests | 3 | asyncio.Semaphore(3) |
Balance throughput vs. embedding llama-server capacity |
| Database writes | 1 | asyncio.Lock |
SQLite transaction safety |
| OCR processing | 1 file | Sequential | CPU-intensive Tesseract, avoids thrashing |
| Pipeline exclusion | 1 pipeline | Mutual exclusion checks | Processing, OCR, exploration, and backup/restore block each other |
| Embedding batch size | 128 texts | Config constant | llama-server request size limit |
| Processing batch | 10 files | Config constant | Memory bounded pipeline stage |
| Package | Version | Role |
|---|---|---|
| fastapi | 0.115.6 | Async web framework with OpenAPI |
| uvicorn[standard] | 0.34.0 | ASGI server |
| sqlalchemy | 2.0.36 | ORM and database toolkit |
| aiosqlite | 0.20.0 | Async SQLite driver |
| chromadb | 0.5.23 | Vector database |
| httpx | 0.28.1 | Async HTTP client (llama-server communication) |
| pdfplumber | 0.11.4 | PDF text extraction |
| PyMuPDF | 1.25.1 | PDF support |
| python-docx | 1.1.2 | Word document parsing |
| openpyxl | 3.1.5 | Excel parsing |
| python-pptx | 1.0.2 | PowerPoint parsing |
| langchain-text-splitters | 0.3.4 | Recursive character text splitting |
| sse-starlette | 2.2.1 | Server Sent Events |
| ocrmypdf | latest | OCR scanned PDFs via Tesseract (requires system tesseract binary) |
| jinja2 | 3.1.4 | HTML templating |
| pydantic | (via FastAPI) | Data validation |
| Technology | Role |
|---|---|
| Vanilla JavaScript | No framework; direct DOM manipulation, module pattern controllers |
| PicoCSS | Minimal classless CSS framework |
| Marked.js | Markdown to HTML rendering for assistant responses |
| KaTeX | LaTeX math rendering in chat responses (loaded from CDN) |
| Inter (Google Fonts) | UI typography |
| EventSource API | SSE streaming for real time chat |
| BroadcastChannel API | Cross tab state synchronization |
| History API | Client side SPA routing |
| localStorage | Persists retrieval depth setting and panel state |
| Component | Port | Role |
|---|---|---|
| GeoRAG (Uvicorn) | 3000 | Web server + API |
| llama-server (chat) | 8001 | Local chat model + vision (OpenAI compatible API) |
| mlx_lm.server (chat, optional) | 8001 | Replaces the chat llama-server while the MLX model is selected; text only, Apple Silicon only |
| llama-server (embeddings) | 8002 | Local embedding model (OpenAI compatible API) |
| Tesseract OCR | N/A | System binary for OCR (used by ocrmypdf) |
_0RAG/
├── start.sh # macOS/Linux startup: venv, deps, server
├── start.ps1 # Windows PowerShell startup script
├── kill.sh # Find and kill all GeoRAG processes (by port + cmdline)
├── run_server.py # Standalone entry point (used by the PyInstaller EXE)
├── georag.spec # PyInstaller spec for the Windows EXE build
├── requirements.txt # Pinned Python dependencies
├── requirements-mlx.txt # MLX runtime for the optional MLX chat model (installed into venv-mlx/)
├── .gitignore # Excludes venv/, venv-mlx/, data/, __pycache__/
│
├── backend/
│ ├── app.py # FastAPI app entry point, lifespan (llama + scheduler)
│ ├── config.py # All paths, model files, launch argv, tuning params
│ │
│ ├── models/
│ │ ├── database.py # SQLAlchemy engine (WAL), session factory, migrations
│ │ ├── schemas.py # ORM: File, Tag, FileTag, Conversation, Message,
│ │ │ # AppSettings
│ │ └── pydantic_models.py # Request/response validation schemas
│ │
│ ├── routers/
│ │ ├── chat.py # POST /api/chat (SSE), stream mirror, stop
│ │ ├── conversations.py # CRUD, soft delete, trash, restore
│ │ ├── documents.py # Search (name + semantic), filter, paginate, tag, open
│ │ ├── tags.py # Tag CRUD
│ │ ├── processing.py # Scan, start/stop pipeline, onboarding
│ │ ├── explore.py # Tag exploration: discover new categories
│ │ ├── ocr.py # OCR start/stop/status for failed PDFs
│ │ ├── backup.py # Export/import .georag archives
│ │ ├── system.py # Settings, status pill, downtime, llama pause/start
│ │ └── health.py # llama-server health probe
│ │
│ └── services/
│ ├── document_processor.py # Pipeline orchestrator (batch + concurrency)
│ ├── ocr_processor.py # OCR scanned PDFs via ocrmypdf/Tesseract
│ ├── scanner.py # Filesystem walk + DB synchronization
│ ├── chunker.py # RecursiveCharacterTextSplitter wrapper
│ ├── embedding_client.py # Async batch embeddings via llama-server
│ ├── vector_store.py # ChromaDB per tag collection management
│ ├── llm_client.py # Chat completions + vision (streaming)
│ ├── tagger.py # 94 pattern heuristic + LLM classification
│ ├── scheduler.py # Downtime window reconcile loop
│ ├── llama_supervisor.py # Owns the two llama-server subprocesses
│ ├── backup_service.py # .georag export/import (exporter, importer)
│ └── extractors/
│ ├── pdf_extractor.py # pdfplumber with [Page N] markers
│ ├── docx_extractor.py # python-docx text + tables
│ ├── xlsx_extractor.py # openpyxl multi sheet
│ ├── pptx_extractor.py # python-pptx slides + tables
│ ├── image_extractor.py # Vision model via base64
│ └── text_extractor.py # TXT, HTML, RTF, CSV, MD
│
├── frontend/
│ ├── templates/
│ │ ├── base.html # Shared nav, status pill, library panel, about/settings modals
│ │ ├── spa.html # SPA shell (chat + documents in one page)
│ │ ├── index.html # Server rendered chat page
│ │ └── documents.html # Server rendered documents page
│ │
│ └── static/
│ ├── css/
│ │ ├── app.css # Brand colors, layout, components, animations
│ │ └── pico.min.css # PicoCSS framework
│ ├── js/
│ │ ├── chat.js # Chat: SSE streaming, tags, conversations, depth
│ │ ├── documents.js # Documents: search (name + semantic), filter, batch tag
│ │ ├── panel.js # Library: scan, process, OCR, explore, progress polling
│ │ ├── system_status.js # Status pill, downtime settings, llama pause/start
│ │ ├── router.js # SPA routing via History API
│ │ ├── utils.js # HTTP helpers, tag badges, sync channel
│ │ └── marked.min.js # Markdown parser (vendored)
│ └── images/ # Logo, icons, favicons
│
└── data/ # Runtime data (gitignored)
├── metadata.db # SQLite: files, tags, conversations, settings
├── chroma/ # ChromaDB: vector collections (per tag)
├── cache/ # Application cache
├── exports/ # Backup .georag archives + uploads
├── logs/ # app.log, llama-chat.log, llama-embed.log
├── chat-model # Key of the chat model in use (config.CHAT_MODELS)
├── llama-chat.pid # PID of the chat server (for adopt/stop)
├── llama-chat.ctx # Context window (or budget) of the chat server
├── llama-embed.pid # PID of the embedding llama-server
└── onboarding_dismissed # Flag file
FastAPI auto generates interactive docs at /docs (Swagger) and /redoc.
| Method | Path | Description |
|---|---|---|
POST |
/api/chat |
Stream a chat response via SSE. Body: {message, conversation_id?, tag_names?, depth?, top_k_per_tag?, max_context_chunks?}; depth is a Retrieval Depth level key (default optimal), and the last two override that level's retrieval sizes. Events: token then done, or error. |
GET |
/api/chat/streaming |
Check streaming status. Returns {streaming: bool, conversation_id: int or null}. |
POST |
/api/chat/stop |
Cancel the active stream. |
GET |
/api/chat/stream-mirror |
SSE mirror of active stream for cross tab sync. |
| Method | Path | Description |
|---|---|---|
GET |
/api/conversations |
List active conversations (newest first, limit 50). |
GET |
/api/conversations/{id} |
Full conversation with messages and sources. |
DELETE |
/api/conversations/{id} |
Soft delete (move to trash). |
DELETE |
/api/conversations/{id}/permanent |
Hard delete conversation + messages. |
POST |
/api/conversations/{id}/restore |
Restore from trash. |
GET |
/api/conversations/trash |
List trashed conversations. |
| Method | Path | Description |
|---|---|---|
GET |
/api/documents |
Paginated list. Params: search, extension, tag (__none__ for untagged), status, page, per_page (max 200). |
GET |
/api/documents/stats |
Counts by status and extension. |
POST |
/api/documents/{id}/open |
Open file in system default app. |
POST |
/api/documents/open-by-path |
Open by relative path. Body: {file_path}. |
POST |
/api/documents/{id}/tags/{tag} |
Add tag to file. |
DELETE |
/api/documents/{id}/tags/{tag} |
Remove tag from file. |
POST |
/api/documents/batch/tags/{tag} |
Batch add tag. Body: {file_ids: []}. |
DELETE |
/api/documents/batch/tags/{tag} |
Batch remove tag. Body: {file_ids: []}. |
| Method | Path | Description |
|---|---|---|
GET |
/api/tags |
All tags with live file counts. |
POST |
/api/tags |
Create tag. Body: {name, display_name, description?, color?}. |
PUT |
/api/tags/{id} |
Update tag. Body: {display_name?, description?, color?}. Slug is immutable. |
DELETE |
/api/tags/{name} |
Delete tag, all file associations, and its vector collection. |
| Method | Path | Description |
|---|---|---|
POST |
/api/processing/scan |
Scan filesystem, sync DB. Returns {files_found, files_new, files_removed}. |
POST |
/api/processing/start |
Start pipeline. Body: {tag_names?, file_ids?, reprocess?}. |
POST |
/api/processing/stop |
Gracefully stop processing. |
GET |
/api/processing/status |
Pipeline state: {is_running, total_files, processed_files, failed_files, skipped_files, current_file, errors}. |
GET |
/api/processing/onboarding-status |
Full onboarding state with phase, progress, extension breakdown, and OCR status. |
POST |
/api/processing/onboarding-dismiss |
Dismiss onboarding wizard. |
DELETE |
/api/processing/onboarding-dismiss |
Reset onboarding visibility. |
| Method | Path | Description |
|---|---|---|
POST |
/api/ocr/start |
Start OCR on all failed PDF files. Mutually exclusive with processing and exploration. |
POST |
/api/ocr/stop |
Gracefully stop OCR after current file. |
GET |
/api/ocr/status |
OCR state: {is_running, total_files, processed_files, ocr_success, ocr_failed, current_file, errors}. |
| Method | Path | Description |
|---|---|---|
GET |
/api/system/settings |
Current schedule settings (downtime window, flags). |
PUT |
/api/system/settings |
Update settings. Body: {schedule_enabled?, downtime_start?, downtime_end?} (HH:MM). |
GET |
/api/system/status |
Composite status: llama state, downtime, next boundary, and the status-pill {state_label, state_severity}. |
POST |
/api/system/end-downtime |
Force uptime until the current window's end. |
GET |
/api/system/chat-depths |
Chat models and Retrieval Depth levels: {default, model, models, levels, server_ctx}. models lists each chat model with available and, if not, unavailable_reason; levels are resolved for the active model with each level's limits, context window and assumed memory. |
POST |
/api/system/chat-depth |
Prepare the chat server for a level. Body: {depth}. Reloads the model with a larger context if needed (llama.cpp) or checks the memory estimate (MLX) and returns {ok, checked}, or {ok: false, error} if it does not fit in memory. |
POST |
/api/system/chat-model |
Switch the chat model. Body: {model, depth?}. Returns {ok, model, depth, notice, levels}, where depth is the level to use (the requested one, or the default with a notice if the new model cannot run it here), or {ok: false, error} with the previous model restored. |
GET |
/api/system/llama/memory |
What is loaded and the memory it holds: {chat_up, embed_up, chat_model, chat_backend, chat_ctx, chat_mib, embed_mib, total_mib}. llama-server sizes are summed from its load logs; the MLX server's is its measured process footprint. Shown in the pause dialog. |
POST |
/api/system/llama/pause |
Manually stop both llama-servers to free memory (409 if LLM work is active). |
POST |
/api/system/llama/start |
Manually start both llama-servers (returns immediately). |
| Method | Path | Description |
|---|---|---|
POST |
/api/backup/export |
Start a background export. Params: include_rag (default true). |
GET |
/api/backup/export/status |
Poll export progress. |
GET |
/api/backup/export/download |
Download the completed .georag archive. |
POST |
/api/backup/export/stop |
Cancel a running export. |
POST |
/api/backup/import/validate |
Upload and validate a .georag archive without importing. |
POST |
/api/backup/import |
Upload and import a .georag archive (background). |
GET |
/api/backup/import/status |
Poll import progress. |
POST |
/api/backup/import/stop |
Cancel a running import. |
| Method | Path | Description |
|---|---|---|
GET |
/api/health/llm |
Probe both llama-servers; returns an actionable checklist of any problems (unreachable, wrong/missing model alias, etc.). |
All values live in backend/config.py.
| Variable | Default | Description |
|---|---|---|
ENGINEERING_ROOT |
Parent of _0RAG/ |
Root directory scanned for documents |
DATA_DIR |
_0RAG/data/ |
Runtime data root |
CHROMA_DIR |
data/chroma/ |
ChromaDB on disk storage |
METADATA_DB |
data/metadata.db |
SQLite database path |
CACHE_DIR |
data/cache/ |
Application cache |
LOG_DIR |
data/logs/ |
Application logs |
| Variable | Default | Description |
|---|---|---|
CHAT_BASE_URL |
http://127.0.0.1:8001 |
Chat llama-server root |
EMBEDDING_BASE_URL |
http://127.0.0.1:8002 |
Embedding llama-server root |
CHAT_URL |
{CHAT_BASE_URL}/v1/chat/completions |
Chat completion endpoint |
EMBEDDING_URL |
{EMBEDDING_BASE_URL}/v1/embeddings |
Embedding endpoint |
EMBEDDING_MODEL |
nomic-embed-text-v1.5 |
Embedding model name (must match --alias on the embedding llama-server) |
EMBEDDING_DIM |
768 |
Vector dimensionality |
LLM_PARALLEL_SLOTS |
1 |
Concurrent inference slots (must match the chat server's -np) |
CHAT_MODELS |
(2 models) | The selectable chat models: backend (llama or mlx), path, the name sent in requests, whether it describes images, and its memory tiers (context and assumed memory per tier), plus the memory-estimate constants and the crash guard for MLX |
DEFAULT_CHAT_MODEL |
qwen3.5-9b |
Chat model used until one is chosen in the dialog. Env: GEORAG_CHAT_MODEL |
LLAMA_CHAT_MODEL |
(GGUF path) | Chat model GGUF file the supervisor launches |
LLAMA_CHAT_MMPROJ |
(GGUF path) | Multimodal projector GGUF for image captioning |
LLAMA_EMBED_MODEL |
(GGUF path) | Embedding model GGUF file |
MLX_CHAT_MODEL |
(model folder) | MLX chat model directory. Env: MLX_CHAT_MODEL |
MLX_LM_SERVER |
venv-mlx/bin/mlx_lm.server |
The MLX runtime's server executable. Env: MLX_LM_SERVER |
chat_launch(model, ctx) / LLAMA_EMBED_LAUNCH |
(argv lists) | Exact command lines the supervisor spawns; the chat one takes the model key and the context window size |
| Variable | Default | Description |
|---|---|---|
CHUNK_SIZE |
1000 |
Tokens per chunk |
CHUNK_OVERLAP |
200 |
Overlap between adjacent chunks |
EMBEDDING_BATCH_SIZE |
128 |
Texts per embedding API call |
BATCH_SIZE |
10 |
Files processed concurrently per batch |
TOP_K_PER_TAG |
5 |
Default chunks retrieved per tag by a vector search (chat uses its depth level's top_k) |
CONVERSATION_HISTORY_TURNS |
4 |
Past turns (8 messages) included in prompt |
CHAT_THINKING |
False |
Whether the chat model reasons before answering. Off by default; when on, the reasoning is unlimited and shown as a collapsible block |
CHAT_DEPTH_LEVELS |
(5 levels) | Retrieval Depth levels: tier, top_k and max_chunks for each; see Retrieval Depth Presets |
chat_depth_levels(model) |
(function) | The levels as they apply to one model: context and assumed memory of each level's tier |
DEFAULT_CHAT_DEPTH |
optimal |
Level used when a chat request names none |
The daily downtime window (
schedule_enabled,downtime_start,downtime_end) is not inconfig.py; they live in the single-rowapp_settingstable and are edited from the UI (defaults: disabled, 06:30 to 09:30).
| Variable | Default | Description |
|---|---|---|
OCR_LANGUAGES |
["eng"] |
Tesseract language codes for OCR. Add codes for other languages (e.g., ["eng", "tur"]). Requires corresponding Tesseract language packs. |
| Variable | Value |
|---|---|
SKIP_DIRS |
_0RAG, .git, __pycache__, node_modules, .DS_Store, venv |
SUPPORTED_EXTENSIONS |
28 formats: pdf, doc, docx, xls, xlsx, xlsm, ppt, pptx, png, jpg, jpeg, gif, bmp, tiff, txt, html, htm, rtf, csv, md, dwg, dxf, mp4, avi, mov, mkv |
┌───────────────┐ ┌──────────────┐ ┌───────────────┐
│ File │──1:M──│ FileTag │──M:1──│ Tag │
│ │ │ │ │ │
│ id (PK) │ │ id (PK) │ │ id (PK) │
│ relative_path │ │ file_id (FK) │ │ name (UNIQUE) │
│ filename │ │ tag_id (FK) │ │ display_name │
│ extension │ │ source │ │ description │
│ size_bytes │ │ confidence │ │ color │
│ modified_time │ │ created_at │ │ file_count │
│ parent_dir │ └──────────────┘ └───────────────┘
│ content_hash │
│ scan_status │ ┌───────────────┐ ┌──────────────┐
│ text_preview │ │ Conversation │──1:M──│ Message │
│ chunk_count │ │ │ │ │
│ processed_at │ │ id (PK) │ │ id (PK) │
│ auto_tagged │ │ title │ │ conv_id (FK) │
│ auto_tag_conf │ │ selected_tags │ │ role │
└───────────────┘ │ created_at │ │ content │
│ updated_at │ │ sources (JSON│
│ deleted_at │ │ created_at │
└───────────────┘ └──────────────┘
┌──────────────────────────────┐
│ AppSettings (single row) │
│ │
│ id = 1 │
│ schedule_enabled │
│ downtime_start / _end (HH:MM)│
│ auto_shutdown_on_manual_pause│
│ updated_at │
└──────────────────────────────┘
AppSettingsis a single row (id=1) holding the downtime schedule and the manual-pause behavior.- Legacy
wiki_pages/wiki_logtables and thewikiChroma collection may still exist in older databases but are unused.
| Field | Values | Meaning |
|---|---|---|
File.scan_status |
new, processed, failed, skipped |
Processing lifecycle |
FileTag.source |
auto, manual, folder_hint |
How the tag was assigned |
FileTag.confidence |
0.0 to 1.0 | Classification confidence (0.7 heuristic, 0.8 LLM) |
Conversation.deleted_at |
NULL or timestamp |
NULL = active; timestamp = in trash |
Message.role |
user, assistant, system |
Message author |
Message.sources |
JSON array | [{filename, file_path, page, chunk_index, relevance_score}] |
On startup, init_db() creates all tables if missing and runs lightweight ALTER TABLE migrations against existing databases:
- add
deleted_attoconversations(soft delete)
It also seeds the app_settings row (id=1) if missing. seed_tags() inserts the 34 default geotechnical tags (DEFAULT_TAGS) if the tag table is empty. SQLite runs in WAL mode with a busy_timeout so background writes don't lock out API reads.
The frontend is intentionally framework free. Vanilla JavaScript with module pattern controllers (IIFEs) and PicoCSS for styling. No build step, no transpilation, no node_modules. Static files served directly by FastAPI.
| File | Lines | Responsibility |
|---|---|---|
chat.js |
~1170 | Welcome state, tag selection, message submission, SSE streaming, KaTeX rendering, conversation sidebar, trash management, depth settings, cross tab mirroring |
documents.js |
~690 | File table rendering, name + semantic search, type/status/tag filters, pagination, single and batch tagging, mark-as-new/skipped, tag modal, Manage Tags |
panel.js |
~915 | Library panel lifecycle (scan, process, OCR, explore, complete), processing status polling (2s interval), OCR start/stop, reprocess all confirmation |
system_status.js |
~290 | Status pill rendering + polling, downtime schedule settings, manual llama pause/start, upcoming-event indicator |
router.js |
55 | SPA routing via History API, page toggle (chat/documents), spa:pageshow custom event for lazy init |
utils.js |
~125 | apiGet/apiPost/apiDelete wrappers, escapeHtml, tag badge factory, syncChannel (BroadcastChannel) |
Two BroadcastChannel instances coordinate state across browser tabs:
app-sync: Non streaming events (conversation created/deleted, processing started/stopped)chat-streaming: Real time streaming coordination (stream started, token, stream ended)
When Tab A starts streaming, Tab B detects this via polling GET /api/chat/streaming, then connects to GET /api/chat/stream-mirror to receive the same token stream.
- Brand color:
#14bf98(teal/mint) - CSS variables:
--brand-color,--sidebar-width(280px),--chat-max-width(900px),--panel-width(340px) - Responsive: Sidebar hides at 768px or narrower viewport width
- Animations: Tag pill pop in (0.3s), panel slide out (0.3s), loading dots spinner
- Theme: Light mode only (PicoCSS dark mode ready but not wired up)
Two modes supported:
- SPA mode (
spa.html): Single HTML page with both chat and documents sections toggled by client side router usinghistory.pushState - Server rendered mode: Separate
index.htmlanddocuments.htmlpages with full page reloads
The SPA router fires a custom spa:pageshow event, which triggers lazy initialization of the documents page on first visit.
1. Detect Python 3.13 (fallback to 3.12 → 3.11 → python3)
2. Create venv/ if missing, then pip install requirements.txt
(on Apple Silicon, if the MLX model folder exists: also create venv-mlx/ and install requirements-mlx.txt)
3. mkdir -p data/{chroma,cache,logs}
4. Verify the llama-server binary and all three model files exist (fail fast)
5. exec uvicorn backend.app:app --host 0.0.0.0 --port 3000 --reload
(the app itself launches/manages the two llama-servers)
PowerShell equivalent of start.sh: sets up the venv, installs dependencies, creates data directories, and starts the server.
Windows support is experimental and unsupported. The Windows startup script and the packaged EXE (below) have not been tested and may not work. GeoRAG is developed and run on macOS (Apple Silicon); treat the Windows path as a work in progress. Use
start.shon macOS/Linux for a supported setup.
Standalone entry point used by the PyInstaller EXE. run_server.py sets the working directory next to the executable, creates data directories, prints the banner, and runs the app; georag.spec builds it into a Windows .exe via pyinstaller georag.spec.
Experimental / needs testing. The bundled EXE build is not yet supported. Do not rely on it for production use.
1. Find app + llama-server PIDs by port (3000/8001/8002) and command line
2. kill each PID (no sudo)
This project is licensed under the GNU Affero General Public License v3.0 (AGPLv3).
You are free to use, modify, and distribute this software. If you run a modified version of GeoRAG as a network service, you must make your source code available to users of that service under the same license.
See LICENSE for the full text.