Harbor Clerk

Harbor Clerk turns watched folders into a private, searchable document archive.

Harbor Clerk packages a serious local document intelligence stack into a Mac-native app: OCR for scans, hybrid lexical and semantic search, local embeddings and reranking, cited AI answers, and MCP/CLI access for agents. A technical operator could assemble pieces of this from open-source tools; Harbor Clerk makes that power available to small teams without turning document search into an infrastructure project.

It reads your files in place, extracts text (including OCR for scanned documents), splits them into searchable passages, and indexes everything locally so you can search or ask questions across your entire collection. Results always come with clear citations so you can jump straight back to the original document and verify the source.

Everything runs on your machine. No SaaS account. No background sync. No shared tenancy. Your originals never move — Harbor Clerk just reads them where they live.

Harbor Clerk is designed for document-heavy small teams, independent operators, researchers, and privacy-focused individuals. It runs comfortably on a Mac mini or similar hardware and includes cited answers powered by local AI models.

If you want to connect external AI tools, Harbor Clerk exposes an MCP endpoint that allows them to search your knowledge base safely. With the recommended cloud-connector scope, they receive retrieved passages and citations rather than the whole corpus.

Cloud models are supported through scoped MCP/connectors in the current release. Direct cloud-backed Ask or Research inside the app is a separate fast-follow, not a requirement for the initial release; see Cloud model boundaries for the release gate.

This isn't a platform. It's a tool. It keeps your documents where they belong — and makes them useful.

graph LR
    docs["Drop documents into a watched folder<br/>PDFs • Scans • Notes"]
    process["Harbor Clerk organizes them<br/>Text Extraction • OCR • Chunking • Hybrid Search + Embeddings"]
    ask["Search, ask, or research your archive<br/>Answers include citations"]
    docs --> process --> ask

External models can ask questions — they don't get your archive.

graph LR
    subgraph local["Stays Local"]
        docs["Your Documents"]
        harbor["Harbor Clerk<br/>OCR · Search · Cited AI"]
    end
    subgraph external["Optional External Models"]
        models["Claude / GPT / other MCP clients"]
    end
    docs --> harbor
    models -. "MCP queries" .-> harbor
    harbor -. "retrieved snippets + citations only" .-> models

Once your documents are indexed, you can reach them three ways — pick whichever fits the moment.

graph LR
    corpus[("Your indexed corpus")]
    web["Web UI<br/>Search · Ask · Research"]
    mcp["MCP endpoint<br/>Cloud LLMs · Claude · ChatGPT"]
    cli["harbor-clerk CLI<br/>Local agent harnesses<br/>OpenClaw · Claude Code · Codex · Aider"]
    web --> corpus
    mcp --> corpus
    cli --> corpus

1. Web UI — search, cited answers, deep research. Open Harbor Clerk in your browser (or the native Mac window) and search, browse, or ask questions with local AI models. Deep Research runs structured plan→search→read→synthesize over the whole corpus and produces a cited report.

2. MCP endpoint — for cloud LLMs. Connect Claude, ChatGPT, Claude Desktop, Gemini CLI, or any MCP-compatible client. They authenticate with a scoped, read-only API key and receive only the cited snippets needed to answer a question — never your full corpus.

3. harbor-clerk CLI — for local agent harnesses. OpenClaw, Claude Code, Codex, Aider, and other harnesses that prefer composing shell commands over speaking MCP can drive Harbor Clerk through a first-class CLI that mirrors the MCP tools as subcommands, with full --help pages and JSON output ready to pipe through jq. Same auth model as MCP, same audit trail (logged distinctly as request_type="cli_tool"). Off by default; admin opt-in.

export HARBOR_CLERK_API_KEY=hc_...
harbor-clerk --help                  # full subcommand list
harbor-clerk search "termination clause" | jq '.hits[] | {title: .doc_title, pages, chunk_id}'

A copy-pasteable agent skill for these harnesses is in Settings → Integrations. See Integrations for setup details and Agent memory eval plan for the evidence needed before stronger OpenClaw claims.

Harbor Clerk includes evaluation tooling for retrieval quality, cited answers, and agent-memory workflows. The safe release claim is that Harbor Clerk provides local document search, cited local AI answers, and controlled MCP/CLI access for external models and agent harnesses. Stronger claims should come from current, reproducible eval runs.

See Evaluation and release claims for the current claim posture, eval methodology pointers, and the OpenClaw claim ladder. Cloud model boundaries explains the current MCP/cloud connector path and the go/no-go bar for future in-app cloud mode. Release smoke test is the manual QA matrix for release-candidate builds.

Known limitations for the initial release:

• Local AI quality varies by model, corpus, prompt, and task.
• Messy email corpora and exhaustive "find every instance" questions remain difficult, especially when completeness matters.
• OCR quality depends on scan quality, layout, and installed language packs.
• Citations ground answers, but important outputs should still be checked against source documents.
• Harbor Clerk is local-first and single-tenant, not a hosted multi-tenant enterprise platform.
• Advanced integrations such as MCP, CLI, cloud LLMs, OpenClaw, and Docker require operator setup.
• Direct cloud-backed Ask/Research inside the app is a fast-follow unless the citation, key-storage, disclosure, path-policy, and failure-mode checklist is satisfied.
• Backup and restore are documented but not yet a polished in-app workflow.

Your documents stay private Everything runs locally. No uploads to SaaS services, no background syncing, and no shared infrastructure.

Your files become searchable knowledge Harbor Clerk reads documents, performs OCR when needed, builds hybrid full-text and semantic search, and lets you explore everything through search, cited answers, or research — always with citations.

Use any model you trust Ask locally with built-in models, connect external AI tools through MCP, or hand the CLI to a local agent harness. They see only the passages needed to answer a question — never your full corpus.

1. Download Harbor Clerk from the releases page
2. Launch Harbor Clerk Server (menubar app — starts all backend services)
3. Open Harbor Clerk (web UI inside a native window)
4. Click Folders → Add Folder, pick a directory full of documents, and let ingestion run
5. Ask questions, browse the corpus, or kick off a deep research task

That's it — everything runs locally. To wire in an external LLM or agent harness, head to Settings → Integrations for MCP and CLI setup.

git clone https://github.com/r0shi/harborclerk.git
cd harborclerk
cp .env.example .env
mkdir -p data/watch/inbox       # any subdirectory of ./data/watch becomes a watched folder
docker compose up --build

Then open https://localhost, accept the self-signed certificate, and create your admin account. Drop documents into ./data/watch/inbox/ (or any subdirectory of ./data/watch/) and they'll start ingesting within ~60 seconds. See docs/watched-folders-docker.md for mounting external paths and other operator details.

To enable the harbor-clerk CLI for local agent harnesses, set ENABLE_CLI_ACCESS=true in .env and restart the app service.

• Document-heavy small teams without a formal knowledge base
• Researchers with large document collections
• Consultants, lawyers, and analysts managing private files
• Anyone who wants LLM-style document search without uploading data to the cloud

Harbor Clerk follows a simple rule:

Documents stay local. Models come and go.

Your corpus should live on infrastructure you control. AI models — local or cloud — should interact with it through well-defined interfaces.

Harbor Clerk is designed to be:

• Local-first — your data never has to leave your machine
• Model-agnostic — use local models, cloud models, or both
• Transparent — answers always cite their sources
• Simple to operate — runs comfortably on a single small machine

Harbor Clerk can run in two ways:

Both deployments use the same Python watchdog-based watcher under the hood (FSEvents on macOS, inotify on Linux, polling fallback for NFS/SMB/fuse mounts).

Requirements: Mac mini M2 or newer (M1 works, M2+ recommended), macOS 15.0+, 16 GB RAM minimum.

App data lives in ~/Library/Application Support/Harbor Clerk/ — PostgreSQL database, downloaded LLM models, logs, and settings. Your source documents stay where you put them; Harbor Clerk only references them by path.

Open Preferences (Cmd+,) from the menubar to configure network access, worker preset, ports, and log level. Manage watched folders from the Folders tab in the web UI — the native folder picker dialog walks you through picking a directory, and macOS bookmark data keeps it tracked across renames and moves.

For a basic backup, quit Harbor Clerk Server and copy ~/Library/Application Support/Harbor Clerk/ to backup storage. Keep the watched-folder source files backed up separately because Harbor Clerk reads those files in place. See Backup and restore for the cautious initial guidance.

Requirements: Docker Desktop or Docker Engine + Compose, 4 GB RAM minimum (8 GB recommended).

Edit .env and change SECRET_KEY to a random string before starting:

python3 -c "import secrets; print(secrets.token_urlsafe(48))"

Open https://localhost/ and accept the self-signed certificate. Create your admin account on the setup page.

Watched folders on Docker are operator-controlled: the watcher service auto-discovers any top-level subdirectory of WATCH_ROOT (/data/watch inside the container, bind-mounted from ./data/watch on the host) and ingests files dropped into it. There is no "Add Folder" button on Docker — you create directories, or mount external paths into /data/watch/<name>. See docs/watched-folders-docker.md for the full operator guide.

docker compose up --build         # build and start (foreground)
docker compose up --build -d      # build and start (background)
docker compose down               # stop (keeps data)
docker compose down -v            # stop and delete all data
docker compose logs -f app        # tail app logs
docker compose logs -f watcher    # see what the watcher is picking up

Services: gateway (Caddy), app (FastAPI + React SPA), watcher (folder watching + ingest queueing), worker-io, worker-cpu, worker-llm (summarize), embedder (Granite-R2 multilingual, 768-dim), reranker (cross-encoder re-ranking), llama-server (local LLM inference), postgres (pgvector + pg_trgm), minio, tika.

Drop a file into a watched folder and it flows through seven idempotent stages:

1. Extract — pull text from PDF, Office, eBook, HTML, email, and other formats via Apache Tika (TXT/MD/CSV decoded directly)
2. OCR — conditional: always for images (JPEG/PNG/TIFF), for PDFs with little extractable text; never for text-native formats. Uses Tesseract (English + French)
3. Chunk — split into ~1000 character segments with 150 char overlap, preserving page references and detecting language per chunk
4. Entities — extract named entities (people, places, organizations) via spaCy NER (English + French models)
5. Embed — generate 384-dim vectors via the embedder service
6. Summarize — generate a document summary (local LLM, with extractive fallback)
7. Finalize — mark ingestion complete

Progress is streamed to the UI via server-sent events with a visual stage ring showing each step. Processing can be cancelled from the UI. Renames, edits, and deletions in the watched folder propagate automatically — Harbor Clerk reprocesses the file or soft-deletes the document, and a 30-day reaper purges originals that stay missing.

Because Harbor Clerk reads originals in place, getting back to the actual file on disk works differently in each deployment — and intentionally so:

On macOS, the document detail page shows a Reveal in Finder button that opens the file's enclosing folder via the native bridge — no HTTP, no bytes leaving the machine. On Docker the same button isn't available (no native bridge), and GET /api/docs/{id}/download returns 403 unless an admin sets ALLOW_SOURCE_DOWNLOAD=true in the compose env.

The reason the download endpoint is locked down by default: it returns the raw bytes of any document the caller can already see in search, which is a meaningful escalation over the chunk-excerpt access read-only API keys are designed for. Reveal in Finder sidesteps this entirely because it never touches the API.

Results combine PostgreSQL full-text search (bilingual English/French) and pgvector cosine similarity, normalized and merged into a single score with a small boost for higher-confidence OCR text. A cross-encoder re-ranking pass (bge-reranker-v2-m3, served by a dedicated reranker container/subprocess; on by default, gracefully degrades to merged-score ordering if unavailable) refines the top-K pool before returning results. All results include source citations with page numbers. Search supports filtering by document, date range, language, and MIME type, with faceted results grouping hits by document.

Ask runs a local AI model (via llama-server) with access to the knowledge base through tool calls. Models can be downloaded and managed from Settings. No data leaves the machine unless you deliberately connect an external model through MCP or another integration.

The local AI path uses tool calls to search, read passages, explore document structure, and query entities during the conversation. Results include source citations with page numbers so you can verify findings against the original documents.

For complex questions that require systematically examining your corpus, Research mode runs a structured plan→search→read→extract→synthesize pipeline that uses the LLM for bounded steps (query planning, note extraction, synthesis) and drives retrieval from Python with measurable coverage. The result is a cited report.

Configuration per task:

• Strategy: Search-driven (follows leads) or Systematic sweep (reviews every document)
• Depth: Light / Standard / Thorough (controls planning frequency)
• Time limit: 15 minutes to 3 hours

Research runs in the background with live progress streaming: elapsed time, tool-by-tool activity log, and research notes visibility. Results include the agent's raw findings and a polished synthesis report with citations.

Beyond basic search, Harbor Clerk builds a navigable knowledge graph:

• Document outlines — heading structure extracted during ingestion
• Entity index — people, places, and organizations extracted by spaCy, searchable and browsable
• Cross-document similarity — find related documents using embedding-based nearest neighbors
• Corpus overview — aggregate stats (language distribution, MIME types, page counts, date ranges)
• Topic modeling — BERTopic-based clustering discovers themes across the corpus, surfaced in Observatory visualizations (treemap, bar chart, keywords, cluster map) and injected as context for chat
• Stats dashboard — visual corpus analytics: language/file type/OCR charts, pipeline timing, entity co-occurrence network (d3-force), UMAP document cluster map (colorable by topic), and topic distribution charts

Connect ChatGPT, Claude Desktop, Claude Code, Gemini CLI, OpenClaw, or any MCP-compatible tool. ChatGPT connects via OAuth (requires a public URL); all others use API key authentication. Connection guides are available in the Integrations settings page and in docs/integrations.md.

Scoped API keys control what each external tool can access: permission tiers (search / read / full), document scope (by topic or watched folder), snippet size limits, tool overrides, and expiry dates.

Rate limiting (per-minute + per-hour) protects against runaway AI agents. System-wide defaults apply to all keys; per-key overrides available.

Audit dashboard provides full per-key introspection: request counts, tool breakdowns, error/denial/rate-limit tracking, and a filterable request log.

Security note for agentic tools (OpenClaw, etc.): Autonomous AI agents can make many tool calls in rapid succession. Always create a dedicated, scoped API key with rate limits and an expiry date. Monitor usage via the per-key audit dashboard.

The harbor-clerk CLI mirrors the MCP tool surface as shell subcommands, including find-all for document enumeration. See the Three ways to use Harbor Clerk overview above for the framing; this section is the operator detail.

Enabling it:

• macOS: toggle in Harbor Clerk Server → Preferences. The middleware re-reads config on every CLI request, so changes take effect within seconds — no restart.
• Docker: set ENABLE_CLI_ACCESS=true in the env and restart the app service.

Auth: the CLI talks to the same /mcp endpoint as MCP clients do, using the same API key:

export HARBOR_CLERK_API_KEY=hc_...   # mint in Settings → API Keys
harbor-clerk --help                  # full subcommand list
harbor-clerk <command> --help        # man-page-class help with JSON return shape + examples

Audit: CLI traffic is logged as request_type="cli_tool" (distinct from mcp_tool) so you can split per-key dashboards by surface. Settings → Integrations has a copy-pasteable agent skill markdown for OpenClaw, Claude Code, and similar runtimes — drop it into the harness's skill directory and the agent learns the corpus.

• Human users: email + password, JWT access tokens + refresh cookies. Roles: admin / user.
• API keys: admin-created, read-only, scoped. Per-key permission tiers, document scope, rate limits, snippet caps, expiry. Stored as SHA-256 hashes.
• OAuth 2.1: Dynamic client registration for ChatGPT and other OAuth-based MCP clients.

141 operations across 16 groups. Full reference: docs/architecture.md.

POST /mcp — Streamable HTTP transport. The CLI mirrors the same practical tool surface, including kb_find_all via harbor-clerk find-all.

19 tools. Authenticate with Authorization: Bearer <api_key>.

cd macos
make all

This builds both apps into macos/build/output/. Requires Xcode command-line tools, Python 3.12+, and Homebrew (for Tesseract).

cd frontend
npm install
npm run dev     # dev server with HMR
npm run build   # production build → dist/

The project uses uv for Python package management:

uv sync
uv run harbor-clerk-api      # API server
uv run harbor-clerk-worker   # background worker

MIT — see LICENSE for details. Third-party dependencies are listed in THIRD_PARTY_NOTICES.md.