Skip to content

pkyanam/graphbrain

Repository files navigation

Graphbrain

Hosted, graph-native memory and knowledge layer for AI agents, individuals, and teams. Agents use it through MCP; humans supervise it through the web UI; every brain is an isolated Postgres database with pgvector and pgGraph. See PRD.md for the full product spec — this repo is the internal prototype (PRD §20).

Quick start

Requires Node 22+ and Docker.

npm install
npm run db:up        # builds Postgres 17 + pgvector + pgGraph, starts on :55432
npm run dev          # control plane migrations, API + MCP + worker + web UI on :4100

Open http://localhost:4100, sign up (first account adopts any pre-auth local brains), create a brain, and capture some markdown. Then:

npm run smoke        # end-to-end check against the running server
npm test             # unit tests
npm run typecheck

Provider keys (optional, BYO)

Without keys, Graphbrain uses a deterministic local mock embedder (search still works), heuristic entity extraction, and ask degrades to cited search results. With keys it uses real embeddings, LLM extraction, and synthesized cited answers:

cp .env.example .env
# set OPENAI_API_KEY (embeddings + chat) and/or ANTHROPIC_API_KEY (chat)

CLI (thin client)

bin/graphbrain is a thin-client CLI for agents and humans: every command routes to the hosted server's MCP endpoint with a brain token, and the command surface is discovered from /v1/cli-manifest — generated from the same operation registry (src/ops.ts) as the MCP tools, so CLI and MCP can never drift.

graphbrain connect http://localhost:4100 gbt_...   # token from web UI → Connect
graphbrain commands                                # list all operations
graphbrain capture "Jane committed to the Q3 launch" --title "Q3 commitment"
graphbrain ask "what is open with Jane?"
graphbrain explore jane-smith --depth 2
graphbrain suggestions
graphbrain experts "Q3 launch"
graphbrain maintain-now
graphbrain sync ~/notes --source notes        # live markdown folder sync

GRAPHBRAIN_URL/GRAPHBRAIN_TOKEN env vars override the saved config.

Live markdown sync

graphbrain sync <dir> keeps a local markdown folder (vault, docs/, repo) in the brain, thin-client style: the CLI sends {path, sha256} pairs, the server plans the diff (sync_plan), and only changed files upload (sync_file). Frontmatter title/type/tags and [[wikilinks]] are honored, every push is a new document version, local deletions soft-delete with --prune, and a sync that overwrites server-side edits records _sync_conflict metadata in the document frontmatter (the overwritten state stays in version history). graphbrain source-status <name> shows document counts and last_synced_at.

Agent skills

skills/ is a fat-markdown skill pack (the GBrain pattern: thin harness, fat skills) teaching agents how to use the brain well — brain-first lookup, capture discipline, enrichment loops, review triage, briefings, maintenance. skills/RESOLVER.md is the trigger→skill dispatch table.

graphbrain skills install                 # → .claude/skills (or ~/.claude/skills)
graphbrain skills install --target <dir>  # OpenClaw / Hermes / Codex skill dirs

Current batch (20): memory-ops, ask, capture, ingest, enrich, filing, review, briefing, maintain, automation, minion-orchestrator, signal-detector, concept-synthesis, setup, smoke-test, and the Phase D ingestion family — meeting-ingestion, conversation-transcripts, media-ingest, markdown-sync, webhook-transforms. Remaining GBrain skill classes (taxonomist, citation-fixer, …) land with their backing capabilities per docs/GBRAIN_AUDIT.md phases.

Connecting an agent

In the web UI: open a brain → Connect → create a token. Then for Claude Code:

claude mcp add graphbrain http://localhost:4100/mcp \
  --transport http \
  --header "Authorization: Bearer <TOKEN>"

Smoke-test prompts: "Call graphbrain's brain_status tool", "Save that Graphbrain is my hosted agent memory project", "What is Graphbrain?".

MCP tools: brain_status, capture_memory, search_brain, ask_brain, list_documents, get_document, save_document, search_entities, get_entity, explore_graph, find_path, recall_facts, propose_fact, propose_relationship, list_suggestions, approve_suggestion, reject_suggestion.

Default agent tokens carry read, write, propose scopes — agents can propose facts and relationships but cannot approve them (PRD §16.2).

Architecture

One Node/TypeScript process serves the REST API (/v1), the stateless MCP gateway (/mcp, streamable HTTP, bearer token → brain), the static web UI, and an in-process job worker. Two database layers:

  • Control plane (graphbrain_control): users, workspaces, brain registry, API tokens, global audit log. No brain content.
  • Per-brain databases (graphbrain_brain_<slug>): documents, versions, chunks (pgvector embeddings), events, entities, aliases, mentions, facts, claims, relationships, memories, suggestions, jobs, config. Provisioned and migrated on brain creation (src/core/provision.ts, migrations/brain/).

Key flows:

  • Capture (src/core/documents.ts): document + version + event, then queued chunk_embed and extract_entities jobs.
  • Ingestion breadth (src/core/sync.ts, src/core/ingestion.ts, src/core/webhooks.ts): live markdown sync keyed by (source, source_uri) with checksum plans, frontmatter/tag/wikilink handling, and conflict metadata; ingest_meeting / ingest_conversation / ingest_media jobs with deterministic keyless parsers (binary media without text completes with a skipped status describing what to extract); tokenized webhook endpoints (POST /v1/ingest/webhook/<token>) that transform JSON payloads into documents or meeting ingests with provenance and sanitization.
  • Jobs (src/core/jobs.ts): Postgres-backed queue per brain, FOR UPDATE SKIP LOCKED pickup, retries, lifecycle controls (cancel/retry/pause/resume/replay), progress/messages, watchdog, in-process poller.
  • Maintenance cycle (src/core/schedules.ts, src/core/intel.ts): control-plane schedule rows queue per-brain maintenance jobs with quiet hours; the cycle re-extracts stale docs, refreshes hot entity summaries, rebuilds graph, scores salience, and exports a bundle.
  • Intel (src/core/intel.ts): agent-facing ops for experts, contradictions, anomalies, orphans, salience scoring, and enrichment jobs. All work keyless through SQL/search heuristics and the deterministic summarizer fallback.
  • Search (src/core/search.ts): vector + keyword + entity-alias + one-hop graph expansion, merged with reciprocal rank fusion (PRD §11.3).
  • Ask (src/core/ask.ts): retrieve → pack context → cited synthesis with gap reporting; degrades cleanly without an LLM key.
  • Extraction (src/core/extraction.ts): LLM extraction when a chat key exists, capitalization heuristic otherwise. Entities auto-create (low-risk); relationships and facts become suggestions awaiting human review (PRD §7.5).
  • Graph (src/core/graph.ts): canonical truth lives in the entities and relationships tables. pgGraph is registered as derived acceleration (graph.add_table/add_edge/build) for traversal and pathfinding, with plain recursive-SQL fallbacks when the extension is unavailable.
  • Provider gateway (src/providers.ts): embed() / complete() abstraction over OpenAI, Anthropic, or the local mock.

Docker image

docker/Dockerfile layers pgvector (PGDG apt) onto the official pgGraph image and runs with shared_preload_libraries=graph. Brain DBs run CREATE EXTENSION vector; CREATE EXTENSION graph; at provision time, and brains stay functional (SQL fallback, health badge) if graph is missing.

Deployment (Coolify + Cloudflare Tunnel)

Production runs as a three-service Docker Compose stack (docker-compose.prod.yml) designed for Coolify:

  • postgres — Postgres 17 + pgvector + pgGraph (docker/Dockerfile), data on the pgdata volume.
  • app — the single Node process (docker/app.Dockerfile: API, MCP gateway, web UI, job worker, plus postgresql-client-17 so export jobs can pg_dump), exports on the exports volume.
  • cloudflared — bundled Cloudflare Tunnel sidecar. No host ports are published; the tunnel is the only ingress.

Setup:

  1. In Cloudflare Zero Trust → Networks → Tunnels, create a tunnel and add a public hostname pointing at http://app:4100. Copy the tunnel token.
  2. In Coolify, create a Docker Compose resource from this repo and set the compose file to docker-compose.prod.yml.
  3. Set environment variables: DATABASE_PASSWORD (strong), CLOUDFLARE_TUNNEL_TOKEN, and optionally OPENAI_API_KEY / ANTHROPIC_API_KEY (+ model overrides). Keyless mode works.
  4. Deploy. Migrations run on boot; the app healthcheck polls /v1/status.

CI/CD

.github/workflows/ci.yml runs on every push/PR:

  • checksnpm run typecheck + npm test.
  • smoke — builds the pgvector+pgGraph Postgres image, boots the server, and runs the full end-to-end npm run smoke suite.
  • prod-image — validates docker-compose.prod.yml and builds the app image.
  • deploy — on green main only: triggers the Coolify deploy webhook if the COOLIFY_WEBHOOK_URL (and optional COOLIFY_API_TOKEN) repo secrets are set, so prod only redeploys after CI passes. (Alternatively let Coolify watch the repo directly and skip the secrets.)

Export (Stage 8)

Per-brain exports run as background jobs and land in exports/ as .tar.gz (web UI Export tab, or POST /v1/brains/:id/exports {"format": ...}):

  • markdown — human-readable projection: documents/, entities/ (profiles with relationships + facts), memories/, sources/.
  • dumppg_dump -Fc of the brain database (host pg_dump, docker-exec fallback).
  • bundle — both, plus manifest.json (versions, counts, embedding/pgGraph config, applied migrations), schema.json, restore.sh, provider-config.example.json.

Teams (Stage 7)

Email/password auth with server-side sessions (HttpOnly cookie). Workspaces hold brains and members with roles owner / admin / member / viewer — read needs viewer+, write/review needs member+, tokens/policy/member management needs admin+. Invite teammates by email from the dashboard; they claim membership by signing up with that address (no outbound email in the prototype). Agent tokens are brain-scoped with selectable scopes (read/write/propose/approve) and default to no approval rights.

Prototype status vs PRD

Done (PRD §20 prototype definition and beyond): local run, brain provisioning, capture/upload, hybrid search, MCP with real agents (full §12.1 initial tool surface, Phase C-expanded tools), cited ask_brain, entity extraction, relationships + review queue with a configurable auto-approval policy, entity profile summaries (LLM or deterministic), pgGraph build/traverse/path with SQL fallback, markdown/pg_dump/bundle export, email/password auth, team workspaces with roles and shared team brains, thin-client CLI + skills, job lifecycle controls, scheduled maintenance, salience/intel ops, server-side enrichment jobs, live markdown sync with conflict metadata, meeting/conversation/media ingestion jobs, and tokenized webhook ingestion.

Not yet (MVP stage 9 + GBrain parity): encrypted per-workspace provider keys (env-only today), rate limits, billing, self-host package, transcription/OCR providers for binary media, schema packs, code intelligence, evals. See docs/GBRAIN_AUDIT.md for the full GBrain capability audit.

About

Hosted, graph-native memory and knowledge layer for AI agents, individuals, and teams — isolated Postgres per brain with pgvector + pgGraph, MCP-first.

Resources

Stars

Watchers

Forks

Contributors