A persistent memory system for AI agents built on Neo4j with native vector indexes. Implements semantic retrieval with hidden layer concept abstraction, recursive self-improvement and Hebbian learning.

Key insight: The critical metric isn't average retrieval score—it's state survival under context compaction. Baseline agents forget architectural decisions after compactions. MDEMG maintains decision persistence indefinitely. If you do a quick review and come to the conclusion that this is just another tool built for small dev teams, please look closer. I would be happy to engage in discussion related to the mdemg framework, how it works, the problems it solves, and most importantly how can WE make it better.

• Docker Desktop (macOS/Windows) or Docker Engine (Linux) with Compose v2
• Embedding provider (choose one):
  • OpenAI API key (recommended, requires account)
  • Ollama (local, free, no API key needed)

# macOS (Homebrew)
brew tap reh3376/mdemg
brew install mdemg

# Linux / Windows (WSL2)
curl -fsSL https://raw.githubusercontent.com/reh3376/mdemg/main/scripts/install.sh | bash

# Verify installation
mdemg version

Windows: Install WSL2 first, then run all commands inside WSL2.

git clone https://github.com/reh3376/mdemg.git && cd mdemg
go build -o bin/mdemg ./cmd/mdemg
./bin/mdemg version

cd /path/to/your/project
mdemg init                  # Interactive setup: scans ports, generates .env, starts Docker stack
# OR
mdemg init --quick          # Non-interactive with sensible defaults

This will:

1. Check Docker is available and has adequate resources
2. Scan for 6 free TCP ports (all dynamically assigned — no hardcoded defaults)
3. Generate .env with port assignments and credentials
4. Create .mdemg/config.yaml and .mdemgignore
5. Run docker compose up -d (starts all 5 services)
6. Wait for the MDEMG server health check

All ports are dynamic. Each project gets its own COMPOSE_PROJECT_NAME for multi-instance isolation. See docs/user/quickstart-docker.md for the full Docker deployment guide.

For the local-LLM path (mdemg-llm-v1, Phase 5 Qwen3-14B fine-tune served via llama.cpp llama-server):

brew install ollama          # one-time — Ollama is the distribution channel
mdemg model pull             # RAM-auto picks Q4_K_M / Q5_K_M / Q8_0 quant
# → prints the MDEMG_MODEL_PATH line to add to .env

Three quants serve three RAM tiers: Q4_K_M (~9 GB, 16 GB Macs), Q5_K_M (~11 GB, 24 GB — production canonical), Q8_0 (~16 GB, 32+ GB). See docs/features/local-model-distribution.md for the full Configurability Contract (every value overridable via env or flag — namespace, name, backend, quant allowlist, RAM-tier map, paths).

Skip this step if you're using OpenAI for inference; OpenAI is the framework's fallback.

Once the stack is running, open the built-in browser dashboard:

http://localhost:${MDEMG_PORT}/ui/

The dashboard has 10 tabs: Status (health + Grafana links + restart), Memory (layer breakdown + export/import), Learning (Hebbian stats + freeze/prune), Config (editable with Save All), Logs (searchable viewer), RSIC (service controls + trigger), Plugins (lifecycle management), and Features (service status). Links to 7 Grafana dashboards for detailed metrics.

mdemg ingest --path .       # Index your codebase into the memory graph
mdemg hooks install         # Auto-ingest on every git commit (optional)

Performance tip: Use --speed fast for large codebases, --speed thorough for pre-benchmark runs. See Ingest Performance Guide for details.

That's it. Your AI agent now has persistent memory.

For projects using Claude Code, the sidecar quickstart handles everything — config, services, MCP integration, and hooks — in a single command:

cd /path/to/your/project
mdemg sidecar quickstart    # init + install + up + attach-agent + generate-hooks

This creates .mdemg/sidecar.yaml, starts Neo4j + MDEMG server, writes .claude/mcp.json, generates session-start.sh and prompt-context.sh hooks, registers them in settings.local.json, and enables enableAllProjectMcpServers.

See the Quickstart Guide for a detailed walkthrough, or run mdemg demo to try it with sample data.

Upgrading? Run mdemg upgrade to self-update to the latest release.

Everything needed to independently verify our results is included.

• Docker (Neo4j)
• Go 1.24+
• Python 3.10+ (grading + utilities)

Embeddings (choose one):

• OpenAI API key, or
• Ollama (local embeddings)

Agent under test (choose one):

• Claude Code (recommended baseline runner), or
• Any tool-using LLM agent that can call the MDEMG API

# 1. Clone and setup
git clone https://github.com/reh3376/mdemg.git && cd mdemg
cp .env.example .env  # Add your embedding provider credentials

# 2. Build and start services
go build -o bin/mdemg ./cmd/mdemg
./bin/mdemg db start              # Launch Neo4j container
./bin/mdemg start --auto-migrate  # Start server daemon with migrations

# 3. Ingest test codebase (or use your own)
./bin/mdemg ingest --space-id=benchmark --path=/path/to/target-repo

# 4. Run consolidation
curl -X POST http://localhost:9999/v1/memory/consolidate \
  -H "Content-Type: application/json" -d '{"space_id": "benchmark"}'

# 5. Run benchmark
# Questions: docs/benchmarks/whk-wms/test_questions_120_agent.json
# Grader: docs/benchmarks/grader_v4.py

Report output: grades_*.json contains per-question scores with evidence breakdown.

# Question bank hash (SHA-256)
shasum -a 256 docs/benchmarks/whk-wms/test_questions_120_agent.json
# Expected: 24aa17a215e4e58b8b44c7faef9f14228edb0e6d3f8f657d867b1bfa850f7e9e

Full reproducibility details for skeptics:

Baseline definition: Same agent runner and tool permissions, no MDEMG retrieval, relying on long-context + auto-compaction only (memory off).

Category Performance (Edge Attention):

The Q&A battery measures single-turn retrieval accuracy. The critical differentiator is state survival under compaction:

When context windows fill and auto-compaction kicks in, baseline agents lose architectural decisions. MDEMG persists them in the graph.

The baseline forgets under pressure. MDEMG remembers.

MDEMG provides long-term memory for AI agents, enabling them to:

• Store observations: Persist code patterns, decisions, and architectural knowledge
• Semantic recall: Retrieve relevant memories via vector similarity search
• Concept abstraction: Automatically form higher-level concepts from related memories (hidden layers)
• Associative learning: Build connections between memories through Hebbian reinforcement
• LLM re-ranking: Apply GPT-powered relevance scoring for improved retrieval quality

• Multi-layer graph architecture: Base observations (L0) → Hidden concepts (L1) → Abstract concepts (L2+)
• Hybrid search: Combines vector similarity with graph traversal, fused by RRF column voting with sparse gating and context fingerprinting (see docs/features/column-voting-retrieval.md)
• Conversation Memory System (CMS): Persistent memory across agent sessions with surprise-weighted learning
• Symbol extraction (UPTS): Unified Parser Test Schema supporting 28 languages with file:line evidence and SHA256 fixture verification
• Plugin system: Extensible via ingestion, reasoning, and APE (Autonomous Pattern Extraction) modules
• Evidence-based retrieval: Returns symbol-level citations (file:line references) with results
• Capability gap detection: Identifies missing knowledge areas for targeted improvement
• Codebase ingestion API: Background job processing for large codebase ingestion with consolidation
• Git commit hooks: Automatic incremental ingestion on every commit via post-commit hook
• Freshness tracking: TapRoot-level staleness detection with configurable thresholds
• Scheduled sync: Periodic background sync to keep memory graphs up-to-date
• Webhook integration: Linear webhook endpoint with HMAC signature verification and debouncing
• File watcher: Standalone mdemg-watch binary for automatic re-ingestion on file changes
• Orphan detection: Timestamp-based detection of nodes missing from re-ingestion with archive/delete/list actions
• Graph orphan cleanup: Cross-space zero-edge node scan with consolidate/archive/delete fix actions and protected space enforcement
• Edge consistency: Automatic staleness tracking and edge weight refresh during consolidation
• Backup & restore: Automated full database dumps and partial space exports with retention policies and scheduler
• Neo4j state monitoring: Single endpoint for consolidated database health, per-space statistics (nodes, edges, layers, health score, staleness), and backup overview
• Meta-cognition enforcement: Server-side anomaly detection (empty-resume, empty-recall), hook circuit breakers with CRITICAL warnings, multi-dimensional watchdog monitoring, Hebbian signal learning for adaptive enforcement
• Jiminy Inner Voice: Proactive guidance service injected into every prompt — surfaces constraints, prior corrections, contradictions, and frontier exploration opportunities from the knowledge graph via 4-source parallel fan-out under a config-derived guidance budget
• J17 AI-to-AI Communication Protocol: Three-tier encoding protocol (T1 coded ~15 tokens, T2 telegraphic, T3 full NL) for compact agent-to-agent communication. LLM-generated constraint codes, per-session trust scoring, signed session tickets for state persistence across context resets, RSIC-driven protocol evolution, and ML-powered tier prediction via neural sidecar
• Never-silent operations: a goroutine supervisor with panic recovery over every background loop, scheduled-job health that alerts on "failed OR never ran," hook-channel self-monitoring, restore-tested backups, and a server-native alert evaluator — engineered so a dead seam self-reports instead of failing quietly
• Closed guidance→feedback→outcome loop: Jiminy guidance is scored against what the agent actually did — per-constraint effectiveness rates feed RSIC calibration. The jiminy-governance Claude Code skill makes Jiminy a deterministic source of context and constraints over J17, with a fail-closed bash guard and graduated /strict-mode enforcement on edits
• Event-graph federation: mdemg eventgraph surfaces reinforcement and guidance-outcome events in a node's graph neighborhood by federating TimescaleDB event streams with Neo4j traversal — events stay in TSDB, never reified into the graph
• Space Transfer & DevSpace: Export/import space graphs as .mdemg files or via gRPC; optional DevSpace hub for agent registration, publish/pull exports, and inter-agent messaging (see cmd/space-transfer/README.md and docs/specs/development-space-collaboration.md)

MDEMG runs its own fine-tuned model for its internal LLM calls — the classification, guidance synthesis, concept emergence, intent translation, and re-ranking the graph depends on, across ~16 task surfaces. The shipped model is mdemg-llm-v1: a fine-tuned dense Qwen3-14B, distributed as fused GGUF quants and served locally via llama.cpp llama-server (an adapter-only pull is also available — see Step 2b below for installation). Each task has a fixed JSON output contract; the fine-tune exists to follow those contracts more reliably and cheaply than a stock base. The LLM provider is operator-configurable — MDEMG requires a reachable LLM endpoint to start.

The model improves through an operator-driven curate → train → benchmark → promote loop: every internal LLM interaction is captured (quality-filtered and sanitized) into TimescaleDB; mdemg data curate builds temporally-split datasets whose held-out test slice training never sees; candidates train via the local MLX LoRA pipeline and are benchmarked on the held-out slice (UBENCH); promotion is gated on measured lift and operator approval — a candidate that regresses is rejected, which is the gate doing its job. This is the offline outer loop of MDEMG's self-improvement, complementing RSIC's online micro-corrections; the fully-automated recursive-retraining loop is specced but not yet built, so today the loop runs operator-driven, not closed.

Feature docs: fine-tuning-pipeline.md, neural-training-pipeline.md, ubench-framework.md, local-model-distribution.md.

┌─────────────────────────────────────────────────────────────┐
│                      AI Coding Agent                        │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐  │
│  │   IDE/CLI    │◄──►│  MCP Server  │◄──►│ MDEMG Client │  │
│  │  (Cursor)    │    │   (tools)    │    │    (HTTP)    │  │
│  └──────────────┘    └──────────────┘    └──────┬───────┘  │
└─────────────────────────────────────────────────┼──────────┘
                                                  │
                    ┌─────────────────────────────▼─────────┐
                    │     MDEMG Service (dynamic port)      │
                    │  ┌─────────┐  ┌───────────────────┐  │
                    │  │Embedding│  │    Neo4j Graph    │  │
                    │  │ Provider│  │ (Vector + Graph)  │  │
                    │  └─────────┘  └───────────────────┘  │
                    └───────────────────────────────────────┘

• Go 1.24+
• Docker (for Neo4j)
• Embedding provider: OpenAI API key or Ollama (local, free)

git clone https://github.com/reh3376/mdemg.git && cd mdemg
go build -o bin/mdemg ./cmd/mdemg
./bin/mdemg --help            # See all commands

cd /path/to/your/project
mdemg init                    # Interactive wizard creates .mdemg/config.yaml + .env
mdemg db start                # Project-scoped Neo4j container (auto-selects free port)
mdemg start --auto-migrate    # Server daemon with schema migrations

mdemg ingest --space-id=my-project --path=/path/to/repo
mdemg ingest --space-id=my-project --path=/path/to/repo --incremental  # Changed files only
mdemg hooks install  # Auto-ingest on git commit

Requires UNTS_ENABLED=true and UNTS_BASE_PATH (default: .). Returns 503 when disabled.

A background auto-prune scheduler runs every 24 hours by default (SPACE_PRUNE_INTERVAL_HOURS, 0 to disable).

Requires AUTH_API_KEYS. Operator escape hatch when a breaker trips on a transient incident but hasn't auto-recovered. See docs/user/api-reference.md#admin-circuit-breakers.

For the comprehensive guide to UxTS methodology (architecture, spec writing, CI integration, governance, and the full framework family), see docs/guides/UXTS_DEVELOPER_GUIDE.md.

MDEMG extracts code symbols during ingestion using the Unified Parser Test Schema (UPTS):

Supported Languages (27 UPTS-validated, 100% pass rate):

• Systems: Go, Rust, C, C++, CUDA
• JVM: Java, Kotlin
• .NET: C#
• Scripting: Python, TypeScript/JavaScript, Lua, Shell
• API Schemas: Protocol Buffers, GraphQL, OpenAPI
• Configuration: YAML, TOML, JSON, INI
• Infrastructure: Terraform/HCL, Dockerfile, Makefile
• Database: SQL, Cypher (Neo4j)
• Documentation: Markdown, XML, Scraper Markdown (web-scraped content with section chunking)

Extracted Symbol Types:

• Functions, methods, classes, interfaces, types
• Constants, variables, enums
• Imports, exports, module declarations

Symbols include file:line references for evidence-based retrieval.

CMS provides persistent memory for AI agents across sessions:

# Resume session context (call at session start)
curl -X POST http://localhost:9999/v1/conversation/resume \
  -H "Content-Type: application/json" \
  -d '{"space_id": "my-agent", "session_id": "session-1", "max_observations": 10}'

# Record an observation (decision, learning, correction)
curl -X POST http://localhost:9999/v1/conversation/observe \
  -H "Content-Type: application/json" \
  -d '{
    "space_id": "my-agent",
    "session_id": "session-1",
    "content": "User prefers TypeScript for new files",
    "obs_type": "preference"
  }'

Observation Types: decision, correction, learning, preference, error, progress

Observations are surprise-weighted (novel information persists longer) and form themes via consolidation.

MDEMG provides an MCP server for IDE integration.

mdemg sidecar attach-agent claude-code   # Writes .claude/mcp.json + enables MCP in settings
# Or as part of full setup:
mdemg sidecar quickstart                 # init + install + up + attach + hooks

Add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "mdemg": {
      "command": "/path/to/mdemg/bin/mdemg-mcp",
      "args": [],
      "env": {
        "MDEMG_ENDPOINT": "http://localhost:9999"
      }
    }
  }
}

mdemg/
├── cmd/                  # CLI tools (server, ingest, MCP, etc.)
├── internal/             # Core logic
│   ├── api/              # HTTP handlers
│   ├── retrieval/        # Search algorithms
│   ├── hidden/           # Concept abstraction
│   ├── learning/         # Hebbian edges
│   ├── conversation/     # Conversation Memory System (CMS)
│   ├── backup/           # Backup & restore (full, partial, scheduler, retention)
│   ├── symbols/          # Code symbol extraction
│   └── plugins/          # Plugin system
├── docs/                 # Documentation
│   └── benchmarks/       # Benchmark questions, graders, results
├── migrations/           # Neo4j schema (Cypher)
├── tests/                # Integration tests
└── docker-compose.yml    # Neo4j container

MDEMG includes a complete observability stack for monitoring and debugging. The TimescaleDB telemetry plane that feeds it is governed end-to-end — every hypertable carries retention and compression policies, and the alert evaluator runs server-native (Grafana is dashboards-only, not required for alerting); see docs/features/tsdb-data-management.md.

cd deploy/docker
docker compose -f docker-compose.observability.yml up -d

# Access dashboards
open http://localhost:3000  # Grafana (admin/admin)

Pre-configured dashboard with 10 panels:

• Request Rate, P95 Latency, Error Rate, Circuit Breakers
• Request Latency Distribution (p50/p95/p99)
• Requests by Status, Cache Hit Ratios
• Retrieval Latency, Rate Limit Rejections, Embedding Latency

curl http://localhost:9999/v1/metrics/snapshot

Returns a JSON metrics snapshot including counters, gauges, and histograms from the MetricsRecorder. Metrics are persisted to TimescaleDB.

| 98 | Cross-Platform Build + Release (goreleaser, Homebrew, self-update) | ✅ Complete | | 99 | Onboarding + Polish (quickstart, demo, FAQ) | ✅ Complete |

See AGENT_HANDOFF.md for detailed phase specifications.

• CLI Implementation - CLI internals and command structure
• API Implementation - Internal API documentation
• Architecture - System design and components
• Graph Schema - Neo4j labels and relationships
• Retrieval & Scoring - Scoring algorithm details
• Benchmarking Guide - Running and validating benchmarks
• Backup & Restore Guide - Backup configuration and retention
• Agent Handoff - Complete development context and phase registry

See CONTRIBUTING.md for development setup and guidelines.

See SECURITY.md for the vulnerability reporting policy.

MIT License - see LICENSE for details.