Go Embedded AI Memory + Knowledge Graph Database
gracedb is a Go embedded AI memory and knowledge graph database built on a pluggable KV backend (Badger by default). It provides vector search, full-text search, knowledge management, session management, property graphs, RDF/SPARQL, and MCP services — all in a single, zero-external-dependency library.
gracedb's core engine is mature: the storage, vector search, FTS, semantic memory, graph, RDF, and MCP surfaces are implemented and covered by nearly ten thousand lines of Go tests and benchmarks. The advanced AI workflows built on top are still simplified or caller-extended, and the whole system has not yet been validated under production load. Treat the project as suitable for local applications, experiments, and controlled internal use before hardening it for production.
Embedder interface, text auto-vectorization, Quick API, vector CRUD, HNSW/IVF/Flat/LSH indexes, full-text search, metadata filtering, semantic Agent Memory, knowledge storage, session/document management, backup/restore, and OpenTelemetry hooks.
| Area | Current state |
|---|---|
| LLM-driven entity extraction | Built-in extraction is heuristic; callers can inject LLM extractors. |
| LLM Reflect | Built-in reflection is rule-based; callers can inject LLM reflectors. |
| SPARQL | SELECT/ASK with FILTER (comparison), ORDER BY, OFFSET, LIMIT, {...} UNION {...} alternation, OPTIONAL { } left-outer-join groups, GROUP BY with seven aggregate functions (COUNT/SUM/MIN/MAX/AVG/GROUP_CONCAT/SAMPLE), AS alias projection and HAVING group filtering. Full SPARQL 1.1 (aggregates over subqueries, federated queries, etc.) is not yet supported. |
| GraphRAG workflows | Useful pipeline primitives, not a full managed RAG product. |
| Index recovery | Stored vectors are durable and searchable after reopen; indexes load lazily on first search (from snapshot or rebuilt from vectors), auto-rebuilding from vectors if a snapshot is corrupt; production deployments should still add crash/recovery validation for their workload. |
| Document | Content |
|---|---|
| Getting Started | Installation, configuration, quick start, FAQ |
| API Reference | Complete API, data types, storage key formats |
| Architecture | Layered architecture, data flow, index system, extension points |
| Module Guide | Package responsibilities, key types, dependency graph |
| Implementation Plan | Phased development roadmap |
| Advanced Memory | Memory strength model, consolidation, layered wake-up, FTS rebuild |
| 中文版 | 中文文档 |
- Vector Search — HNSW / IVF / Flat / LSH indexes with cosine similarity; HNSW falls back to an exact scan on small collections for deterministic results
- Full-Text Search — Chinese segmentation + stop words + Levenshtein fuzzy + RRF hybrid fusion
- Knowledge Storage — auto-chunking + FTS indexing + document-level aggregation
- Agent Memory — scope/namespace/TTL isolation with semantic + lexical hybrid retrieval
- Property Graph — node/edge CRUD, BFS/DFS/shortest path traversal
- RDF/SPARQL — N-Triples import/export, SPARQL SELECT/ASK, GROUP BY aggregation, RDFS inference, SHACL validation
- GraphRAG Toolbox — 9 built-in tools for LLM orchestration
- MCP Service — full Model Context Protocol server (tools + resources + prompts + capability negotiation), stdio transport
- Backup/Restore — protocol-agnostic snapshot, restorable across storage engines
- OpenTelemetry — automatic spans and metrics on core operations
- KnowledgeMemory — fused memory + knowledge recall with reflection and consolidation
- Auto-Retain — automatic fact extraction during conversation
go get github.com/dshmyz/gracedbpackage main
import (
"fmt"
"log"
"github.com/dshmyz/gracedb/pkg/gracedb"
)
func main() {
// Open database
db, err := gracedb.Open("/tmp/gracedb-data")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Create collection
coll, _ := db.CreateCollection("my_docs")
fmt.Println("created:", coll.Name)
// Insert vector
vec := []float32{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8}
embID, _ := db.Upsert("my_docs", "doc-1", vec, "Hello world", nil, nil)
fmt.Println("embedded:", embID)
// Search
results, _ := db.Search("my_docs", vec, gracedb.SearchOptions{
TopK: 5,
UseVectorSearch: true,
})
fmt.Println("found:", len(results), "results")
// Backup
db.Backup("/tmp/gracedb-backup.db")
// Quick API
q := db.Quick()
id, _ := q.AddToCollection(ctx, "my_docs", vec, "Quick add")
fmt.Println("quick add:", id)
}db, _ := gracedb.Open("/tmp/data",
gracedb.WithIndexType("hnsw"), // hnsw / ivf / flat / lsh
gracedb.WithIndexTypes([]string{"hnsw", "lsh"}), // multi-index hybrid
gracedb.WithSimilarity("cosine"), // cosine / euclidean
gracedb.WithEmbedder(myEmbedder), // types.Embedder interface
)┌─────────────────────────────────────────────┐
│ gracedb.DB │ ← Facade
│ Quick / Toolbox / Backup / Trace / Ontology │
├─────────────────────────────────────────────┤
│ KnowledgeMemory │ ← Recall/Reflect/Consolidate
│ AutoRetain / GroupAggregate │
├─────────────────────────────────────────────┤
│ BadgerStore │ ← Persistence (with in-memory index)
│ CRUD / Search / FTS / Index / Aggregation │
├─────────────────────────────────────────────┤
│ GraphStore / RDF │ ← Graph engine
│ Nodes/Edges/Traversal/SPARQL/RDFS/SHACL │
├─────────────────────────────────────────────┤
│ KVBackend interface │ ← Pluggable storage seam
│ Update / View / Backup / Restore / Sync │
├─────────────────────────────────────────────┤
│ badgerKV | memkv (in-memory) │ ← Storage engines
│ LSM-tree / MVCC / ACID | zero-dep │
└─────────────────────────────────────────────┘
The storage engine is fully pluggable through the KVBackend interface (pkg/store/kv.go). By default gracedb uses badgerKV (Badger v4); pass store.NewMemKV() via WithKVBackend (pkg/gracedb/db.go) to run on a pure in-memory engine:
db, _ := gracedb.Open("/tmp/data", gracedb.WithEmbedder(myEmbedder))
// Insert text (auto-vectorized)
id, _ := db.InsertText("docs", "text-1", "This is Chinese text", nil)
// Text search (vectorized or FTS fallback)
results, _ := db.SearchText("docs", "Chinese", 10)// Save knowledge (auto-chunked)
record, _ := db.SaveKnowledge("docs", "wiki-1", "Go Language",
"Go is a statically typed, compiled language...",
types.KnowledgeSaveRequest{ChunkSize: 500, ChunkOverlap: 50})
// Search knowledge
resp, _ := db.SearchKnowledge("docs", "programming language", 5)// Save memory with scope/namespace
db.SaveMemory(types.MemorySaveRequest{
MemoryID: "mem-1",
Content: "User prefers Go over Python",
Scope: "user",
UserID: "user-123",
Namespace: "preferences",
TTLSeconds: 3600,
})
// Search memory
resp, _ := db.SearchMemory(types.MemorySearchRequest{
Query: "prefers",
UserID: "user-123",
Scope: "user",
TopK: 5,
})g := db.Graph()
g.UpsertNode(&graph.GraphNode{
ID: "person-1", Type: "person",
Properties: map[string]string{"name": "Alice"},
})
g.UpsertNode(&graph.GraphNode{
ID: "lang-1", Type: "language",
Properties: map[string]string{"name": "Go"},
})
g.UpsertEdge(&graph.GraphEdge{
FromNodeID: "person-1", ToNodeID: "lang-1",
Type: "likes", Weight: 1.0,
})
// Query neighbors
nodes, edges, _ := g.GetNeighbors("person-1", graph.NeighborOptions{
Direction: "out", Limit: 10,
})
// Traversal
result, _ := g.BFS("person-1", graph.NeighborOptions{MaxDepth: 2})rdf := db.RDF()
// Insert triple
rdf.UpsertTriple(&rdf.Triple{
Subject: rdf.NewIRI("http://example.org/person/1"),
Predicate: rdf.NewIRI("http://example.org/likes"),
Object: rdf.NewIRI("http://example.org/lang/go"),
})
// SPARQL query
results, _ := rdf.SPARQLSelect(`SELECT ?s ?p ?o WHERE { ?s ?p ?o . }`)
// Solution modifiers: FILTER, ORDER BY, LIMIT
top, _ := rdf.SPARQLSelect(
`SELECT ?person ?name WHERE { ?person <http://schema.org/name> ?name FILTER(?name >= "A") } ORDER BY DESC(?name) LIMIT 3`)
// ASK query
exists, _ := rdf.SPARQLAsk(`ASK WHERE { <http://example.org/person/1> ?p ?o . }`)
// GROUP BY aggregation: count items per type, keep groups with count > 1
counts, _ := rdf.SPARQLSelect(`
SELECT ?type (COUNT(?item) AS ?c) WHERE {
?item <http://ex.com/type> ?type
} GROUP BY ?type HAVING (COUNT(?item) > 1)`)o := db.Ontology()
// Define ontology
o.DefineClass("http://example.org/Person", "")
o.DefineClass("http://example.org/Developer", "http://example.org/Person")
o.DefineProperty("http://example.org/knows", "http://example.org/Person", "http://example.org/Person")
// Add facts
o.AddFact("http://example.org/person/alice", "http://example.org/knows", "Bob")
// RDFS inference (materialize new triples)
count, _ := o.Infer()
// SHACL validation
report, _ := o.Validate()km := db.KnowledgeMemory(nil) // nil = use rule-based reflector
// Recall: fused memory + knowledge + graph expansion
resp, _ := km.Recall(ctx, knowledge.KnowledgeMemoryRecallRequest{
Query: "what does the user like?",
TopKMemories: 5,
TopKKnowledge: 4,
MaxHops: 2, // graph expansion depth
})
// Reflect: synthesize structured summary
reflection, _ := km.Reflect(ctx, knowledge.KnowledgeMemoryReflectRequest{
Query: "user preferences",
})
fmt.Println("Summary:", reflection.Summary)
fmt.Println("Themes:", reflection.Themes)
// Consolidate: store summary + optionally promote to knowledge
consolidated, _ := km.Consolidate(ctx, knowledge.KnowledgeMemoryConsolidateRequest{
Reflect: knowledge.KnowledgeMemoryReflectRequest{
Query: "user preferences",
},
PromoteToKnowledge: true,
})db.SetFactExtractor(func(ctx context.Context, msgs []*types.Message) ([]gracedb.ExtractedFact, error) {
// Extract facts from conversation (can use LLM here)
return []gracedb.ExtractedFact{
{ID: "fact-1", Content: "User likes Go", Type: "preference"},
}, nil
})
db.SetAutoRetain(gracedb.AutoRetainConfig{
Enabled: true,
WindowSize: 6,
TriggerEvery: 2, // extract every 2 messages
})
// AddMessage now triggers extraction automatically
db.AddMessage(&types.Message{SessionID: "sess-1", Role: "user", Content: "I like Go"})
db.AddMessage(&types.Message{SessionID: "sess-1", Role: "assistant", Content: "Go is great!"})
// → AutoRetain fires, extracts facts, stores as memory// Simple aggregation
result, _ := db.Aggregate("docs", "score", store.AggAvg)
fmt.Printf("Average score: %.2f\n", result.Avg)
// GROUP BY aggregation
groups, _ := db.GroupAggregate("docs", "category", "price", store.AggAvg)
for category, r := range groups {
fmt.Printf("%s: avg=%.2f, count=%d\n", category, r.Avg, r.Count)
}db.MCPServer() composes the tools, resources and prompts clusters into one
complete Model Context Protocol server — the drop-in entry point for standard
MCP clients:
server := db.MCPServer()
server.RunStdio(context.Background())A full server exposes:
- Tools — the 15-tool GraphRAG toolbox (
search_knowledge,save_memory,vector_search,sparql_query, …), callable from any MCP client. - Resources — addressable
gracedb://URIs read live on demand, e.g.gracedb://memory/{id},gracedb://knowledge/{collection}/{id},gracedb://graph/node/{id},gracedb://rdf/triple/{id}andgracedb://collections. - Prompts — four templates (
memory_augmented_reply,knowledge_qa,consolidation,graph_rag_expand) that render to message sequences with live database context. - Capabilities — an
initializehandshake advertises the enabled clusters; passgracedb.WithCapabilities(tools, resources, prompts bool)when opening the DB to trim them.
For a tools-only server (backward compatible), keep using
db.NewMCPServer(name, version).
db.Backup("/tmp/backup.db")
db2, _ := gracedb.Open("/tmp/restored")
db2.Restore("/tmp/backup.db")// Preload index (from snapshot or rebuild) — optional
db.LoadIndex("docs")
// Save index snapshot
db.SaveIndex("docs")
// Rebuild FTS index
db.RebuildIndex("docs")Vector indexes are loaded lazily: the first search on a collection auto-builds
the in-memory index from its persisted snapshot (or rebuilds from stored vectors),
so forgetting to call LoadIndex no longer silently degrades to a slow full scan.
LoadIndex is only needed if you want the index warm before the first query.
| Key Pattern | Purpose |
|---|---|
coll:<name> |
Collection metadata |
emb:<cid>:<eid> |
Embedding metadata |
emb:vec:<cid>:<eid> |
Vector data |
fts:<token>:<cid>:<eid> |
FTS inverted index (value = TF count) |
mem:<bucket>:<id> |
Memory metadata |
mem:content:<bucket>:<id> |
Memory content |
mem:vec:<bucket>:<id> |
Memory semantic vector |
mem:fts:<token>:<bucket>:<id> |
Memory lexical inverted index |
mem:idx:<id> |
Memory ID to bucket index |
graph:node:<id> |
Graph node |
graph:edge:<id> |
Graph edge |
rdf:t:<id> |
RDF triple |
idx:snapshot:<cid> |
Vector index snapshot |
sess:<id> |
Session |
go run examples/main.go # full API tour
go run examples/llm_reflector # wire your own LLM KnowledgeMemoryReflector
go run examples/llm_extractor # wire your own LLM entity/relation extractor
go run examples/sparql_modifiers # SPARQL FILTER / ORDER BY / LIMIT via embedded Go APIgo test ./... # All tests
go test -v ./pkg/index/ # Verbose output
go test -bench=. ./pkg/store/ # Benchmarksgracedb is inspired by and built upon the work of these excellent projects:
- CortexDB — gracedb targets CortexDB feature parity as its design reference. The KnowledgeMemory, AutoRetain, GraphRAG toolbox, and MemoryFlow workflows are all modeled after CortexDB's architecture and APIs.
- Badger — gracedb's default storage engine via the
badgerKVbackend. Badger's high-performance LSM-tree, MVCC, and ACID transaction support make gracedb's embedded design possible. TheKVBackendinterface allows swapping it for other engines (e.g. the in-memorymemkv). - 智算多多 (zsdodo) — 致谢智算多多(zsdodo)提供算力支持。
MIT