Local-first orthopedic clinical intelligence for clinicians. OrthoAI provides attending-level decision support for orthopedic workflows including clinical consults, surgical planning, complications/risk assessment, imaging diagnosis, rehabilitation/return-to-play protocols, and evidence briefs.

Built with privacy-first principles using local LLM inference (llama.cpp) and persistent memory with RAG (ChromaDB + SQLite).

OrthoAI automatically detects context and adapts to seven specialized workflows:

• Multi-Model Strategy System: Intelligent model selection with ensemble voting, model chaining, and ML-driven quality prediction
• RAG + Persistent Memory: Long-term conversation memory with ChromaDB vector store and SQLite
• Knowledge Base: Ingest and retrieve orthopedic literature (PDF parsing, chunking, embeddings)
• Patient Case Management: Track cases with demographics, history, imaging, labs, and linked conversations
• DICOM Viewer: View and annotate medical imaging with Cornerstone.js integration
• Medical Coding: ICD-10, CPT code lookup, and drug reference database
• Voice Interface: Speech-to-text input and text-to-speech output
• Learning System: Quality prediction, pattern recognition, and parameter tuning based on feedback
• Code Execution: Sandboxed Python/JavaScript execution via Pyodide for calculations and analysis

OrthoAI/
├── app/
│   ├── api/              # Next.js API routes
│   │   ├── llm/          # LLM chat endpoint
│   │   ├── memory/       # Memory and consent APIs
│   │   ├── knowledge/    # Knowledge base APIs
│   │   ├── cases/        # Case management APIs
│   │   ├── codes/        # Medical coding APIs
│   │   ├── analytics/    # Usage analytics
│   │   └── ...
│   ├── lib/
│   │   ├── strategy/     # Multi-model orchestration
│   │   ├── domain/       # Mode detection and system prompts
│   │   ├── memory/       # SQLite + RAG memory system
│   │   ├── knowledge/    # Knowledge base and PDF parsing
│   │   ├── cases/        # Case management
│   │   ├── imaging/      # DICOM handling
│   │   ├── codes/        # ICD-10, CPT, drugs
│   │   ├── learning/     # ML-based quality prediction
│   │   ├── voice/        # Voice input/output
│   │   ├── tools/        # Calculator, code execution
│   │   └── llm/          # LLM configuration
│   └── pages/            # Next.js pages
├── components/           # React components
│   ├── Chat.tsx          # Main chat interface
│   ├── CaseForm.tsx      # Case management UI
│   ├── DicomViewer.tsx   # DICOM image viewer
│   ├── LearningDashboard.tsx  # Analytics dashboard
│   └── ...
└── .data/                # Local data storage
    ├── orthoai.db        # SQLite database
    └── chroma/           # ChromaDB vector store

• Node.js 18+
• llama.cpp (for local LLM inference)
• Docker Desktop (for ChromaDB)
• Hugging Face account (for model downloads)

git clone <repo-url>
cd OrthoAI
npm install

brew install llama.cpp

llama-server \
  --hf-repo tensorblock/BioMistral-7B-GGUF \
  --hf-file BioMistral-7B-Q4_K_M.gguf \
  --alias biomistral-7b-instruct \
  --port 8080

Verify the server is running:

curl http://127.0.0.1:8080/v1/models

llama-server \
  --hf-repo nomic-ai/nomic-embed-text-v1.5-GGUF \
  --hf-file nomic-embed-text-v1.5.Q5_K_M.gguf \
  --embedding \
  --alias nomic-embed-text \
  --port 8081

Create .env.local in the project root:

# LLM Configuration
LLM_BASE_URL=http://127.0.0.1:8080/v1
LLM_DEFAULT_MODEL=biomistral-7b-instruct
EMBEDDING_BASE_URL=http://127.0.0.1:8081/v1
EMBEDDING_MODEL=nomic-embed-text

# Storage Configuration
MEMORY_DB_PATH=./.data/orthoai.db
CHROMA_HOST=localhost
CHROMA_PORT=8000
CHROMA_DB_PATH=./.data/chroma

npm run chroma-start

npm run dev

Open http://localhost:3000

Primary (clinical decision support):

• BioMistral 7B (Q4_K_M or Q5_K_M quantization)

Embeddings:

• Nomic Embed Text v1.5

Multi-model orchestration with:

• Workflow Strategy: Combined ensemble + chain execution
• Quality Prediction: ML-based response quality forecasting
• Parameter Tuning: Automatic temperature/token optimization
• A/B Testing: Strategy experiment framework

Persistent conversation memory:

• SQLite Storage: Conversations, messages, summaries, user profiles
• RAG Retrieval: Semantic search via ChromaDB embeddings
• Auto-Summarization: Periodic conversation summaries for long-term recall
• Profile Memory: User preferences with explicit consent

Automatic context detection:

• Mode classification based on keywords and patterns
• Domain-specific system prompts
• Complexity scoring for model selection

Literature management:

• PDF parsing and chunking
• Document metadata (subspecialty, diagnosis tags)
• Hybrid search (FTS + vector similarity)

Continuous improvement:

• Pattern recognition for successful interactions
• Quality prediction before generation
• Parameter optimization based on outcomes

• Framework: Next.js 16 (App Router)
• UI: React 19, Tailwind CSS 4
• LLM: llama.cpp (OpenAI-compatible API)
• Vector Store: ChromaDB
• Database: SQLite (better-sqlite3)
• Imaging: Cornerstone.js, dicom-parser
• Math: math.js
• Code Execution: Pyodide (Python in browser)
• 3D Rendering: Three.js
• Validation: Zod

OrthoAI is designed for local-first operation:

• All LLM inference runs locally via llama.cpp
• Data stored locally in SQLite and ChromaDB
• No external API calls for core functionality
• Explicit consent required for profile memory features

See DISCLAIMER.md. OrthoAI is not medical advice and does not replace independent clinical judgment.

See LICENSE.md for details.