Docker-based AI toolkit for real-time speech-to-text, audio transcription, and LLM-powered summarization—all running locally with no cloud dependencies.

• 🎤 Live Captions - Real-time speech-to-text overlay for meetings, videos, and calls
• 📝 Transcription - Convert audio to searchable text with high accuracy
• 🤖 AI Summarization - Automatically extract key points and create summaries
• 💬 Chat with Transcripts - Ask questions about your content using local LLMs
• 🔒 100% Local - No cloud APIs, no data sent elsewhere, full privacy

# Start all services
docker compose up -d audio-notes ollama whisper-asr

# Access the web UI
open http://localhost:7860

Platform: Windows 10/11 (required for Live Captions desktop app)

GPU Notes:

• Vosk: CPU only, works on any system
• Parakeet/Whisper: Requires NVIDIA GPU with CUDA support
• VRAM usage: ~4-6GB per ASR model loaded
• Multiple GPU models can run simultaneously with 12GB+ VRAM

This is the typical workflow showing how Live Captions and Audio Notes work together.

Launch Live Captions from system tray and configure your settings:

• Audio Source: Choose "System Audio" (for videos/meetings) or "Microphone" (for your voice)
• ASR Model: Select Whisper (accurate), Parakeet (fast), or Vosk (CPU)
• Enable Recording: Turn on to save audio for later transcription
• Live Transcription: Enable to see real-time captions

Play a video, join a meeting, or speak into your microphone:

Live Captions displays real-time transcription as audio plays:

• Captions appear in a floating overlay window
• The window shows "Speak now..." when ready to capture
• Recording saves audio files automatically when enabled

Open Audio Notes at http://localhost:7860 to process your recordings:

1. Select Recording - Your saved recordings appear in the Recordings panel
2. Choose ASR Backend - Whisper (accurate) or Parakeet (fast)
3. Click Transcribe - Generates full text transcript
4. View Tabs:
   • Full Transcript - Complete searchable text
   • Summary - AI-generated summary of key points
   • Chat - Ask questions about the content

Use the Chat tab to interact with your transcript:

• "What are the main topics discussed?"
• "Summarize the key action items"
• "What did the speaker say about machine learning?"
• "Create a study guide from this content"

Result: Audio → Real-time captions → Saved recording → Transcript → Summary → Interactive Q&A

Notes:

• Transformer: Encoder-decoder architecture (Whisper)
• FastConformer: Optimized Conformer with cache-aware streaming (FastConformer)
• Conformer: CNN + Transformer hybrid for better audio modeling (Parakeet)
• TDT-CTC: Time-delay Transformer with CTC loss for streaming
• RNN-T: Recurrent Neural Network Transducer for offline transcription
• Hybrid RNNT/CTC: Multitask training with both decoders for better accuracy
• TDNN-LSTM: Time-Delay Neural Network with LSTM (Vosk)

GPU Memory Usage:

• Single ASR model: 1.5-4GB VRAM (varies by model)
• Running 2 models simultaneously: 3.5-6GB VRAM
• All services + LLM: 10-14GB VRAM recommended

Recommended for Streaming:

• FastConformer: Best balance of latency and accuracy (WER 5.4%, 240ms avg latency)
• Parakeet TDT: Good for cache-aware streaming (WER ~6%, higher latency)
• Whisper: Best for multilingual, but higher latency

AI-Tools/
├── apps/live-captions/     # Desktop tray app for live captions
├── services/
│   ├── audio-notes/        # Web UI (Gradio)
│   ├── fastconformer/      # FastConformer ASR service (streaming-only)
│   ├── whisper/            # Whisper ASR service
│   ├── parakeet/           # Parakeet ASR service
│   ├── vosk/               # Vosk ASR service
│   └── text-refiner/       # Text post-processing
├── shared/                 # Common utilities
├── integration/            # End-to-end tests
└── docker-compose.yaml     # Service orchestration

# Run all tests
python -m pytest apps/ services/ shared/ integration/ -v

# Run with coverage
python -m pytest --cov=apps --cov=services --cov=shared

# Lint code
ruff check apps/ services/ shared/

MIT License - see LICENSE file for details.