ox-whisper — Self-Hosted Whisper API Alternative

Self-hosted, OpenAI-compatible speech-to-text (STT) HTTP server in Rust. Drop-in replacement for the OpenAI Whisper API — runs on a single ARM64 CPU, no GPU. 8 languages. Real-time WebSocket streaming. Word-level timestamps. Built on sherpa-onnx + Moonshine v2.

Built for voice AI agents, live captioning, edge deployments, and privacy-sensitive transcription on Oracle Free Tier / Hetzner CAX / Raspberry Pi 5.

Quick start · API · Languages · Benchmarks · Metrics · Limitations

ox-whisper wins on a narrow but real wedge: a polished HTTP/WebSocket server with OpenAI-compatible API, sub-100 ms RTF on a CPU, no GPU, ARM64-native. Pair it with Pipecat / LiveKit / Vapi for self-hosted voice agents.

curl -fsSL https://raw.githubusercontent.com/anatolykoptev/ox-whisper/master/install.sh | bash

Linux aarch64. Docker auto-installed if missing. Pulls ghcr.io/anatolykoptev/ox-whisper:latest, fetches ~463 MB models, starts on :8092 (HTTP/WS) + :9092 (metrics).

curl http://localhost:8092/health
curl http://localhost:8092/v1/models

docker run -d --name ox-whisper --restart unless-stopped \
  -p 127.0.0.1:8092:8092 -p 127.0.0.1:9092:9092 \
  -v $(pwd)/models/en:/models:ro \
  -v $(pwd)/models/ru:/ru-models:ro \
  -v $(pwd)/models/vad:/vad:ro \
  -v $(pwd)/models/punct-en:/punct:ro \
  ghcr.io/anatolykoptev/ox-whisper:latest

services:
  ox-whisper:
    image: ghcr.io/anatolykoptev/ox-whisper:latest
    restart: unless-stopped
    ports:
      - "127.0.0.1:8092:8092"
      - "127.0.0.1:9092:9092"
    environment:
      MOONSHINE_THREADS: "4"
      POOL_SIZE: "2"
    volumes:
      - ./models/en:/models:ro
      - ./models/ru:/ru-models:ro
      - ./models/vad:/vad:ro
      - ./models/punct-en:/punct:ro

curl -fsSL https://raw.githubusercontent.com/anatolykoptev/ox-whisper/master/scripts/download-models.sh | bash

Drop-in replacement for openai.audio.transcriptions.create. Point your OpenAI SDK at http://localhost:8092/v1.

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8092/v1", api_key="unused")
result = client.audio.transcriptions.create(
    model="whisper-1", file=open("recording.mp3", "rb"), language="en"
)
print(result.text)

import asyncio, websockets, json
async def stream():
    async with websockets.connect("ws://localhost:8092/v1/listen?language=en") as ws:
        await ws.send(pcm_16khz_mono_chunk)  # repeat
        async for msg in ws:
            print(json.loads(msg))  # {"type":"partial"|"final","text":"..."}
asyncio.run(stream())

curl -X POST http://localhost:8092/transcribe \
  -H 'Content-Type: application/json' \
  -d '{"audio_path":"/data/recording.wav","language":"en"}'

Response: { "text", "duration_ms", "words":[{"word","start","end","confidence"}], "confidence" }. Optional fields: vad, punctuate, max_chunk_len.

curl -F file=@recording.mp3 -F language=en http://localhost:8092/transcribe/upload

curl -N -F file=@long.mp3 -F language=en http://localhost:8092/transcribe/stream
# data: {"index":0,"text":"...","type":"chunk"}
# data: {"type":"done"}

Need 99 languages? Use whisper-large-v3 instead — ox-whisper trades coverage for speed and CPU footprint.

Oracle Cloud A1 free tier, 4 ARM threads, CPU-only. Best of 3 runs after warmup.

For comparison on the same box: faster-whisper tiny int8 ~0.075 RTF, whisper.cpp tiny-q8_0 ~0.13 RTF.

Downloaded by scripts/download-models.sh (run automatically by install.sh).

Prometheus exposition on port 9092. Scrape:

scrape_configs:
  - job_name: ox-whisper
    static_configs:
      - targets: ['ox-whisper:9092']

• aarch64 only. Pre-built .so libs are ARM64; x86_64 needs source rebuild of vendor/sherpa-rs-sys.
• No streaming for /transcribe. Whole-file responses only. Use /transcribe/stream (SSE) or /v1/listen (WebSocket) for incremental output.
• No auth. Bind to 0.0.0.0; put behind nginx / Caddy if exposed to the network.
• 8 languages. For broader coverage, use whisper-large-v3 via faster-whisper or speaches.
• EN punctuation only via the CNN-BiLSTM model. RU punctuation comes from GigaAM v3 when loaded.

git clone https://github.com/anatolykoptev/ox-whisper && cd ox-whisper
cargo build --release          # native aarch64
docker build -t ox-whisper .   # BuildKit + cargo-chef layer cache

CI publishes ghcr.io/anatolykoptev/ox-whisper:vX.Y.Z on every v* tag.

MIT. Built on sherpa-onnx, Moonshine, Silero VAD.