parakeet-rs

Fast speech recognition with NVIDIA's Parakeet models via ONNX Runtime.

Note: CoreML is unstable with this model. For Apple, use WebGPU EP (uses metal under the hood,dont confuse by its name :-). it's a native GPU standard, not only web) or CPU. But even CPU alone is significantly faster on my Mac M3 16GB compared to Whisper metal! :-)

CTC (English-only):

use parakeet_rs::{Parakeet, Transcriber, TimestampMode};

let mut parakeet = Parakeet::from_pretrained(".", None)?;

// Load and transcribe audio (see examples/raw.rs for full example)
let result = parakeet.transcribe_samples(audio, 1600, 1, Some(TimestampMode::Words))?;
println!("{}", result.text);

// Token-level timestamps
for token in result.tokens {
    println!("[{:.3}s - {:.3}s] {}", token.start, token.end, token.text);
}

TDT (Multilingual): 25 languages with auto-detection

use parakeet_rs::{ParakeetTDT, Transcriber, TimestampMode};

let mut parakeet = ParakeetTDT::from_pretrained("./tdt", None)?;
let result = parakeet.transcribe_samples(audio, 16000, 1, Some(TimestampMode::Sentences))?;
println!("{}", result.text);

// Token-level timestamps
for token in result.tokens {
    println!("[{:.3}s - {:.3}s] {}", token.start, token.end, token.text);
}

EOU (Streaming): Real-time ASR with end-of-utterance detection

use parakeet_rs::ParakeetEOU;

let mut parakeet = ParakeetEOU::from_pretrained("./eou", None)?;

// Prepare your audio (Vec<f32>, 16kHz mono, normalized)
let audio: Vec<f32> = /* your audio samples */;

// Process in 160ms chunks for streaming
const CHUNK_SIZE: usize = 2560; // 160ms at 16kHz
for chunk in audio.chunks(CHUNK_SIZE) {
    let text = parakeet.transcribe(chunk, false)?;
    print!("{}", text);
}

Nemotron (Streaming): Cache-aware streaming ASR with punctuation. Two variants share the same API — point from_pretrained at whatever directory holds the ONNX files and the loader auto-detects which variant it is:

• English-only 0.6B — verbatim English, preserves disfluencies (um, uh). Best for transcription where every spoken word matters.
• Multilingual 3.5 0.6B — 40 language-locales across 3 tiers (19 transcription-ready, 13 broad-coverage, 8 that need fine-tuning to reach production quality based on the NVIDIA). Polished output (proper casing/punctuation, drops disfluencies). Same speed and size as the English-only model.

use parakeet_rs::{Nemotron, NemotronMode};

let mut model = Nemotron::from_pretrained(path, None)?;

// Multilingual variant: optionally pick a target language. Defaults to "auto"
// (the model picks the language itself). Pass a specific code when you know
// the language it's strictly more accurate. No-op when an English-only model is loaded.
if model.mode() == NemotronMode::Multilingual {
    model.set_target_lang("es-ES")?; // also: "ja-JP", "tr-TR", "auto", ...
}

// Process in 560ms chunks for streaming
const CHUNK_SIZE: usize = 8960; // 560ms at 16kHz
for chunk in audio.chunks(CHUNK_SIZE) {
    let text = model.transcribe_chunk(chunk)?;
    print!("{}", text);
}

Cohere Transcribe (Offline Multilingual): 14 languages, punctuation & ITN toggles (yes, "parakeets🦜" talk about more than just NVIDIA right?? :-P)

parakeet-rs = { version = "0.3", features = ["cohere"] }

use parakeet_rs::CohereASR;

let mut model = CohereASR::from_pretrained("./cohere", None)?;

// audio: Vec<f32>, 16kHz mono (long-form supported)
let text = model.transcribe_audio(&audio, "en", true, false)?; // lang, pnc, itn
println!("{}", text);

See examples/cohere.rs for a runnable demo.

Multitalker (Streaming Multi-Speaker ASR): Speaker-attributed transcription

parakeet-rs = { version = "0.3", features = ["multitalker"] }

use parakeet_rs::MultitalkerASR;

let mut model = MultitalkerASR::from_pretrained(
    "./multitalker",             // encoder, decoder, tokenizer
    "nemotron3_diar_v3.onnx",    // Nemotron-3 (Sortformer v3) for diarization
    None,
)?;

for chunk in audio.chunks(17920) {  // ~1.12s at 16kHz
    let results = model.transcribe_chunk(chunk)?;
    for r in &results {
        println!("[Speaker {}] {}", r.speaker_id, r.text);
    }
}

See examples/multitalker.rs for full usage with latency modes.

Nemotron-3 Diarization (Sortformer v3): Streaming diarization, up to 8 speakers

parakeet-rs = { version = "0.3", features = ["sortformer"] }

use parakeet_rs::sortformer::{Sortformer, StreamingProfile};

let mut diarizer = Sortformer::new("nemotron3_diar_v3.onnx")?;
let segments = diarizer.diarize(audio, 16000, 1)?;
for seg in segments {
    println!("speaker_{} [{:.2}s - {:.2}s]", seg.speaker_id,
        seg.start as f64 / 16_000.0, seg.end as f64 / 16_000.0);
}

// Streaming/real-time: diarize_chunk() / feed() preserve state across calls.
let segments = diarizer.diarize_chunk(&audio_chunk_16k_mono)?;

// Latency presets from NVIDIA's model card:
// offline() 30.4s (default), low_latency() 1.04s, very_low_latency() 0.64s, ultra_low_latency() 0.32s
diarizer.set_profile(StreamingProfile::low_latency())?;

See examples/diarization.rs for a runnable example (pass low, very-low or ultra to try the latency presets) and examples/streaming_diarization.rs for real-time feed/flush.

See scripts/export_diar_sortformer.py for exporting the ONNX (dual-resolution, self-describing metadata) with custom streaming parameters.

CTC: Download from HuggingFace: model.onnx, model.onnx_data, tokenizer.json

TDT: Download from HuggingFace: encoder-model.onnx, encoder-model.onnx.data, decoder_joint-model.onnx, vocab.txt

Orukeet (optional TDT fine-tune): the pinned Hugging Face INT8 export works with ParakeetTDT. The downloader verifies a release manifest and all required file hashes, including licenses, then prints the cached model directory:

python3 -m pip install huggingface-hub
model_dir=$(python3 scripts/download_orukeet.py)
cargo run --release --example orukeet -- "$model_dir" audio.wav
# After installation, no network is needed:
model_dir=$(python3 scripts/download_orukeet.py --offline)

This uses the existing local TDT runtime; it does not add streaming support or change defaults. The ~672 MB weights are licensed under CC BY-SA 4.0, as recorded in the downloaded LICENSE-WEIGHTS; NOTICE.md preserves the NVIDIA/Parakeet attribution. Model downloads use normal Hugging Face accounting through the required verification manifest. Audio stays local; cached files cause no counting requests. Accuracy numbers on the model card describe its NeMo evaluation, not a benchmark of this Rust runtime.

EOU: Download from HuggingFace: encoder.onnx, decoder_joint.onnx, tokenizer.json

Nemotron (English-only): Download from HuggingFace: encoder.onnx, encoder.onnx.data, decoder_joint.onnx, tokenizer.model (int8 / int4)

Nemotron (Multilingual 3.5): Download from HuggingFace: encoder.onnx, encoder.onnx.data, decoder_joint.onnx, tokenizer.model. Or export it yourself from the base model with scripts/export_nemotron_streaming_multilingual.py.

Unified: Download from HuggingFace: encoder.onnx, encoder.onnx.data, decoder_joint.onnx, tokenizer.model

Multitalker: Download from HuggingFace: encoder.int8.onnx, decoder_joint.int8.onnx, tokenizer.model (also needs the Nemotron-3 diarization ONNX below)

Cohere Transcribe: Download from HuggingFace: encoder_model.onnx (+ .onnx_data*), decoder_model_merged.onnx (+ .onnx_data), tokenizer.json (FP32, FP16, INT8, INT4 variants available)

Diarization (Nemotron-3 / Sortformer v3): Download from HuggingFace: nemotron3_diar_v3.onnx. Or export it yourself from the base model with scripts/export_diar_sortformer.py.

Quantized versions available (int8). All files must be in the same directory.

GPU support (auto-falls back to CPU if fails):

parakeet-rs = { version = "0.3", features = ["cuda"] }  # or tensorrt, webgpu, directml, migraphx or other ort supported EPs (check cargo features)

use parakeet_rs::{Parakeet, ExecutionConfig, ExecutionProvider};

let config = ExecutionConfig::new().with_execution_provider(ExecutionProvider::Cuda);
let mut parakeet = Parakeet::from_pretrained(".", Some(config))?;

Advanced session configuration via ort SessionBuilder:

let config = ExecutionConfig::new()
    .with_custom_configure(|builder| builder.with_memory_pattern(false));

• CTC: English with punctuation & capitalization
• TDT: Multilingual (auto lang detection)
• EOU: Streaming ASR with end-of-utterance detection
• Nemotron: Cache aware streaming ASR (600M params,EN only)
• Unified: Offline + buffered streaming RNNT ASR (600M params, EN only)
• Multitalker: Streaming multi-speaker ASR with speaker-kernel injection (ONNX int8)
• Cohere Transcribe: Offline multilingual ASR (14 languages, long-form supported) (ONNX)
• Nemotron-3 Diarization (Sortformer v3): Streaming speaker diarization (up to 8 speakers) (ONNX)
• Token-level timestamps (CTC, TDT)

• Audio: 16kHz mono WAV (16-bit PCM or 32-bit float)
• CTC/TDT models have ~4-5 minute audio length limit. For longer files, use streaming models or split into chunks

Code: MIT OR Apache-2.0

FYI: The Parakeet ONNX models (downloaded separately from HuggingFace) by NVIDIA. This library does not distribute the models.