VieNeu-TTS is an advanced on-device Vietnamese Text-to-Speech (TTS) model with instant voice cloning.
Tip
Voice Cloning: All model variants (including GGUF) support instant voice cloning with just 3-5 seconds of reference audio.
This project features two core architectures trained on the VieNeu-TTS-1000h dataset:
- VieNeu-TTS (0.5B): An enhanced model fine-tuned from the NeuTTS Air architecture for maximum stability.
- VieNeu-TTS-0.3B: A specialized model trained from scratch, delivering 2x faster inference and ultra-low latency.
These represent a significant upgrade from the previous VieNeu-TTS-140h with the following improvements:
- Enhanced pronunciation: More accurate and stable Vietnamese pronunciation
- Code-switching support: Seamless transitions between Vietnamese and English
- Better voice cloning: Higher fidelity and speaker consistency
- Real-time synthesis: 24 kHz waveform generation on CPU or GPU
- Multiple model formats: Support for PyTorch, GGUF Q4/Q8 (CPU optimized), and ONNX codec
VieNeu-TTS delivers production-ready speech synthesis fully offline.
Author: Phạm Nguyễn Ngọc Bảo
Screen.Recording.2025-12-10.201011.mp4
- Backbone:
- VieNeu-TTS (0.5B): Qwen-0.5B fine-tuned from NeuTTS Air.
- VieNeu-TTS-0.3B: Custom 0.3B model trained from scratch, optimized for extreme speed (2x faster).
- Audio codec: NeuCodec (torch implementation; ONNX & quantized variants supported)
- Context window: 2,048 tokens shared by prompt text and speech tokens
- Output watermark: Enabled by default
- Training data: VieNeu-TTS-1000h — 443,641 curated Vietnamese samples (Used for both versions).
| Model | Format | Device | Quality | Speed |
|---|---|---|---|---|
| VieNeu-TTS | PyTorch | GPU/CPU | ⭐⭐⭐⭐⭐ | Very Fast with lmdeploy |
| VieNeu-TTS-0.3B | PyTorch | GPU/CPU | ⭐⭐⭐⭐ | Ultra Fast (2x) |
| VieNeu-TTS-q8-gguf | GGUF Q8 | CPU/GPU | ⭐⭐⭐⭐ | Fast |
| VieNeu-TTS-q4-gguf | GGUF Q4 | CPU/GPU | ⭐⭐⭐ | Very Fast |
| VieNeu-TTS-0.3B-q8-gguf | GGUF Q8 | CPU/GPU | ⭐⭐⭐⭐ | Ultra Fast (1.5x) |
| VieNeu-TTS-0.3B-q4-gguf | GGUF Q4 | CPU/GPU | ⭐⭐⭐ | Extreme Speed (2x) |
Recommendations:
- GPU users: Use
VieNeu-TTS(PyTorch) for best quality - CPU users: Use
VieNeu-TTS-0.3B-q4-gguffor fastest inference orVieNeu-TTS-0.3B-q8-gguffor best CPU quality. - Streaming: Only GGUF models support streaming inference (Requires
llama-cpp-python >= 0.3.16)
- Publish safetensor artifacts
- Release GGUF Q4 / Q8 models
- Release datasets (1000h and 140h)
- Enable streaming on GPU
- Provide Dockerized setup
- Release fine-tuning code (LoRA)
- LoRA Adapter integration in Gradio
VieNeu-TTS now officially supports LoRA (Low-Rank Adaptation). This allows you to:
- Use custom fine-tuned voices from Hugging Face.
- Achieve much higher quality and similarity than zero-shot voice cloning.
- Switch between different adapters seamlessly in the Gradio UI.
For more details, see docs/LORA_USAGE.md.
You can now train VieNeu-TTS on your own voice dataset!
- Simple Workflow: Follow the step-by-step guide in finetune/README.md.
- Notebook Support: Use
finetune/finetune_VieNeu-TTS.ipynbfor an interactive experience.
git clone https://github.com/pnnbao97/VieNeu-TTS.git
cd VieNeu-TTSPhonemizer requires eSpeak NG to function.
- Windows: Download installer from eSpeak NG Releases (Recommended:
.msi). - macOS:
brew install espeak - Ubuntu/Debian:
sudo apt install espeak-ng - Arch Linux:
paru -S aur/espeak-ng
A. Install uv (Fast Python package manager):
- Windows:
powershell -c "irm https://astral.sh/uv/install.ps1 | iex" - Linux/macOS:
curl -LsSf https://astral.sh/uv/install.sh | sh
B. Install dependencies:
Important
For NVIDIA GPU Users: Update your NVIDIA Drivers & Install CUDA Toolkit! This project uses CUDA 12.8. Please ensure your NVIDIA driver is up-to-date (support CUDA 12.8 or newer) to avoid compatibility issues, especially on RTX 30 series.
To use lmdeploy, you MUST install the NVIDIA GPU Computing Toolkit: https://developer.nvidia.com/cuda-downloads.
# Default setup (Includes GPU support)
uv sync
# CPU-only mode (No GPU dependencies)
uv sync --no-default-groupsOnce environment is ready, start the Web UI with a single command:
uv run gradio_app.pyAccess the Web UI at http://127.0.0.1:7860.
If you want to integrate VieNeu-TTS into your own project:
We provide pre-built CPU wheels for llama-cpp-python (version 0.3.16) for Python 3.10 to 3.14 to avoid compilation errors.
pip install vieneu --extra-index-url https://pnnbao97.github.io/llama-cpp-python-v0.3.16/cpu/pip install vieneuFor high-performance GPU inference without local complexity, you can set up a remote server using lmdeploy.
A. On the Server (with GPU):
- Install LMDeploy:
pip install lmdeploy[all] - Launch the API Server:
lmdeploy serve api_server pnnbao-ump/VieNeu-TTS-0.3B --server-port 23333 --tp 1B. On the Client (CPU/Laptop): Connect to the server using the SDK:
from vieneu import Vieneu
# Connect to the remote server
tts = Vieneu(mode="remote", api_base="http://your-server-ip:23333/v1", model_name="pnnbao-ump/VieNeu-TTS-0.3B")Here is a comprehensive example showing how to initialize, manage voices, clone custom voices, and control generation.
"""
Demo VieNeuSDK v1.1.3 - Full Features Guide
"""
import time
import soundfile as sf
from vieneu import Vieneu
from pathlib import Path
def main():
print("🚀 Initializing VieNeu SDK (v1.1.3)...")
# Initialize SDK
# Default: "pnnbao-ump/VieNeu-TTS-0.3B-q4-gguf" (Speed & CPU Optimized)
#
# You can change 'backbone_repo' to balance Quality vs Speed:
# 1. Better Quality (slower than q4): "pnnbao-ump/VieNeu-TTS-0.3B-q8-gguf"
# 2. PyTorch 0.3B (Fast, uncompressed): "pnnbao-ump/VieNeu-TTS-0.3B"
# 3. PyTorch 0.5B (Best Quality, heavy): "pnnbao-ump/VieNeu-TTS"
# You can also use a GGUF version merged with your own LoRA adapter.
# See finetuning guide: https://github.com/pnnbao97/VieNeu-TTS/tree/main/finetune
# Mode selection:
# - mode="standard" (Default): Runs locally using GGUF (CPU) or PyTorch
# - mode="remote": Connects to the LMDeploy server setup above for max speed
tts = Vieneu()
# Or to use Remote mode (Must start 'lmdeploy serve api_server pnnbao-ump/VieNeu-TTS-0.3B --server-port 23333 --tp 1' in another tab/machine first):
# tts = Vieneu(mode="remote", api_base="http://localhost:23333/v1", model_name="pnnbao-ump/VieNeu-TTS-0.3B")
# Example for using Q8 for better quality:
# tts = Vieneu(backbone_repo="pnnbao-ump/VieNeu-TTS-0.3B-q8-gguf")
# ---------------------------------------------------------
# PART 1: PRESET VOICES
# ---------------------------------------------------------
print("\n--- 1. Available Preset Voices ---")
available_voices = tts.list_preset_voices()
print("📋 Voices:", available_voices)
# Select a preset voice
current_voice = tts.get_preset_voice("Binh")
print("✅ Selected voice: Binh")
# ---------------------------------------------------------
# PART 2: CREATE & SAVE CUSTOM VOICE
# ---------------------------------------------------------
print("\n--- 2. Create Custom Voice ---")
# Replace with your actual .wav file path and its exact transcript (including punctuation)
sample_audio = Path(__file__).parent / "example.wav"
sample_text = "ví dụ 2. tính trung bình của dãy số."
if sample_audio.exists():
voice_name = "MyCustomVoice"
print(f"🎙️ Cloning voice from: {sample_audio.name}")
# 'clone_voice' now supports saving directly with 'name' argument
custom_voice = tts.clone_voice(
audio_path=sample_audio,
text=sample_text,
name=voice_name # <-- Automatically saves voice to system
)
print(f"✅ Voice created and saved as: '{voice_name}'")
# Verify functionality
print("📋 Voice list after adding:", tts.list_preset_voices())
# Switch to new voice
current_voice = custom_voice
else:
print("⚠️ Sample audio not found. Skipping...")
# ---------------------------------------------------------
# PART 3: SYNTHESIS WITH ADVANCED PARAMETERS
# ---------------------------------------------------------
print("\n--- 3. Speech Synthesis ---")
text_input = "Xin chào, tôi là VieNeu-TTS. Tôi có thể giúp bạn đọc sách, làm chatbot thời gian thực, hoặc thậm chí clone giọng nói của bạn."
# Generate with specific temperature
print("🎧 Generating...")
audio = tts.infer(
text=text_input,
voice=current_voice,
temperature=1.0, # Adjustable: Lower (0.1) -> Stable, Higher (1.0+) -> Expressive
top_k=50
)
sf.write("output.wav", audio, 24000)
print("💾 Saved: output.wav")
# ---------------------------------------------------------
# CLEANUP
# ---------------------------------------------------------
tts.close()
print("\n✅ Done!")
if __name__ == "__main__":
main()Best if you have make installed (standard on Linux/macOS, or via Git Bash on Windows). It handles configuration swaps automatically.
- Setup:
make setup - Run Demo:
make demo
Then access the Web UI at http://127.0.0.1:7860.
For a quick start or production deployment without manually installing dependencies, use Docker.
Copy .env.example to .env
cp .env.example .env
Build and start container
# Run with CPU
docker compose --profile cpu up
# Run with GPU (requires NVIDIA Container Toolkit)
docker compose --profile gpu upAccess the Web UI at http://localhost:7860.
For detailed deployment instructions, including production setup, see docs/Deploy.md.
VieNeu-TTS/
├── vieneu/ # Core engine implementation (VieNeuTTS & FastVieNeuTTS)
├── finetune/ # LoRA training pipeline
│ ├── configs/ # Training & LoRA configurations
│ ├── data_scripts/ # Data filtering & VQ encoding tools
│ ├── dataset/ # Training data storage
│ ├── output/ # Saved checkpoints & LoRA adapters
│ └── train.py # Main training script
├── utils/ # Text normalization and phonemization logic
├── sample/ # Built-in reference voices (audio + transcript + codes)
├── docs/ # Detailed documentation for LoRA, Deployment, and Docker
├── examples/ # Usage examples and testing audio references
├── gradio_app.py # Modern Web UI with LoRA & Streaming support
├── config.yaml # Model, Codec, and Voice registry
├── pyproject.toml # Unified dependency management (UV/PIP)
├── Makefile # Shortcuts for setup and execution
└── docker-compose.yml # Docker orchestration for CPU/GPU modes
- GitHub Repository
- Hugging Face Model (0.5B)
- Hugging Face Model (0.3B)
- LoRA Usage Guide
- Fine-tuning Guide
- VieNeu-TTS-1000h dataset
- VieNeu-TTS (0.5B): Original terms (Apache 2.0).
- VieNeu-TTS-0.3B: Released under CC BY-NC 4.0 (Non-Commercial).
- This version is currently experimental.
- Commercial use is prohibited without authorization. Please contact the author for commercial licensing.
@misc{vieneutts2026,
title = {VieNeu-TTS: Vietnamese Text-to-Speech with Instant Voice Cloning},
author = {Pham Nguyen Ngoc Bao},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/pnnbao-ump/VieNeu-TTS}}
}Contributions are welcome!
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-feature - Commit your changes:
git commit -m "Add amazing feature" - Push the branch:
git push origin feature/amazing-feature - Open a pull request
- GitHub Issues: github.com/pnnbao97/VieNeu-TTS/issues
- Hugging Face: huggingface.co/pnnbao-ump
- Discord: Join with us
- Facebook: Phạm Nguyễn Ngọc Bảo
This project builds upon NeuTTS Air for the original 0.5B model. The 0.3B version is a custom architecture trained from scratch using the VieNeu-TTS-1000h dataset.
Made with ❤️ for the Vietnamese TTS community