Official PyTorch Implementation of "Diffusion Transformers with Representation Autoencoders"

semantic tokenizer for speech and music

dh vq-q or vae exp

Production First and Production Ready End-to-End Speech Recognition Toolkit

BLSP: Bootstrapping Langauge-Speech Pre-training via Behavior Alignment of Continuation Writing

This is the code for the SpeechTokenizer presented in the SpeechTokenizer: Unified Speech Tokenizer for Speech Language Models. Samples are presented on

Models and code for RepCodec: A Speech Representation Codec for Speech Tokenization

NeMo text processing for ASR and TTS

High-Resolution Image Synthesis with Latent Diffusion Models

The Microsoft Scalable Noisy Speech Dataset (MS-SNSD) is a noisy speech dataset that can scale to arbitrary sizes depending on the number of speakers, noise types, and Speech to Noise Ratio (SNR) l…

MaTe3D: Mask-guided Text-based 3D-aware Portrait Editing

A curated list of papers in Test-time Adaptation, Test-time Training and Source-free Domain Adaptation

FAIR Sequence Modeling Toolkit

Promting Whisper for Audio-Visual Speech Recognition, Code-Switched Speech Recognition, and Zero-Shot Speech Translation

An implementation of "Retentive Network: A Successor to Transformer for Large Language Models"

微调Whisper语音识别模型和加速推理，支持Web部署和Android部署

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration

The PyTorch-based audio source separation toolkit for researchers

An Audio Language model for Audio Tasks

Easy-to-use Speech Toolkit including SOTA/Streaming ASR with punctuation, influential TTS with text frontend, Speaker Verification System, End-to-End Speech Translation and Keyword Spotting. Won NA…

🔊 Text-prompted Generative Audio Model