This repository contains the official implementation of ComVo, a complex-valued neural vocoder for waveform generation based on iSTFT.

Overall architecture of ComVo

Neural vocoders have recently advanced waveform generation, yielding natural and expressive audio. Among these approaches, iSTFT-based vocoders have gained attention. They predict a complex-valued spectrogram and then synthesize the waveform via iSTFT, thereby avoiding redundant, computationally expensive upsampling. However, current approaches use real-valued networks that process the real and imaginary parts independently. This separation limits their ability to capture the inherent structure of complex spectrograms. We present ComVo, a complex-valued neural vocoder whose generator and discriminator use native complex arithmetic. This enables an adversarial training framework that provides structured feedback directly in the complex domain. To guide phase transformations in a structured manner, we introduce phase quantization, which discretizes phase values and regularizes the training process. Finally, we propose a block-matrix computation scheme to improve training efficiency by reducing redundant operations. Experiments demonstrate that ComVo achieves higher synthesis quality than comparable real-valued baselines, and that its block-matrix scheme reduces training time by 25%. Audio samples and code are available at https://hs-oh-prml.github.io/ComVo/.

pip install -r requirements

• Python >= 3.8
• PyTorch >= 2.0
• CUDA-enabled GPU

• from_waveform performs end-to-end inference from raw audio.
• build_feature_extractor allows explicit feature extraction, useful for debugging or custom pipelines.

We provide pretrained ComVo checkpoints for quick inference:

Model	Parameters	Hugging Face	Checkpoint	Sampling rate	n_fft	hop size	win length
Base	13.28M			24 kHz	100	12000	256
Large	114.56M			24 kHz	100	12000	256

1. End-to-end inference from waveform

import torch
import torchaudio
from hf_model import ComVoHF

device = "cuda" if torch.cuda.is_available() else "cpu"

# Load model
model = ComVoHF.from_pretrained("hsoh/ComVo-base")
model = model.eval().to(device)

# Load audio
wav, sr = torchaudio.load("input.wav")

# Convert to mono if needed
if wav.size(0) > 1:
    wav = wav.mean(dim=0, keepdim=True)

wav = wav.to(device)  # [1, T]

# Inference
with torch.inference_mode():
    audio = model.from_waveform(wav)

# Save output
torchaudio.save("output.wav", audio.squeeze(0).cpu(), model.sample_rate)

2. Inference from extracted features

import torch
import torchaudio
from hf_model import ComVoHF

device = "cuda" if torch.cuda.is_available() else "cpu"

# Load model
model = ComVoHF.from_pretrained("hsoh/ComVo-base")
model = model.eval().to(device)

# Build feature extractor
feature_extractor = model.build_feature_extractor().to(device)

# Load audio
wav, sr = torchaudio.load("input.wav")

# Convert to mono if needed
if wav.size(0) > 1:
    wav = wav.mean(dim=0, keepdim=True)

wav = wav.to(device)  # [1, T]

# Feature extraction + inference
with torch.inference_mode():
    features = feature_extractor(wav)
    audio = model(features)

# Save output
torchaudio.save("output.wav", audio.squeeze(0).cpu(), model.sample_rate)

python infer.py -c configs/configs.yaml --ckpt /path/to/comvo.ckpt --wavfile /path/to/input.wav --out_dir ./results

python train.py -c configs/configs.yaml

Hyperparameters are specified in configs/configs.yaml.

python infer.py -c $CONFIG --ckpt=$CKPT --wavfile=$FILE_NAME --out_dir $OUTPUT_DIR

@inproceedings{
  oh2026toward,
  title={Toward Complex-Valued Neural Networks for Waveform Generation},
  author={Hyung-Seok Oh and Deok-Hyeon Cho and Seung-Bin Kim and Seong-Whan Lee},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026},
  url={https://openreview.net/forum?id=U4GXPqm3Va}
}