Summary

• Add compile_codec parameter to Qwen3TTSModel.from_pretrained() that applies torch.compile(mode="max-autotune", dynamic=True) to the speech tokenizer codec decoder
• Add _compile_codec() instance method for post-construction compilation
• Document the optimization in a new "Performance Tips" README section

Motivation

The codec decoder contains 136+ attention modules whose Python dispatch overhead dominates waveform decoding time. Profiling shows:

Applying torch.compile eliminates this overhead:

Tested on AMD RX 7900 XTX (ROCm 6.3) and the optimization is hardware-agnostic — torch.compile with dynamic=True works on NVIDIA, AMD, and CPU backends.

API

# Opt-in via from_pretrained (default: no compilation, fully backward-compatible)
model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    compile_codec=True,  # or "max-autotune", "reduce-overhead", "default"
)

# Or compile after construction
model._compile_codec()

Trade-offs

• First call incurs ~30-60s warmup (one-time compilation cost)
• Recommended for batch/repeated generation (audiobooks, datasets), not single one-off calls
• No behavior change when compile_codec=False (default)

Test plan

• Verify compile_codec=False produces identical results to current behavior
• Verify compile_codec=True compiles codec and generation works correctly
• Verify compile_codec="max-autotune" string mode works
• End-to-end batch generation benchmarks showing 3-4x improvement
• Tested with CustomVoice model (generate_custom_voice)