Training a Compact, Strong, and Realistic Text-to-Speech Model
We present Audio8-TTS, a compact 0.6B-parameter model for zero-shot voice cloning that achieves state-of-the-art performance in intelligibility and speaker similarity, with particularly strong results in perceptual naturalness.
ProofPaper ↗
Key points
- Text-to-speech (TTS) systems are conventionally optimized for intelligibility and speaker similarity.
- To align the training signal with human perception rather than signal-level fidelity, we construct a quality ladder from widely used open-source TTS systems and train a realism critic to assess generated speech against matched human recordings of the same sentences.
- We train the model in three stages: multilingual supervised fine-tuning; group relative policy optimization (GRPO) with word error rate (WER) and speaker similarity (SIM) rewards; and GRPO guided by the realism critic.
- The final stage improves perceived prosodic naturalness and speaker similarity while preserving WER performance.
Sources (1)
- [1]Training a Compact, Strong, and Realistic Text-to-Speech ModelarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 09:59 AM
We present Audio8-TTS, a compact 0.6B-parameter model for zero-shot voice cloning that achieves state-of-the-art performance in intelligibility and speaker similarity, with particularly strong results in perceptual naturalness.
Text-to-speech (TTS) systems are conventionally optimized for intelligibility and speaker similarity.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 10, 2026KV4-RL: Unlocking Faster Asynchronous Rollouts for RLVR with INT4 KV Cache Quantization
- Oct 10, 2026EviScene: Beyond "Is It AI?" for Agentic Image Factuality Assessment
- Oct 9, 2026The Thinking Leakage in Reinforcement Learning for LLMs
- Oct 9, 2026Rethinking Relative Policy Differences Transfer of On-Policy Distillation
- Oct 9, 2026SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models
- Oct 8, 2026huggingface/trl v1.15.0