VoTSpeech: Decoupling Voice Design from Speech Generation in Instruction-Following TTS
We propose VoTSpeech, which decouples voice design from speech generation via an explicit intermediate voice latent.
ProofPaper ↗
Key points
- Instruction-following text-to-speech (TTS) typically leaves voice design implicit, as conventional speaker embeddings encode reference audio rather than text-driven targets.
- Built on a shared language model backbone, VoTSpeech first predicts a continuous voice latent from text instructions using a flow-based module.
- To capture speaker identity and complementary acoustic nuances, the voice latent is jointly supervised by speaker embeddings and temporally pooled audio VAE features.
- These results confirm that explicit voice latent design and its conditioning interface effectively empower text-driven voice design for controllable speech synthesis.
Sources (1)
- [1]VoTSpeech: Decoupling Voice Design from Speech Generation in Instruction-Following TTSarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 08:13 AM
We propose VoTSpeech, which decouples voice design from speech generation via an explicit intermediate voice latent.
Instruction-following text-to-speech (TTS) typically leaves voice design implicit, as conventional speaker embeddings encode reference audio rather than text-driven targets.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 10, 2026We Have Been Interpreting Diffusion Transformer Backbone Behaviors Wrong
- Oct 10, 2026LightVAE: Towards Compact and Efficient Video Autoencoders
- Oct 9, 2026Image Diffusion Models Are Good 3D Human Motion Generators
- Oct 9, 2026PhysForge: Material-Controllable 3D Reconstruction with Separable Components
- Oct 9, 2026Latent World Action Model for GUI Agents
- Oct 9, 2026GeoFix: Geometry Fixing for 4D Consistent Generation