StyleStream 2.0: Fast and Controllable Streaming Voice Style Conversion

Key points

• StyleStream 1.0 introduced the first streamable zero-shot VSC system with state-of-the-art conversion quality, but remained limited in two ways: approximately 1s end-to-end latency on consumer hardware, and the lack of a text-based interface for designing or editing target voices.1
• We present StyleStream 2.0, a fast and controllable streaming VSC framework that addresses these limitations.1
• In this embedding space, we train a unified text-conditioned flow matching model that supports both text-based voice design, which maps natural language prompts to voice styles, and instruction-based voice editing, which modifies a specific attribute of a source utterance while preserving the rest.1
• Experiments show that StyleStream 2.0 achieves the strongest target style fidelity on VSC, voice design, and voice editing, while remaining competitive on intelligibility and non-edited attribute preservation.1

Sources (1)

• [1]StyleStream 2.0: Fast and Controllable Streaming Voice Style Conversion
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 09:34 PM

  Voice style conversion (VSC) aims to transform a source utterance to match the timbre, accent, and emotion of a target voice while preserving linguistic content.

  StyleStream 1.0 introduced the first streamable zero-shot VSC system with state-of-the-art conversion quality, but remained limited in two ways: approximately 1s end-to-end latency on consumer hardware, and the lack of a text-based interface for designing or editing target voices.

Extractive summary: sentences quoted from the sources.