SLM-SS: Token-Based Speech Separation with Speech Language Models
To address this gap, we propose SLM-SS, a token-based generative speech separation framework that models multi-talker mixtures with speech language models.
ProofPaper ↗
Key points
- Speech separation (SS) has witnessed substantial progress in recent years with the advent of neural network-based methods, achieving notable improvements on signal-level evaluation metrics.
- We first discretize clean speech into multi-codebook token sequences with neural codecs and serialize multi-speaker targets via Serialized Output Training (SOT).
- An encoder-decoder autoregressive speech language model predicts the zero-order sequence to establish speaker ordering and coarse linguistic content, and a non-autoregressive module then refines higher-order residual codebooks for efficient reconstruction.
- We further introduce a per-talker non-autoregressive strategy that simplifies high-order modeling and improves the stability.
Sources (1)
- [1]SLM-SS: Token-Based Speech Separation with Speech Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 07:50 AM
To address this gap, we propose SLM-SS, a token-based generative speech separation framework that models multi-talker mixtures with speech language models.
Speech separation (SS) has witnessed substantial progress in recent years with the advent of neural network-based methods, achieving notable improvements on signal-level evaluation metrics.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Exploring Recurrent-Depth Language Models for Multitask Clinical Text Processing
- Oct 19, 2026The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models
- Oct 19, 2026TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
- Oct 18, 2026A failure of Imagination: VLMs fail to use visual representations for text, but self-distillation can help
- Oct 18, 2026ViCoR: Estimating Visual Necessity via Counterfactual Residuals for Multimodal Medical Data Selection
- Oct 18, 2026PACT: NVFP4 Data-Flow LoRA Fine-Tuning on Consumer GPUs