Learning Speech Attribute Representations via Multi-Teacher Distillation

Key points

• Recent post-training approaches have shown that self-supervised speech encoders can be adapted to learn a single utterance-level speech attribute through knowledge distillation.1
• In this work, we investigate whether a single speech encoder can simultaneously preserve multiple complementary utterance-level speech representations.1
• Experiments on multilingual semantic retrieval, speaker verification, language identification, and speech emotion recognition show that the proposed model achieves performance close to dedicated single-task students while requiring only a single shared encoder.1
• Furthermore, the learned layer aggregation strategies reveal that different speech attributes naturally emerge at different depths of the SSL encoder, providing new insights into the organization of speech representations.1

Sources (1)

• [1]Learning Speech Attribute Representations via Multi-Teacher Distillation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 02:15 PM

  We propose a multi-attribute multi-teacher distillation framework that learns semantic, speaker, language, and emotion representations from four frozen teacher models using a shared SSL backbone and attribute-specific branches.

  Recent post-training approaches have shown that self-supervised speech encoders can be adapted to learn a single utterance-level speech attribute through knowledge distillation.

Extractive summary: sentences quoted from the sources.