Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models

Key points

• We propose a lightweight method that detects CS directly from speech.1
• Specifically, we identify language-selective hidden-state dimensions using monolingual recordings and aggregate their frame-level activations into a CS prediction.1
• With a small CS set used to tune a decision rule, we show that the Whisper-small encoder achieves a mean F1 of 0.798 across 14 language pairs in CS-FLEURS, compared with 0.776 for a CNN and 0.751 for a HuBERT linear baseline trained for monolingual language identification.1
• Code-switching (CS), the alternation between languages within an utterance, is common in multilingual speech but underrepresented in pretraining corpora.2

Sources (2)

• [1]Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models
  Apple Machine Learning Research · Oct 14, 12:00 AM

  Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models

  We propose a lightweight method that detects CS directly from speech.

• [2]Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 05:05 PM

  Code-switching (CS), the alternation between languages within an utterance, is common in multilingual speech but underrepresented in pretraining corpora.

Extractive summary: sentences quoted from the sources.