PEDBench: Can Speech Models Detect Phonetic Errors?

Key points

• Despite recent advances in speech models for phoneme recognition, their ability to reliably detect fine-grained phonetic errors remains unclear because of a lack of high-quality evaluation data.1
• To address this gap, we introduce PEDBench, the first PED benchmark spanning a continuum from natural to controlled speech.1
• Our evaluation of 13 state-of-the-art speech models shows that they fail to reliably detect phonetic errors, even under controlled conditions with acted and synthetic speech, where acoustic variability is reduced.1
• Our speech-language models, fine-tuned on synthetic data, outperform commercial and open-source baselines on word-level PED, although a sim-to-real domain gap remains.1

Sources (1)

• [1]PEDBench: Can Speech Models Detect Phonetic Errors?
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 07:15 PM

  Phonetic error detection (PED) aims to identify phoneme-level substitutions, deletions, and insertions and is essential for educational and clinical speech assessment and intervention.

  Despite recent advances in speech models for phoneme recognition, their ability to reliably detect fine-grained phonetic errors remains unclear because of a lack of high-quality evaluation data.

Extractive summary: sentences quoted from the sources.