SNOAR: Self-training with Noise filtering, Observation adding, and Remixing for Speech Denoising Without Clean References

Key points

• Unsupervised speech denoising methods such as MixIT and noisy-target training can be trained directly on noisy recordings, avoiding the sim-to-real gap of synthetic training data, but remain well below supervised performance.1
• Teacher--student self-training, as in RemixIT, narrows this gap by remixing a teacher's estimates into new mixtures that supervise a student; however, since the student is trained only toward the teacher's estimates, teacher errors, in particular over-suppressed speech, are never recovered and propagate as the teacher is updated from the student.1
• We show that the two corrections depend on each other, as without DNF the loop collapses to the identity mapping.1
• Compared to RemixIT, SNOAR is more stable along training and outperforms it in signal-based, perceptual, and word error rate metrics, reaching 97% of the SI-SDR improvement of a fully-supervised topline on WHAM!; on real CHiME-3 recordings, it improves every metric of three different teachers, while RemixIT degrades their perceptual quality.1

Sources (1)

• [1]SNOAR: Self-training with Noise filtering, Observation adding, and Remixing for Speech Denoising Without Clean References
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 07:54 PM

  We propose SNOAR, a self-training framework that corrects the teacher's estimates before use: the noise leaked into its speech estimate is removed by differential noise filtering (DNF), and part of the noisy observation is added back in to restore suppressed speech and attenuate artifacts.

  Unsupervised speech denoising methods such as MixIT and noisy-target training can be trained directly on noisy recordings, avoiding the sim-to-real gap of synthetic training data, but remain well below supervised performance.

Extractive summary: sentences quoted from the sources.