Oído: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural Accelerator

Key points

• Speech recognition on low-cost microcontrollers is, in practice, limited to keyword spotting or a fixed list of commands.1
• Oído executes a 13M-parameter Conformer-CTC model with int8 weights, per-frame dynamic int8 activations and int8 attention on the chip's SIMD instructions, with a schedule designed around flash and PSRAM bandwidth.1
• On LibriSpeech test-clean/test-other the on-device arithmetic reaches 3.70%/8.23% word error rate (WER), within 0.12 points of full precision, and 3.33%/7.15% with an on-chip 1.3M-parameter language model; a transducer head gives 2.99%/6.67% and an int4 variant fits in 8.3 MB.1
• This is 2.3-2.8x fewer errors than the vendor's recognizer on the same chip, and fewer than Whisper tiny.en and Moonshine tiny in full precision on a laptop, including in real noise and reverberation.1

Sources (1)

• [1]Oído: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural Accelerator
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 11:48 AM

  We present Oído, an inference engine that runs open-vocabulary English speech recognition entirely on an ESP32-S3 (240 MHz dual-core Xtensa LX7, 512 KB of SRAM, 8 MB of PSRAM, 16 MB of flash, no neural accelerator).

  Speech recognition on low-cost microcontrollers is, in practice, limited to keyword spotting or a fixed list of commands.

Extractive summary: sentences quoted from the sources.