Merit: Making every modality earn its contribution in multimodal attention decoding
Auditory attention decoding (AAD) aims to solve this by reading a listener's brain signals to find the attended talker, letting the device amplify only them.
ProofPaper ↗
Key points
- More than 1.5 billion people have hearing loss.
- A decoder relying heavily on them fails when a listener cannot move their head or listens without looking - exactly when a brain-based device is needed.
- We present MERIT, a decoder and evaluation method asking: how much accuracy does each input earn?
- On a public dataset with EEG, eye tracking, head motion, and scene video, MERIT improves on the published benchmark, reaching 0.82 four-way accuracy within listeners and 0.74 on unseen listeners (chance is 0.25).
Sources (1)
- [1]Merit: Making every modality earn its contribution in multimodal attention decodingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 12:10 AM
Auditory attention decoding (AAD) aims to solve this by reading a listener's brain signals to find the attended talker, letting the device amplify only them.
More than 1.5 billion people have hearing loss.
Extractive summary: sentences quoted from the sources.