UA-Bench: Benchmarking Large Audio-Language Models on Underwater Acoustic Recognition

Key points

• Large audio-language models (LALMs) are fast becoming a general interface to sound, yet how far their broad pretraining carries into specialized domains remains largely open.1
• Vessel monitoring, maritime safety, and ocean sensing all depend on recognizing underwater signals, but to our knowledge LALMs have not been systematically evaluated on them.1
• Annotated recordings are costly to collect, and a faithful synthetic alternative has to bring together many interacting factors: the structure of detection pulses, communication waveforms, and ship-radiated noise, and the sound-speed profile, geometry, and multipath propagation through which each reaches the receiver.1
• We evaluate on a balanced 2,600-record subset with a gated coarse-to-fine protocol that locates each error at source routing, fine-grained classification, or output format, and add a paired silent-audio control to test whether the waveform is actually used.1

Sources (1)

• [1]UA-Bench: Benchmarking Large Audio-Language Models on Underwater Acoustic Recognition
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 03:18 PM

  We address this by proposing UA-Bench, to our knowledge the first large-scale benchmark for evaluating LALMs on underwater acoustic signal recognition.

  Large audio-language models (LALMs) are fast becoming a general interface to sound, yet how far their broad pretraining carries into specialized domains remains largely open.

Extractive summary: sentences quoted from the sources.