Repairing Speech Access After Text Updates with Frozen Affine Fields

Key points

• Speech-capable language models inherit most of their knowledge from text-rich language backbones, and continued model maintenance is likewise dominated by text updates.1
• Knowledge acquired through a text update can nevertheless remain difficult to access when the same query is spoken.1
• We formulate this mismatch as post-update speech access repair, where the knowledge-bearing checkpoint stays frozen and a detachable operator repairs speech-only access in one model pass.1
• On balanced synthetic-speech PopQA, paired comparisons do not establish a clear advantage for AFFIRM over global SALSA steering or low-rank ReFT interventions under the same frozen single-state constraint.1

Sources (1)

• [1]Repairing Speech Access After Text Updates with Frozen Affine Fields
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 02:24 PM

  We introduce Affine Fields for Interface Repair across Modalities (AFFIRM), which first learns and freezes a base correction for speech hidden states, then adds query-specific corrections with zero mean over the operator-training queries.

  Speech-capable language models inherit most of their knowledge from text-rich language backbones, and continued model maintenance is likewise dominated by text updates.

Extractive summary: sentences quoted from the sources.