Prosodic Structure as a Critical Component in Phonemic Language Modelling at BabyLM Scale

Key points

• Human children learn language from continuous, prosodically structured speech long before they can read, yet almost all language models train on orthographic text.1
• We compare an orthographic baseline to three conditions forming a ladder of phonological richness.1
• Our new lexical decision task offers improved insights into the distinction between wordlikeness, token familiarity, and lexicality as factors.1
• These findings suggest that prosodic structure is a key component of phonemic language modelling in the small-data regime most analogous to infant learning.1

Sources (1)

• [1]Prosodic Structure as a Critical Component in Phonemic Language Modelling at BabyLM Scale
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 07:46 PM

  Building on previous phonemic BabyLM training (Goriely et al., 2024; arXiv:2410.22906), we ask whether supplying an estimate of the prosody infants exploit improves phonemic language modelling at BabyLM scale.

  Human children learn language from continuous, prosodically structured speech long before they can read, yet almost all language models train on orthographic text.

Extractive summary: sentences quoted from the sources.