Revisiting the Estimation of Unigram Surprisal in Computational Psycholinguistics

Key points

• While contextual surprisal is typically derived from neural language models, unigram surprisal is usually obtained from independent corpus counts.1
• We propose three principles for estimating unigram surprisal in psycholinguistic research, including the requirement that both quantities be derived from the same language model.1
• We operationalize this principle by introducing a Monte Carlo estimator of unigram surprisal from a language model, along with a marginalized variant that has lower variance empirically.1
• When unigram surprisal is re-estimated under this unified framework, we find evidence that predictors change in their predictive contribution to reading time.1

Sources (1)

• [1]Revisiting the Estimation of Unigram Surprisal in Computational Psycholinguistics
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 01:54 PM

  Contextual and unigram surprisal are standard predictors of reading time in computational psycholinguistics.

  While contextual surprisal is typically derived from neural language models, unigram surprisal is usually obtained from independent corpus counts.

Extractive summary: sentences quoted from the sources.