Neither True nor False: Do LLMs Represent Truth-Value Gaps in Plural Predication?

Key points

• We investigate the Homogeneity Gap, a semantic phenomenon where a plural definite (e.g., the kids laughed) and its negation (the kids didn't laugh) are both neither true nor false when only some of the kids laughed, whereas their quantified counterparts (all, not all) remain strictly bivalent.1
• We introduce HomogeneityNLI, a diagnostic dataset of 1,000 minimal-pair items crossing determiner, polarity, and group composition across ten logical conditions.1
• Evaluating seven open-weight models, we uncover a pervasive Bivalence Bias: models assign plural definites their homogeneous truth conditions, reading the kids as all of them and the kids didn't as none of them, yet collapse the truth-value gap into falsity.1
• Taken together, our findings indicate that current open-weight LLMs have acquired the truth conditions of plural predication but not its trivalent logic.1

Sources (1)

• [1]Neither True nor False: Do LLMs Represent Truth-Value Gaps in Plural Predication?
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 01:12 AM

  Do Large Language Models (LLMs) genuinely grasp the truth conditions of plural descriptions, or do they force every sentence into true or false?

  We investigate the Homogeneity Gap, a semantic phenomenon where a plural definite (e.g., the kids laughed) and its negation (the kids didn't laugh) are both neither true nor false when only some of the kids laughed, whereas their quantified counterparts (all, not all) remain strictly bivalent.

Extractive summary: sentences quoted from the sources.