When Form Changes but Logic Doesn't: Building Logic-invariant LLMs through Logical Structure

Key points

• A key indicator of such understanding is whether models treat logically equivalent inputs consistently, producing the same predictions despite differences in expression.1
• Through systematic evaluation on curated data, we show that even strong LLMs often fail this test: they do not consistently preserve predictions across logically equivalent inputs and generalize poorly to unseen logical forms.1
• To address this limitation, we propose LoGIcal STructure-guided reasoning (LoGIST), a framework that guides LLM reasoning with explicit logical structure.1
• Across multiple training settings, LoGIST reduces inconsistency under logical equivalence and improves generalization to unseen logical forms, suggesting that logical invariance is a useful target for building more robust and generalizable logical reasoning in LLMs.1

Sources (1)

• [1]When Form Changes but Logic Doesn't: Building Logic-invariant LLMs through Logical Structure
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 04:56 PM

  Large language models (LLMs) have achieved strong performance across diverse reasoning tasks, yet it remains unclear whether this performance reflects an understanding of underlying logical structure or a reliance on shallow semantic cues.

  A key indicator of such understanding is whether models treat logically equivalent inputs consistently, producing the same predictions despite differences in expression.

Extractive summary: sentences quoted from the sources.