Poison Valleys: Diagnosing Path-Dependent Failure in LLM Code Repair

Key points

• When code-repair agents fail, final tests reveal that a patch is wrong but not which earlier commitment caused it or which software obligation failed.1
• We define plausible but incorrect software mechanisms, place each in a model's hypothesis, implementation plan, or both, and evaluate the generated code with an Oracle and targeted diagnostic coordinates.1
• Moving the misconception from hypothesis to plan increased failure by 32.2% in PersistQueue and 28.9% in Dijkstra after matching the experimental mix.1
• This creates a measured connection from upstream misconception to code mechanism and failure form.1

Sources (1)

• [1]Poison Valleys: Diagnosing Path-Dependent Failure in LLM Code Repair
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 08:49 AM

  Poison Valleys traces this path in 1,844 model requests on two synthetic Python cases: PersistQueue adds atomic batch claiming to a disk-backed queue, while Dijkstra adds source-sensitive filtering to shortest-path search.

  When code-repair agents fail, final tests reveal that a patch is wrong but not which earlier commitment caused it or which software obligation failed.

Extractive summary: sentences quoted from the sources.