From Outcomes to Components: A Counterfactual Evaluation Framework for LLM-Proposed Repairs
An LLM-generated repair action specifies which records to change and by how much.
ProofPaper ↗
Key points
- A failed action may still contain a useful target set or numerical update, depending on how the two are paired.
- We evaluate this problem in traffic networks and databases, where aggregate observations obscure the records that need correction.
- We examine the pairings by executing each combination from the same initial state, using a reference action that restores the affected records.
- In database repair, replacing only the proposed update with the reference update worsens 17 of 108 model actions, all of which select unaffected records.
Sources (1)
- [1]From Outcomes to Components: A Counterfactual Evaluation Framework for LLM-Proposed RepairsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 04:08 PM
An LLM-generated repair action specifies which records to change and by how much.
A failed action may still contain a useful target set or numerical update, depending on how the two are paired.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
- Oct 19, 2026From Black Box to Controller: Inference-Time Steering of LLM via Feature Amplification
- Oct 18, 2026MOSAIC: Multi-Objective Search for Allocating Inference Cache Across LLM Decoder Layers
- Oct 18, 2026Conjugate Compilation: Self-Improving Agentic Co-Compilation for Speculative Decoding
- Oct 17, 2026Which Long Documents Matter? A Controlled Study of Data Selection for Context Extension
- Oct 17, 2026EdgeKernelBench: Unified LLM-Driven Kernel Generation for Edge Backends