Passing Tests, Missing Semantics: Evidence of Interpretive Lock-In in Plan-Based DataFrame Code Generation
We investigate this risk on 30 DataFrame tasks derived from TPC Benchmark DS, comparing Frozen One-Pass, Test-Guided Revision, and PlanAudit-Guided Revision.
ProofPaper ↗
Key points
- Planning and revision are intended to bring generated code closer to the task specification.
- Yet a mistaken interpretation can persist across the plan, implementation, and tests - a pattern we call interpretive lock-in.
- Same-engine SQL re-execution reproduced 28/30 references; all 77 test-passing runs on these tasks still failed reference agreement.
- Fresh generation on ten targeted tasks produced 0/30 reference matches and 25/25 false passes among test-passing runs.
Sources (1)
- [1]Passing Tests, Missing Semantics: Evidence of Interpretive Lock-In in Plan-Based DataFrame Code GenerationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 05:41 PM
We investigate this risk on 30 DataFrame tasks derived from TPC Benchmark DS, comparing Frozen One-Pass, Test-Guided Revision, and PlanAudit-Guided Revision.
Planning and revision are intended to bring generated code closer to the task specification.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 11, 2026AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards
- Oct 11, 2026Apertus 1.5 Release Notes
- Oct 10, 2026LYNX: Analytical Program Synthesis via Grounding and Closed-Loop Refinement
- Oct 10, 2026vllm-sr/d3
- Oct 10, 2026KV4-RL: Unlocking Faster Asynchronous Rollouts for RLVR with INT4 KV Cache Quantization
- Oct 9, 2026LiquidAI/LFM2.5-24B-A2B