OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents

Key points

• Reliable repository-level coding requires agents not only to generate correct code but also to preserve constraints and respond to tool feedback across long interactions.1
• However, our diagnostic analysis shows that agents often fail to act on relevant information already available in context, while final Pass/Fail outcomes provide only sparse process supervision.1
• Successful rollouts provide autoregressive self-imitation targets, whereas failed rollouts receive Reminder-conditioned reverse-KL supervision from a teacher that re-evaluates the same student-generated prefixes.1
• On SWE-bench Verified, SWE-bench Pro, and SWE-bench Multilingual, OPRD improves the benchmark-averaged task resolution rate by 11.3 points over the open-source Qwen3.5-4B base model.1

Sources (1)

• [1]OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 08:32 AM

  We introduce On-Policy Reminder Distillation (OPRD), which combines Multi-Scope Reminder supervision with outcome-based routing on reminder-free student rollouts.

  Reliable repository-level coding requires agents not only to generate correct code but also to preserve constraints and respond to tool feedback across long interactions.

Extractive summary: sentences quoted from the sources.