CodeWalk: Generating Coding Benchmarks by Walking a Problem Graph
We formulate problem generation as code composition: verified atomic problems are combined into level-$N$ tasks, and a task is kept only if every atomic problem remains unsolved and a joint reference solution passes the same check used at evaluation time.
ProofPaper ↗
Key points
- High-quality coding problems are expensive to collect.
- In this work, we show that new coding problems can be created at almost no cost by composing existing ones.
- We instantiate this approach in four coding settings: multi-bug repair, proof completion, complexity analysis, and clone equivalence.
- The generated tasks are also useful as training data: rejection-sampling fine-tuning on them improves Qwen3.5-9B at every composition level.
Sources (1)
- [1]CodeWalk: Generating Coding Benchmarks by Walking a Problem GrapharXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 04:57 AM
We formulate problem generation as code composition: verified atomic problems are combined into level-$N$ tasks, and a task is kept only if every atomic problem remains unsolved and a joint reference solution passes the same check used at evaluation time.
High-quality coding problems are expensive to collect.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 11, 2026Apertus 1.5 Release Notes
- Oct 10, 2026TIDE: Trajectory-Informed Skill Distillation for Efficient Text-to-Video Prompt Optimization
- Oct 10, 2026DiffGA: Steering Frozen Diffusion Models for Materials Design via Evolutionary Search
- Oct 9, 2026Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices
- Oct 9, 2026Continually Post-training Small Models via Synthetic Self-distillation
- Oct 9, 2026Rethinking Relative Policy Differences Transfer of On-Policy Distillation