Harness-Delta Attribution: Dissecting Gains From Harness Evolution
In this work, we introduce Harness-Delta Attribution (HDA), an evaluation protocol that dissects an evolution gain into artifact-driven overfitting, gains explained by test-time scaling, and the remainder, which gives an upper bound on residual generalizable improvement.
ProofPaper ↗
Key points
- Harness evolution automatically optimizes the prompts, skills, tools, and orchestration code around a fixed-weight model, offering a way to improve agentic systems without updating model weights.
- We apply HDA to 11 executor models and three harness-evolution frameworks across math, coding, creativity, and agentic tasks.
- In 21 of these settings that show statistically significant search-set gains, we find that a large share of the improvement is explained by overfitting or test-time scaling.
- These results motivate attribution-aware evaluation in future harness evolution research.
Sources (1)
- [1]Harness-Delta Attribution: Dissecting Gains From Harness EvolutionHugging Face Daily Papers · Oct 15, 12:00 AM
In this work, we introduce Harness-Delta Attribution (HDA), an evaluation protocol that dissects an evolution gain into artifact-driven overfitting, gains explained by test-time scaling, and the remainder, which gives an upper bound on residual generalizable improvement.
Harness evolution automatically optimizes the prompts, skills, tools, and orchestration code around a fixed-weight model, offering a way to improve agentic systems without updating model weights.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026Grok Voice 3.0 from SpaceXSI outscores OpenAI and Google on speech-to-speech index
- Oct 14, 2026Research@ 2026: 10 ways we're amplifying human ingenuity
- Oct 13, 2026Plan or Retrieve First? A Scale Study of Tool Discovery Architectures
- Oct 13, 2026Google wants to host your personal vibe-coded apps in the Play Store
- Oct 13, 2026Natural language security queries at Orca Security using Amazon Bedrock
- Oct 13, 2026Input-Conditioned Search for Efficient Test-Time Scaling of Recursive Reasoners