SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition

Key points

• Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state.1
• Successful reference executions validate task feasibility and measure interaction workload, while programmatic evaluators assess local completion and whole-task success.1
• We evaluate five language models on 1,050 tasks with reference interaction workloads ranging from 64K to 1M tokens.1
• SkillHorizon provides a configurable testbed for studying sustained progress, dependency coordination, and complete task delivery in growing workflows.1

Sources (1)

• [1]SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 02:36 PM

  We introduce SkillHorizon, a benchmark that constructs long-horizon tasks through skill composition in seven business environments.

  Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state.

Extractive summary: sentences quoted from the sources.