GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation
We introduce GIFT-Eval-CTX, a benchmark that isolates whether forecasting methods can translate verified future information into accurate numerical trajectories.
ProofPaper ↗
Key points
- Contextual forecasting seeks to improve time series forecasting by incorporating forward-looking information, such as planned events, external conditions, and expert expectations.
- We construct these annotations with an agentic pipeline combining LLM-based discovery and deterministic code validation.
- We benchmark TSFMs, frontier LLMs, and tool-using agentic systems.
- The results show that structured effects reduce aggregate relative MASE by 40.7% for GPT-5.6 Sol and 29.6% for Claude Opus 4.8 relative to history-only prompting.
Sources (1)
- [1]GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model EvaluationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 06:21 AM
We introduce GIFT-Eval-CTX, a benchmark that isolates whether forecasting methods can translate verified future information into accurate numerical trajectories.
Contextual forecasting seeks to improve time series forecasting by incorporating forward-looking information, such as planned events, external conditions, and expert expectations.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards
- Oct 12, 2026Compositional RL: Evaluating and Training LLMs for Collaborative Reasoning
- Oct 12, 2026Redesigning source control at Anthropic
- Oct 11, 2026More Reasoning Is Not Always Better: Executing Clinical Guidelines with LLMs and Rules
- Oct 11, 2026Putting Earth AI to work: Geospatial agents for our planet's biggest challenges
- Oct 11, 2026AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards