ResearchResearch paperTraining & Scaling · Large Language Models · Evaluation & Benchmarks1 source · Oct 12, 2026

IND-Eval: Aligning Evaluation and Simulation for Industrial Time Series Foundation Models

We introduce IND-Eval, a public benchmark that organizes industrial time series across domains into source-specific forecasting tasks.

Evaluating Chronos-2, TiRex-2, and TimesFM-3, we find that TimesFM-3 achieves the lowest source-balanced aggregate nMASE and nCRPS and leads seven of nine sources under each metric; source-specific exceptions reinforce the need for domain-representative evaluation for model selection.
These findings also motivate improving domain alignment during pretraining, where real industrial data are often too scarce to close domain-specific gaps directly; we therefore investigate whether a domain's statistical structure can guide simulator design.
In a case study on two offshore well-monitoring regimes, we improve a general simulator's coverage using an oscillator-augmented simulation pipeline.
This lowers forecasting error in the target domain but reduces performance on the subset of GIFT-Eval with natively univariate targets.

[1]IND-Eval: Aligning Evaluation and Simulation for Industrial Time Series Foundation Models
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 12:08 PM
We introduce IND-Eval, a public benchmark that organizes industrial time series across domains into source-specific forecasting tasks.
Evaluating Chronos-2, TiRex-2, and TimesFM-3, we find that TimesFM-3 achieves the lowest source-balanced aggregate nMASE and nCRPS and leads seven of nine sources under each metric; source-specific exceptions reinforce the need for domain-representative evaluation for model selection.

Extractive summary: sentences quoted from the sources.

Before this