Beyond the record: evaluating probabilistic AI weather forecasts deep in the tail

Key points

• We compare four state-of-the-art AI models (ArchesWeatherGen, U-Cast, WeatherNext2 and GenCast-mini) with the physics-based IFS-ENS over land in 2022, at 1.5° resolution and lead times of 2--10 days.1
• Monthly ERA5 records from 1979--2021 define common historical boundaries, and we characterize new records set in 2022 by their depth, i.e., the exceedance margin relative to local historical variability.1
• Verified against ERA5, AI models outperform IFS-ENS for heat and cold records at almost every lead and depth.1
• These findings establish that native probabilistic AI ensembles retain, and for heat increase, their advantage over IFS-ENS as thresholds move beyond local records, although all advantages narrow with lead time.1

Sources (1)

• [1]Beyond the record: evaluating probabilistic AI weather forecasts deep in the tail
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 09:35 PM

  Can probabilistic artificial intelligence (AI) weather forecasts retain skill beyond their training domain?

  We compare four state-of-the-art AI models (ArchesWeatherGen, U-Cast, WeatherNext2 and GenCast-mini) with the physics-based IFS-ENS over land in 2022, at 1.5° resolution and lead times of 2--10 days.

Extractive summary: sentences quoted from the sources.