Wet-Biased Data Pipelines Mislead the Training and Evaluation of Generative Precipitation Nowcasting

Key points

• Selecting test cases based on target observations is known to favor forecasts that match the selected distribution rather than the intended deployment distribution.1
• We show analytically how this target-dependent sampling also shifts the conditional distribution learned by generative models.1
• Using a fixed satellite-to-radar diffusion model, we demonstrate that target-dependent test sampling can reverse the calibration assessment.1
• Utilizing classifier-free guidance as an example, we demonstrate how the contradictory calibration assessment can misdirect downstream model development.1

Sources (1)

• [1]Wet-Biased Data Pipelines Mislead the Training and Evaluation of Generative Precipitation Nowcasting
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 02:28 PM

  Generative models for precipitation nowcasting are trained and evaluated predominantly on wet-biased datasets, which over-represent precipitation events to emphasize rare extremes.

  Selecting test cases based on target observations is known to favor forecasts that match the selected distribution rather than the intended deployment distribution.

Extractive summary: sentences quoted from the sources.