Revisiting Tabular Deep Regression: Objectives and the Limits of Global Early Stopping

Key points

• In this work, we address this gap by performing a systematic re-evaluation of regression objectives across a broad collection of established tabular datasets and multiple neural backbones.1
• Our comparison reveals that many proposed objectives provide limited or inconsistent gains over a properly tuned MSE baseline, while quantile loss yields consistent aggregate improvements over MSE across the tested backbones.1
• Motivated by this limitation, we introduce a heteroskedasticity-aware extension of quantile loss that uses input-dependent label smoothing as a potential remedy and improves over quantile loss on several backbones.1
• Overall, our results provide a modern empirical picture of regression objectives for tabular deep learning and highlight how differences in optimal stopping times across uncertainty regions can limit the effectiveness of global early stopping.1

Sources (1)

• [1]Revisiting Tabular Deep Regression: Objectives and the Limits of Global Early Stopping
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 05:49 PM

  While deep learning models for tabular data have recently received significant research attention, the choice of objective functions for regression tasks remains an understudied degree of freedom.

  In this work, we address this gap by performing a systematic re-evaluation of regression objectives across a broad collection of established tabular datasets and multiple neural backbones.

Extractive summary: sentences quoted from the sources.