Calibration in One Go: Jointly Learning Accurate and Calibrated Neural Networks

Key points

• Neural networks (NNs), while highly flexible and versatile, are often poorly calibrated.1
• Post-hoc calibration methods adjust the outputs of an already trained model using a separate calibration dataset and additional calibration parameters.1
• This formulation eliminates the need for a separate calibration set or a dedicated calibration loss by jointly optimizing accuracy and calibration using cross-entropy, introducing only one additional hyperparameter: the calibrator learning rate.1
• Although our framework supports calibrators ranging from temperature and matrix scaling to NN layers, our results show that using the final classification layer itself as the calibrator yields accurate, well-calibrated models with no additional parameters.1

Sources (1)

• [1]Calibration in One Go: Jointly Learning Accurate and Calibrated Neural Networks
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 02:54 PM

  In this paper, we show that standard (post-hoc) calibrators can be trained end-to-end with the backbone via a bilevel Stackelberg formulation.

  Neural networks (NNs), while highly flexible and versatile, are often poorly calibrated.

Extractive summary: sentences quoted from the sources.