Sketched Calibration for Conformal Prediction under Covariate Shift

Key points

• We propose sketched calibration: weighted conformal prediction with the ratio of compressed covariates $Z=T(X)$, used in the weights and optionally in the score.1
• Compression never increases the shift-dependent calibration cost, and target coverage is at least $1-α-ΔT$, where the leakage $ΔT$ measures how much of the discarded shift reappears as a change in the law of the response, or of the score, given $Z$.1
• The leakage is a covariance between the discarded shift and the response on the fibres of the sketch; it vanishes when the sketch is sufficient or retains the shift, and a minimax construction shows that no threshold rule for a fixed score can avoid it.1
• In a heteroscedastic simulation where unweighted calibration fails, a one-dimensional sketch cuts the fraction of infinite prediction sets from $37.8%$ to $8.0%$ at mean coverage $92.3%$ for a $90%$ target; nonlinear and real-data experiments show when learned sketches succeed and when they fail.1

Sources (1)

• [1]Sketched Calibration for Conformal Prediction under Covariate Shift
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 11:09 PM

  Weighted conformal prediction corrects for covariate shift by reweighting calibration scores with the likelihood ratio between target and source covariates.

  We propose sketched calibration: weighted conformal prediction with the ratio of compressed covariates $Z=T(X)$, used in the weights and optionally in the score.

Extractive summary: sentences quoted from the sources.