Fairness under Prediction Error in Predict-then-Optimize Pipelines

Key points

• Fairness constraints in predict-then-optimize pipelines are necessarily enforced on predicted outcomes $\hat{y}$, since true outcomes $y$ are unobserved at decision time.1
• We study whether a demographic parity constraint enforced this way continues to hold against $y$, and how the resulting violation scales with prediction error.1
• We derive a two-part theoretical bound: a closed-form local sensitivity result showing the gap's first-order response to prediction error depends on only two individuals per group, validated against the solver to within three significant figures; and a universal, error-independent ceiling following from the boundedness of the decision variable, confirmed consistent with but looser than the empirical saturation level.1
• A fairness constraint that appears fully satisfied at decision time can mask a substantial, systematic violation of the outcome it was meant to protect.1

Sources (1)

• [1]Fairness under Prediction Error in Predict-then-Optimize Pipelines
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 03:51 PM

  Using a budget-constrained allocation problem solved as a linear program, we show empirically that the true-outcome fairness gap grows with prediction error $σ$ and saturates, well fit by $c(1 - e^{-σ/τ})$ with $c = 0.1545 \pm 0.0103$ and $τ= 3.9045 \pm 0.3745$; its mean is invariant to sample size while its variance decays at the standard $O(1/\sqrt{n})$ rate.

  Fairness constraints in predict-then-optimize pipelines are necessarily enforced on predicted outcomes $\hat{y}$, since true outcomes $y$ are unobserved at decision time.

Extractive summary: sentences quoted from the sources.