The Optimal Ranking Tradeoffs Between Two Ranking Scores in Two-Class Classification

Key points

• So, how can we establish a ranking that is an optimal tradeoff between any pair of these scores?1
• We consider the tradeoff between two scores to be optimal when it minimizes the Fréchet variance regarding the Kendall distance (also known as the bubble-sort distance) between rankings.1
• For any given set of classifiers to rank, the optimal tradeoff lies at the center of the shortest (geodesic) path on the manifold of all rankings induced by the ranking scores.1
• In this paper, we present an exact analytical solution for determining a ranking score that is an optimal tradeoff for any given pair of initial scores, in the common case where the class priors are the same for all compared performances.1

Sources (1)

• [1]The Optimal Ranking Tradeoffs Between Two Ranking Scores in Two-Class Classification
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 11:00 AM

  The accuracy, true negative rate (negative class recall), true positive rate (positive class recall), negative predictive value (negative class precision), positive predictive value (positive class precision), class-specific intersections over unions, class-specific F-scores, Cohen's kappa, and balanced accuracy are scores that one intuitively wants to maximize when developing machine learning met

  So, how can we establish a ranking that is an optimal tradeoff between any pair of these scores?

Extractive summary: sentences quoted from the sources.