Deconstructing Multi-Task Active Learning: The Paradox of Gradient Conflict and Orthogonal Decomposition

Key points

• We propose a shift from combination to deconstruction and introduce a principled framework that rethinks MTL-AL along three orthogonal dimensions.1
• First, we identify the Conflict Paradox: although the optimization literature treats gradient conflict as interference to be suppressed, penalizing conflict when ranking candidates is measurably worse than not scoring it at all.1
• Second, we introduce Orthogonal Decomposition, separating acquisition into Informativeness and Exploration.1
• Third, we propose Cross-Task Mutual Information (CTMI), a covariance-based statistic of cross-task dependency computed from MC-dropout task entropies.1

Sources (1)

• [1]Deconstructing Multi-Task Active Learning: The Paradox of Gradient Conflict and Orthogonal Decomposition
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 03:32 PM

  Existing multi-task active learning (MTL-AL) methods predominantly rely on linear scalarization of heterogeneous objectives such as uncertainty, diversity, and gradient conflict.

  We propose a shift from combination to deconstruction and introduce a principled framework that rethinks MTL-AL along three orthogonal dimensions.

Extractive summary: sentences quoted from the sources.