Foreword: Pre-Generation Prediction and Characterisation of LLM Test Failures using the Jacobian Lens

Key points

• Testing LLM-based software is expensive: each input needs several sampled responses, each judged by a costly oracle.1
• Foreword pairs these representations with two estimators: a sparse logistic-regression probe or dual Gaussian Mixture Models (dual GMMs), which also yield a reference density (to flag inputs the references do not cover) and failing-class components (to group similar failures).1
• On ten tasks and three open-weight LLMs, all four combinations of representations and estimators outperform Clotho on average, both for prediction of failure (ROC-AUC improves by 0.14 on average and by up to 0.31) and test prioritisation (15% more failures among the 100 top-ranked inputs on average, up to 33%).1
• Importantly, our qualitative case study shows that Foreword supports semantic characterisation of failures by pointing to ambiguities in labels or task specifications.1

Sources (1)

• [1]Foreword: Pre-Generation Prediction and Characterisation of LLM Test Failures using the Jacobian Lens
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 01:23 PM

  We present Foreword, a framework built on two ideas: learn from both passing and failing references, and represent inputs through the Jacobian lens (J-lens), an interpretability technique that reads intermediate activations out as scores over vocabulary tokens.

  Testing LLM-based software is expensive: each input needs several sampled responses, each judged by a costly oracle.

Extractive summary: sentences quoted from the sources.