Detecting Spin in Clinical Trials with Large Language Models

Key points

• This is particularly critical in medicine, where spin is present in more than 50% of randomized controlled trials that fail to reach statistical significance.1
• The comparison of primary and reported outcomes is crucial for detecting several types of spin, including outcome switching.1
• We used 300 pairs of outcomes labeled with semantic similarity to develop a system for automatic detection of outcome switching.1
• We evaluated baseline text similarity models and open-source LLMs using generated similarity scores and the Youden index to determine the classification threshold.1

Sources (1)

• [1]Detecting Spin in Clinical Trials with Large Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 12:33 PM

  Spin in clinical trials includes reporting practices that distort the presentation of results.

  This is particularly critical in medicine, where spin is present in more than 50% of randomized controlled trials that fail to reach statistical significance.

Extractive summary: sentences quoted from the sources.