Hidden Data Challenges in Clinical Machine Learning

Key points

• Using MIMIC-IV as a case study, we identify and quantify violations of three complementary forms of preprocessing alignment, i.e., temporal, semantic, and population alignment, which manifest as temporal leakage, feature-task misalignment, and biased imputation.1
• We show how ambiguities in timestamp semantics, episode definitions, endpoint construction, modality selection, and missingness handling can alter the information available to a model and affect the interpretation of its predictions.1
• Through analyses of MIMIC-IV data and empirical case studies, we demonstrate that such choices may remain largely invisible in aggregate predictive performance while changing the signals on which models rely.1
• We derive practical recommendations for temporally valid, clinically aligned, and reproducible preprocessing of MIMIC-IV and related electronic health record datasets.1

Sources (1)

• [1]Hidden Data Challenges in Clinical Machine Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 10:28 AM

  Widely reused preprocessing pipelines for electronic health records can encode assumptions about when information became available, what clinical episode a label represents, and what missingness means.

  Using MIMIC-IV as a case study, we identify and quantify violations of three complementary forms of preprocessing alignment, i.e., temporal, semantic, and population alignment, which manifest as temporal leakage, feature-task misalignment, and biased imputation.

Extractive summary: sentences quoted from the sources.