ResearchResearch paperLarge Language Models · Interpretability · Speech & Audio1 source · Oct 20, 2026

EviDial: When Correct Depression Predictions Lack Evidence

Depression dialogue benchmarks commonly assess models by comparing predicted symptom labels with clinical scale scores.

However, a prediction may agree with a benchmark label even when the dialogue lacks enough evidence to justify it.
We introduce EviDial, a benchmark that distinguishes label agreement from evidence-grounded assessment.
EviDial comprises 382 depression dialogues from three sources and 2,744 participant-symptom cells, each classified as having absent, partial, conflicting, or sufficient evidence.
Because evidence sufficiency lacks a directly observable gold standard, we derive calibrated evidence references from multiple judgments using generalizability theory, many-facet Rasch measurement, and random-effects latent class analysis.

[1]EviDial: When Correct Depression Predictions Lack Evidence
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 02:56 AM
Depression dialogue benchmarks commonly assess models by comparing predicted symptom labels with clinical scale scores.
However, a prediction may agree with a benchmark label even when the dialogue lacks enough evidence to justify it.

Extractive summary: sentences quoted from the sources.

Before this