ClinEdge: An Expert-Graded Benchmark of Real Patient Edge Cases Where Frontier Multimodal Models Fail
Most benchmarks for large language models in medicine are built from licensing examinations and theoretical questions.
ProofPaper ↗
Key points
- The few that draw on real clinical data do so only in part, and rarely require a model to read a patient's own imaging.
- ClinEdge addresses this gap with 32 real, de-identified patient cases from active clinical practice, each evaluated across seven frontier systems for a total of 224 expert-graded evaluations.
- Each case pairs primary imaging with a short vignette, contains a feature known to defeat pattern-matched reasoning, and is scored against the patient's documented clinical course.
- Four cases are released in full; the remaining 28 are held out, and we invite frontier AI labs and healthcare AI companies to evaluate their models against them (research@futrbridge.com)
Sources (1)
- [1]ClinEdge: An Expert-Graded Benchmark of Real Patient Edge Cases Where Frontier Multimodal Models FailarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 12:17 AM
Most benchmarks for large language models in medicine are built from licensing examinations and theoretical questions.
The few that draw on real clinical data do so only in part, and rarely require a model to read a patient's own imaging.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 11, 2026ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs
- Oct 10, 2026UltraBERT: Beyond the Limit of BERT Pre-Training
- Oct 10, 2026ProgAgent: Learning Tool Orchestration through Programmatic Training
- Oct 10, 2026Vela: User Behavior as Supervision Sparks LLM Intelligence
- Oct 9, 2026New Math from OpenAI
- Oct 9, 2026Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding