Symphony for Text Generation: Benchmarking Clinical Note Generation
We introduce MedConv, a multilingual dataset of 300 clinical encounters in English, Danish, and German, and use it alongside the Ambient Clinical Intelligence benchmark (ACI-BENCH) to compare Corti, a clinical AI platform, with two leading, accessible ambient scribe software applications built on general-purpose AI.
ProofPaper ↗
Key points
- Ambient documentation systems are rapidly gaining adoption, yet their impact on clinical note quality remains poorly characterized.
- We present a controlled clinical evaluation framework that combines entailment metrics with LLM-judged pairwise comparisons across eight dimensions adopted from PDSQI-9.
- Results show that Corti's API-based text-generation infrastructure is on par with or outperforms leading commercial scribes.
- We present the evaluation methodology and release a dataset to support future reproducible comparison of ambient documentation systems.
Sources (1)
- [1]Symphony for Text Generation: Benchmarking Clinical Note GenerationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 11:13 AM
We introduce MedConv, a multilingual dataset of 300 clinical encounters in English, Danish, and German, and use it alongside the Ambient Clinical Intelligence benchmark (ACI-BENCH) to compare Corti, a clinical AI platform, with two leading, accessible ambient scribe software applications built on general-purpose AI.
Ambient documentation systems are rapidly gaining adoption, yet their impact on clinical note quality remains poorly characterized.
Extractive summary: sentences quoted from the sources.