The authors argue that current AI benchmarks in healthcare measure what models know (e.g., medical exam scores) but not whether they perform reliably on real clinical tasks. Frontier models score near-perfect on licensing exams but drop to 0.74–0.85 on clinical documentation, 0.61–0.76 on decision support, and only 0.53–0.63 on administrative workflow tasks. The paper calls for a principled framework for designing benchmarks that test reliability, safety, and clinical relevance under real-world conditions, not just narrow task performance. High benchmark scores create a false sense of deployment readiness.
Main takeaways:
- Frontier models achieve near-perfect scores on medical licensing exams but perform much worse on real clinical tasks (documentation 0.74–0.85, decision support 0.61–0.76, admin/workflow 0.53–0.63)
- The gap between benchmark performance and real-world utility widens as AI systems take on more consequential clinical roles
- Current benchmarks test "what a model knows" rather than "whether it can perform reliably without failing" across complex, high-stakes workflows
- High scores on existing benchmarks give a false sense of deployment readiness
- The field needs systematic methods to measure reliability, safety, and clinical relevance under real-world conditions, not ad hoc dataset construction optimized for narrow tasks