ClinicalBench tests whether retrieval systems can handle the messy realities of clinical notes—negation ("no fever"), temporality (past vs present), and attribution (family history vs patient symptoms)—before feeding information to a reasoning model. The authors build a system (EpiKG) that tags every fact with assertion labels and routes retrieval by question intent, then benchmark it on 400 questions over real MIMIC-IV patient records. Their assertion-aware retrieval improves exact-match accuracy by 22 percentage points over a standard dense retrieval baseline, with physician adjudication confirming the gains.
Main takeaways:
- Clinical reasoning benchmarks usually test on clean inputs, but real EHR retrieval requires handling negation, time, and attribution
- Assertion-aware knowledge-graph retrieval (tracking "negated" vs "affirmed" and "past" vs "present") beats dense embedding retrieval by +22 percentage points
- The gain is larger (+39.5pp) on questions where keyword matching is deterministic, smaller on ambiguous cases
- Larger language models benefit less from better retrieval (beta=-1.12), possibly because they can compensate for retrieval errors
- Physician review found 56% of auto-generated reference answers were defective, highlighting evaluation challenges in clinical NLP