The authors argue that automatic speech recognition (ASR) systems commit "epistemic injustice" when they enforce a single transcription standard as ground truth. Different transcription conventions (verbatim vs. cleaned-up) produce different "correct" transcripts for the same speech, and speakers with aphasia—whose disfluencies carry clinical meaning—are systematically penalized when evaluated against "clean" references that treat those features as errors. The paper proposes WER-Range: reporting performance across multiple legitimate transcription conventions instead of assuming one right answer.
Main takeaways:
- Word Error Rate (WER) varies depending on which transcription convention you pick as ground truth—not because the system changed, but because "ground truth" itself is a choice.
- Speakers with aphasia are disadvantaged when their meaningful disfluencies are treated as errors to be removed from the reference transcript.
- The authors introduce Epistemic Injustice Distance (EID) to quantify the harm of enforcing a single standard.
- WER-Range proposes reporting ASR performance across multiple legitimate conventions rather than picking one.
- The problem isn't just differential performance—it's that the evaluation infrastructure lacks the conceptual tools to recognize some speech as legitimate in the first place.