Establish whether recoverability is the correct evaluation construct

Determine whether recoverability is the appropriate construct for measuring the severity of OCR transcription errors.

Background

The metric assigns severity according to how readily a reader can notice and repair an OCR error, and the human studies evaluate whether the language-model judge applies this recoverability-based rubric in a manner consistent with readers. However, agreement with readers does not establish that recoverability itself is the correct construct for OCR evaluation. The authors explicitly leave that foundational question unresolved.

References

The human studies therefore test whether the judge applies the rubric as readers do and leave open whether recoverability is the right construct.

— Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCR  (2610.20461 - Nguyen et al., 14 Oct 2026) in Section 6, Limitations, paragraph “Construct”