Validate graphemes as the appropriate unit for Thai and abjad scripts

Determine whether graphemes are the appropriate unit for OCR severity evaluation in Thai, which does not write word boundaries, and in abjad scripts, which leave vowels unwritten.

Background

The proposed severity-weighted character error rate operates on extended grapheme clusters rather than words, allowing it to evaluate scripts without requiring word segmentation. The authors report validation by readers in only three languages and explicitly note that they have not tested whether graphemes are the correct unit for Thai or abjads. This leaves unresolved whether the metric’s basic unit adequately represents reader-perceived errors in scripts with different writing conventions.

References

We have not tested whether graphemes are the right unit in Thai, which writes no word boundaries, or in the abjads, which leave vowels unwritten.

— Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCR  (2610.20461 - Nguyen et al., 14 Oct 2026) in Section 6, Limitations, paragraph “Languages”