Robustness to Japanese Character-Transposition Errors

Determine the robustness of Japanese large language models to character-transposition typographical errors that corrupt Romanized Japanese input and produce meaningless strings after IME conversion.

Background

The evaluation finds that Character Transposition errors produce substantial accuracy declines in Japanese LLMs. Such errors can transform a meaningful Japanese phrase into a meaningless string through the interaction between Romanized input and Japanese IME conversion. Although surrounding contextual cues may sometimes allow a model to infer the intended meaning, the paper explicitly leaves the robustness of Japanese LLMs to these errors unresolved.