OCR Language Models with Custom Vocabularies (2308.09671v1)

Abstract: LLMs are useful adjuncts to optical models for producing accurate optical character recognition (OCR) results. One factor which limits the power of LLMs in this context is the existence of many specialized domains with language statistics very different from those implied by a general LLM - think of checks, medical prescriptions, and many other specialized document classes. This paper introduces an algorithm for efficiently generating and attaching a domain specific word based LLM at run time to a general LLM in an OCR system. In order to best use this model the paper also introduces a modified CTC beam search decoder which effectively allows hypotheses to remain in contention based on possible future completion of vocabulary words. The result is a substantial reduction in word error rate in recognizing material from specialized domains.