Recursive Recurrent Nets with Attention Modeling for OCR in the Wild (1603.03101v1)

Abstract: We present recursive recurrent neural networks with attention modeling (R$^2$AM) for lexicon-free optical character recognition in natural scene images. The primary advantages of the proposed method are: (1) use of recursive convolutional neural networks (CNNs), which allow for parametrically efficient and effective image feature extraction; (2) an implicitly learned character-level LLM, embodied in a recurrent neural network which avoids the need to use N-grams; and (3) the use of a soft-attention mechanism, allowing the model to selectively exploit image features in a coordinated way, and allowing for end-to-end training within a standard backpropagation framework. We validate our method with state-of-the-art performance on challenging benchmark datasets: Street View Text, IIIT5k, ICDAR and Synth90k.

Recursive Recurrent Nets with Attention Modeling for OCR in Natural Scenes

Introduction

The paper presents a significant contribution to the domain of Optical Character Recognition (OCR) in natural scene images through the introduction of Recursive Recurrent Neural Networks with Attention Modeling (R²AM). One of the previous challenges within photo OCR has been reliance on constrained lexicon-dependent methodologies, which often necessitate known word lists during inference stages. These methods, while effective in specific scenarios with predefined lexicons, face considerable limitations in handling text without such constraints. This paper shifts focus towards an unconstrained recognition setting, proposing a model that efficiently performs lexicon-free optical character recognition within natural scene images.

Methodology

The proposed method comprises three primary innovations:

1. Recursive Convolutional Neural Networks (CNNs): The paper employs recursive CNNs with weight-sharing tactics for efficient and powerful image feature extraction. This recursive approach broadens the receptive field of convolutions without inflating the model's parametric complexity, enhancing the model's depth and capacity to capture long-range dependencies crucial for OCR tasks.
2. Recurrent Neural Networks (RNNs) for LLMing: Upon feature extraction, R²AM leverages RNNs to implicitly learn character-level LLMing. Traditional approaches have used N-grams for LLM integration, yet this inclusion typically demands exhaustive computation and expansive model output layers. R²AM circumvents these limitations by allowing RNNs to automatically code language statistics from pixel sequences of image word strings.
3. Soft-Attention Mechanism: The model incorporates a soft-attention layer that enables dynamic, context-driven selection of salient image features during the character sequence decoding process. This facilitates end-to-end learning with standard backpropagation, optimizing the model’s performance for sequential attention-based OCR tasks.

Experimental Results

Significant experimental validation was conducted using challenging benchmark datasets including Street View Text, IIIT5k, ICDAR, and Synth90k. The results revealed that R²AM achieved state-of-the-art performance across these datasets, underscoring the efficacy of recursive CNNs and the attention-integrated RNN approach. Notably, the model demonstrated a 9% increase in accuracy on the Street View Text dataset and an 8.2% enhancement on the ICDAR 2013 dataset compared to previous best-performing methods. This improvement is particularly remarkable given that these datasets typically pose considerable challenges due to the diversity and complexity of scene-based text.

Implications and Future Directions

The methodology presented is pivotal in advancing both theoretical and practical aspects of scene text recognition. From a theoretical perspective, the synergistic integration of recursive structures, recurrent learning, and attention mechanisms represents a valuable framework extending beyond simple feature extraction, offering a robust solution to sequence learning in complex visual environments. Practically, the proposed R²AM system promises enhancements in applications requiring OCR capabilities, including automated navigation systems, assistive devices for the visually impaired, and real-time mobile text translation.

Future research directions could delve into further character sequence optimization using advanced forms of RNNs such as LSTMs or GRUs and the integration of additional contextual language predictability models. Expanding this approach to broader language domains or tasks with cross-linguistic variances would also be noteworthy. Additionally, researchers might explore R²AM’s applicability in other sequential pattern recognition contexts to assess generalizability across varied OCR-related deployment scenarios.

Conclusion

The Recursive Recurrent Nets with Attention Modeling framework offers a sophisticated solution for lexicon-free OCR in natural scenes. Through recursive CNNs, RNN-based LLMing, and a soft-attention framework, this paper successfully addresses several long-standing challenges in photo OCR, establishing a new efficacy standard in the field. As the OCR domain advances, methodologies like R²AM will likely inform future innovations in robust text recognition technologies.