Scene Text Recognition from Two-Dimensional Perspective (1809.06508v2)

Abstract: Inspired by speech recognition, recent state-of-the-art algorithms mostly consider scene text recognition as a sequence prediction problem. Though achieving excellent performance, these methods usually neglect an important fact that text in images are actually distributed in two-dimensional space. It is a nature quite different from that of speech, which is essentially a one-dimensional signal. In principle, directly compressing features of text into a one-dimensional form may lose useful information and introduce extra noise. In this paper, we approach scene text recognition from a two-dimensional perspective. A simple yet effective model, called Character Attention Fully Convolutional Network (CA-FCN), is devised for recognizing the text of arbitrary shapes. Scene text recognition is realized with a semantic segmentation network, where an attention mechanism for characters is adopted. Combined with a word formation module, CA-FCN can simultaneously recognize the script and predict the position of each character. Experiments demonstrate that the proposed algorithm outperforms previous methods on both regular and irregular text datasets. Moreover, it is proven to be more robust to imprecise localizations in the text detection phase, which are very common in practice.

Scene Text Recognition from Two-Dimensional Perspective: An Overview

The paper "Scene Text Recognition from Two-Dimensional Perspective" by Liao et al. addresses the challenges in scene text recognition (STR) by proposing a novel approach that diverges from the prevalent sequence prediction methods. Traditional STR methods, inspired by one-dimensional sequence models used in speech recognition, simplify the task by encoding text images into feature sequences. These methods are effective for regular horizontal text but experience difficulties with irregular text arrangements prevalent in natural scenes.

Liao et al. introduce a model called Character Attention Fully Convolutional Network (CA-FCN), which frames text recognition as a two-dimensional prediction problem. The proposed method leverages a semantic segmentation network that includes an attention mechanism for character localization. This approach allows CA-FCN to recognize text of arbitrary shapes directly from images without constraining them to a linear sequence, thereby preserving spatial information critical for understanding complex text layouts.

Methodology

The CA-FCN architecture builds on the Fully Convolutional Network (FCN) concept by incorporating a character attention module to refine character recognition in cluttered environments. The network's backbone is VGG-16, augmented with a pyramidal structure and deformable convolutions to handle textual variations such as curves and orientations. The attention module enhances character feature extraction by focusing on relevant parts of the image while mitigating background noise.

The paper emphasizes the two key innovations:

1. Character Attention Module: It differentiates character pixels from background distractions and separates adjacent characters. This module is applied at several stages within the network, ensuring both fine and coarse feature information is considered.
2. Deformable Convolution: These adapt the traditional convolutional layers to accommodate spatial deviations and enable the network to respond flexibly to text variations in both shape and orientation.

Training is conducted on a vast synthetic dataset, ensuring a model robust to real-world variations. The word formation module interprets the output of CA-FCN to reassemble characters into meaningful words based on their spatial organization within the image.

Results and Implications

The CA-FCN demonstrates substantial improvements over traditional methods, achieving state-of-the-art results across several benchmark datasets, including IIIT5k, SVT, IC13, and CUTE. Notably, the model excels in environments with irregular text forms, addressing the limitations of sequence-based approaches. The model's robustness to imprecise localization—a common challenge in practical applications—further underscores its applicability.

The performance showcases the merit in leveraging a two-dimensional understanding for scene text, offering improved resilience to irregularities and errors introduced during text detection. The implications of this work extend into enhancing OCR systems, improving human-computer interaction technologies, and advancing autonomous navigation systems where accurate text recognition is paramount.

Future Directions

There's a prospect for integrating a learnable component within the word formation module to enhance adaptability and accuracy. Additionally, merging CA-FCN into a holistic end-to-end text spotting pipeline could yield significant advancements in STR applications across various domains.

In summary, this work by Liao et al. represents a meaningful stride in scene text recognition, providing a robust framework adaptable to diverse textual appearances observed in natural settings. The implementation of a two-dimensional perspective challenges the previous reliance on sequence alignment and underlines the importance of spatial context in enhancing recognition accuracy.