CTPN: Text Detection via Convolution & Bi-LSTM

• CTPN is a connectionist text proposal network that detects text lines in natural images by leveraging vertically anchored convolutional features and bidirectional LSTMs.
• It employs a unique vertical anchor mechanism and side-refinement to achieve precise localization and robust chaining of text proposals.
• The network is end-to-end trainable using a multi-task objective and benefits from guided CNN enhancements to deliver state-of-the-art performance.

The Connectionist Text Proposal Network (CTPN) is a deep neural architecture for detecting text lines in natural images. CTPN introduces a vertical anchor mechanism for accurate localization and connects sequential region proposals via a bidirectional recurrent neural network (Bi-LSTM) to aggregate contextual information. This model is end-to-end trainable, obviating the need for multi-stage post-processing pipelines commonly used in earlier methods. CTPN forms the backbone of state-of-the-art scene text detectors and is further accelerated and enhanced by frameworks such as Guided CNN (Tian et al., 2016, Yue et al., 2018).

1. Network Architecture and Pipeline

CTPN utilizes a convolutional-recurrent design. It begins with a base convolutional network—typically VGG-16 with layers up to conv5—transforming an input image into a feature map with a stride of 16 and a receptive field of $228 \times 228$ pixels. Over this feature map, a $3 \times 3$ sliding window extracts local features which are then processed by an in-network Bi-LSTM horizontally along each row. Each time step corresponds to a spatial position $t$, and the Bi-LSTM aggregates information across the row using a hidden state of dimension $256$ ($128$ in each direction). The aggregated hidden vector $H_t$ is fed into a $512$-dimensional fully connected layer, from which three sibling $1 \times 1$ convolutional output heads are branched:

• Classification head: softmax scores for text/non-text across $k=10$ vertical anchors
• Regression head: predicts vertical coordinates using parameterization $v_c, v_h$ for each anchor
• Side-refinement head: outputs $3 \times 3$0, an offset for each anchor's horizontal boundary

This pipeline outputs fine-scale proposals which are linked and refined into full text line detections (Tian et al., 2016).

2. Vertical Anchor Mechanism and Localization

At each position of the conv5 feature map, CTPN defines $3 \times 3$1 vertical anchors of fixed width $3 \times 3$2 pixels (in the input image), with heights geometrically spaced in $3 \times 3$3 pixels and centered at spatially corresponding positions. For each anchor, the regression module predicts normalized offsets $3 \times 3$4: $3 \times 3$5 These allow the recovery of precise bounding box coordinates. Side-refinement is performed by regressing offsets for the leftmost and rightmost proposals, improving horizontal localization.

This approach simplifies the detection of text lines to the detection and chaining of consecutive narrow proposals, substantially improving localization robustness on complex scenes.

3. Recurrent Context Modeling

CTPN’s critical innovation is the inclusion of an in-network Bi-LSTM that operates across feature map rows. For each horizontal sequence of sliding window features $3 \times 3$6, the Bi-LSTM produces context-aware representations. The forward hidden state $3 \times 3$7 and backward hidden state $3 \times 3$8 are concatenated, and the resulting joint state $3 \times 3$9 is used for all output predictions at position $t$0: $t$1 This sequential modeling enables CTPN to utilize dependencies across text proposals, capturing the continuity of text lines and improving performance in ambiguous or occluded cases.

4. Multi-Task Objective and Training Strategy

The loss function in CTPN combines classification, vertical coordinate regression, and side-refinement, optimized jointly over all anchors in a mini-batch: $t$2

• $t$3 is the softmax loss for text/non-text classification.
• $t$4 is the smooth $t$5 loss for regression (vertical coordinates and side offsets).
• Anchor sampling ensures a balanced set with at most 64 positive out of 128 anchors per image.
• The backbone is initialized from pre-trained ImageNet weights, training proceeds with SGD, and only higher layers are unfrozen.

These design decisions yield superior learning efficiency and robust representation across diverse image domains.

5. Inference and Text Line Construction

The inference procedure is as follows:

1. An image is processed through VGG-16 conv5.
2. The $t$6 sliding window features are passed through Bi-LSTM and output heads for each anchor.
3. Anchors with classification score $t$7 are selected; non-maximum suppression (NMS) is applied (IoU threshold 0.5) to produce candidate text proposals.
4. Proposals are linked into chains if horizontally adjacent ($t$8px) with vertical overlap IoU $t$9; chains define text lines.
5. Side-refinement offsets are applied for tighter horizontal boundaries.
6. Final text-line bounding boxes are computed by aggregating constituent proposals.

This inference design eliminates the need for multi-stage candidate grouping or geometric heuristics.

6. Guided CNN Acceleration and Enhancement

The Guided CNN framework further enhances CTPN by introducing a lightweight guidance subnetwork that learns a mask to spatially restrict computation in the primary detector (Yue et al., 2018):

• The guidance mask is learned using a context module with multi-scale pyramid pooling, generating a coarse map indicating likely text regions.
• Primary CTPN convolutions and nonlinearities are computed only where the mask is active, yielding significant speed-ups.
• Background-aware block-wise random synthesis during training randomly activates background blocks, improving generalization and reducing false positives.

Empirically, Guided CTPN achieves $256$0 speed-up on ICDAR 2013 with an increase in F-measure from $256$1 (baseline) to $256$2, with mask coverage $256$3 indicating $256$4 background reduction. The random synthesis strategy with $256$5 provides optimal speed-accuracy tradeoff.

Method	Recall	Precision	F-measure	Speed-up
CTPN (baseline)	0.861	0.899	0.880	×1.0
Guided CTPN	0.874	0.916	0.895	×2.9

These advances demonstrate the extensibility of the CTPN architecture to higher efficiency and accuracy regimes through targeted architectural augmentations.

7. Benchmark Performance and Impact

CTPN achieves state-of-the-art scene text detection results, reporting F-measure scores of $256$6 on ICDAR 2013 and $256$7 on ICDAR 2015, outperforming predecessors by a substantial margin (Tian et al., 2016). Its end-to-end trainable, context-aware design has influenced subsequent region-proposal-based scene text detectors and served as a robust backbone for further innovations such as Guided CNN (Yue et al., 2018). CTPN’s computational efficiency (~0.14 s/image on GPU) and generic applicability to multi-scale, multi-language settings have established it as a foundational text detection module in modern computer vision pipelines.