Corner Proposal for Multi-Oriented Text Detection

• The paper introduces a corner-based region proposal mechanism that constructs quadrilateral text regions by linking detected corners, bypassing pre-defined anchors.
• It utilizes a VGG-16 backbone with multi-level feature fusion and directional encoding to accurately predict four distinct corner types on dense grids.
• Dual-RoI Pooling refines rotation-aware features in the second stage, significantly boosting performance metrics on ICDAR benchmarks.

Corner Proposal denotes the corner-based region proposal mechanism introduced for multi-oriented scene text detection in “Detecting Multi-Oriented Text with Corner-based Region Proposals” (Deng et al., 2018). In that framework, a text instance is hypothesized not by sliding windows or predefined anchors, but by detecting text-box corners on a dense grid and linking compatible corners into two intersecting diagonals that uniquely determine a quadrilateral. The method is explicitly anchor-free, geometry-adaptive, and embedded in a two-stage detector whose second stage uses Dual-RoI Pooling for rotation-aware feature extraction and refinement (Deng et al., 2018).

1. Concept and geometric motivation

A corner proposal is defined as a quadrilateral text region produced by first detecting corners and then linking them into a valid four-corner configuration (Deng et al., 2018). The method uses four corner types—top-left, top-right, bottom-right, and bottom-left—and treats text localization as a bottom-up geometric construction rather than an anchor-matching problem.

The motivation is tied to the limitations of anchor or sliding-window detectors for scene text. Manually designed anchors must cover multiple scales, aspect ratios, and orientations, yet long, thin, or heavily rotated text can still be mismatched. The corner formulation avoids this enumeration because any text box with arbitrary aspect ratio and orientation is determined by its four corners. As a result, the proposal geometry is adaptive to the text instance itself rather than to a predefined anchor catalog (Deng et al., 2018).

This mechanism is especially tailored to multi-oriented text. Only two diagonal types are valid: top-left to bottom-right and top-right to bottom-left. Once both diagonals are instantiated and shown to intersect properly, the resulting quadrilateral aligns directly with the text geometry. The proposal score is then assigned by a Naïve Bayes product of the four corner probabilities, which favors geometrically consistent, high-confidence configurations (Deng et al., 2018).

2. Corner detection network and directional encoding

The detector uses VGG-16 as backbone and constructs its detection feature map through multi-level fusion at “conv4” resolution with stride $1/8$ (Deng et al., 2018). The fusion is lightweight rather than a full FPN: a deconvolution upsamples, a $3\times 3\times 64$ convolution with stride $2$ subsamples, and concatenation is followed by a $1\times 1\times 512$ convolution. The stated goal is to combine fine spatial detail with higher-level semantics (Deng et al., 2018).

Each of the four corner types is handled by an independent branch. For corner type $i$, the branch predicts a multi-class heatmap with $K+1$ channels, where $k=0$ denotes background and $k=1,\dots,K$ denote discretized link directions. The per-pixel corner probability is the sum of the non-background channels,

$P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$

In the reported configuration, $K=24$, corresponding to $3\times 3\times 64$0 per direction bin (Deng et al., 2018).

The directional label is attached to a potential diagonal rather than to an entire box. For a candidate link from corner $3\times 3\times 64$1 to corner $3\times 3\times 64$2, with orientation $3\times 3\times 64$3 relative to the horizontal, the link-direction bin is

$3\times 3\times 64$4

Each detected corner $3\times 3\times 64$5 also carries a predicted direction $3\times 3\times 64$6. To suppress incorrect pairings, the method retains only links satisfying the directional consistency gate

$3\times 3\times 64$7

According to the paper, this gating is particularly useful in dense text scenes because it reduces ambiguous long-range pairings (Deng et al., 2018).

Corner detection is trained with a segmentation-like weighted softmax loss over sampled pixels,

$3\times 3\times 64$8

Class imbalance is handled by class weights $3\times 3\times 64$9, and sampling uses $2$0 pixels per mini-batch per corner type with up to $2$1 positive-to-negative ratio (Deng et al., 2018).

3. Proposal construction by corner linking

Proposal generation proceeds by thresholding each corner map with $2$2, applying local suppression, and keeping the top-$2$3 candidates per type (Deng et al., 2018). The reported hyperparameters are $2$4 during training, $2$5 during testing, and $2$6 corner candidates per type (Deng et al., 2018).

The first diagonal set $2$7 is built from all top-left–bottom-right pairs that satisfy the direction gate. The second diagonal set $2$8 is built analogously from top-right–bottom-left pairs. A quadrilateral proposal $2$9 is formed whenever one diagonal from $1\times 1\times 512$0 and one from $1\times 1\times 512$1 intersect within their segment extents. The corners are then ordered clockwise as $1\times 1\times 512$2 (Deng et al., 2018). The procedure is repeated with the roles of the two diagonal families swapped so that proposal generation is exhaustive (Deng et al., 2018).

Several geometric constraints are imposed. Diagonals must be proper intersecting segments, and corner ordering must remain consistent, for example as a convex quadrilateral in clockwise order (Deng et al., 2018). These conditions, together with directional gating, remove many spurious combinations before any second-stage classification is applied.

The proposal score is

$1\times 1\times 512$3

namely the product of the four corner probabilities at the selected locations (Deng et al., 2018). Proposals are ranked by this score and filtered by quadrilateral IoU NMS. The IoU is defined as

$1\times 1\times 512$4

and polygon area is computed with the shoelace formula,

$1\times 1\times 512$5

The implementation uses the RRPN quadrilateral IoU routine for robust polygon NMS (Deng et al., 2018).

4. Dual-RoI Pooling and second-stage refinement

After corner proposals are generated, each quadrilateral is converted to a rotated rectangle by fitting a minimum-area rectangle,

$1\times 1\times 512$6

This conversion uses minAreaRect (Deng et al., 2018). Standard axis-aligned RoI Pooling is inadequate for such proposals, and the method therefore adopts rotation-aware pooling derived from RRoI Pooling.

The distinctive refinement component is Dual-RoI Pooling. For each rotated RoI $1\times 1\times 512$7, a complementary rotated RoI $1\times 1\times 512$8 is synthesized by swapping height and width and complementing the angle,

$1\times 1\times 512$9

RRoI Pooling is applied independently to $i$0 and $i$1, and the two resulting feature tensors are fused by element-wise addition (Deng et al., 2018). The paper describes this as embedding a rotation-like augmentation inside the region-wise head without extra parameters and without image-level augmentation (Deng et al., 2018).

The region-wise head then predicts text/non-text classification and quadrilateral regression. Classification uses binary cross-entropy,

$i$2

where $i$3 and $i$4 is the text probability (Deng et al., 2018). Localization uses an 8-dimensional regression target for the four corner coordinates with Smooth $i$5 loss,

$i$6

where

$i$7

The corner targets are normalized by the width $i$8 and height $i$9 of the proposal’s minimum-area rectangle (Deng et al., 2018).

Each proposal is assigned to the ground truth with maximum quadrilateral IoU, and the full multi-task objective is

$K+1$0

with all $K+1$1 values set to $K+1$2 (Deng et al., 2018).

5. Training protocol and empirical performance

The training pipeline uses pretraining on SynthText with $K+1$3 images, followed by joint finetuning on ICDAR 2013 training and ICDAR 2015 training sets (Deng et al., 2018). Optimization uses SGD with momentum $K+1$4 and weight decay $K+1$5 (Deng et al., 2018). The learning-rate schedule is $K+1$6 for $K+1$7 iterations and $K+1$8 for $K+1$9 iterations during pretraining, then $k=0$0 for $k=0$1 iterations during finetuning (Deng et al., 2018). Multi-scale training samples the short side randomly in $k=0$2, and no extra image-level augmentation is used (Deng et al., 2018).

At test time, the system forwards the top $k=0$3 scoring proposals after quadrilateral NMS (Deng et al., 2018). Reported results are single-model and single-scale unless otherwise noted (Deng et al., 2018). On ICDAR 2013, the detector attains F-measure $k=0$4, Recall $k=0$5, Precision $k=0$6, and $k=0$7 FPS on a single 1080Ti (Deng et al., 2018). On ICDAR 2015, which emphasizes multi-oriented text, it attains F-measure $k=0$8, Recall $k=0$9, and Precision $k=1,\dots,K$0, surpassing RRPN at $k=1,\dots,K$1 and $k=1,\dots,K$2 at $k=1,\dots,K$3, and remaining close to FTSN at $k=1,\dots,K$4, which uses a stronger ResNet-101 backbone (Deng et al., 2018). On COCO-Text, which is unseen during training, the reported F-measure is $k=1,\dots,K$5, with Recall $k=1,\dots,K$6 and Precision $k=1,\dots,K$7, which the paper uses as evidence of generalization (Deng et al., 2018).

The ablation study on ICDAR 2015 isolates the contribution of the corner proposal mechanism. A Faster R-CNN baseline with conv5 stride 16 yields F-measure $k=1,\dots,K$8. Replacing anchor proposals with CRPN corner proposals and standard RoI Pooling raises this to $k=1,\dots,K$9. Adding feature fusion at stride 8 raises it to $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$0. Replacing standard pooling with RRoI Pooling raises it further to $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$1, and Dual-RoI Pooling gives the final $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$2, corresponding to an additional $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$3 F-measure over RRoI Pooling alone (Deng et al., 2018). The paper concludes from this sequence that CRPN is the main driver of the multi-oriented improvement, while higher-resolution fused features and rotation-aware pooling contribute substantial further gains (Deng et al., 2018).

6. Limitations, implementation, and research context

The proposal is explicitly anchor-free. Positives and negatives are defined by corner labels for $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$4 and by proposal-to-ground-truth matching for $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$5 and $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$6, rather than by anchor IoU assignment (Deng et al., 2018). The implementation is publicly released in Caffe, with key hyperparameters $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$7, $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$8, $P_i(x,y)=\sum_{k=1}^{K} P'_i(k\mid x,y).$9 during training and $K=24$0 during testing, quadrilateral IoU NMS from RRPN, and $K=24$1 proposals retained at test time (Deng et al., 2018).

The paper also states several limitations. Single-character text is often dropped during training for higher precision and may therefore be missed at inference (Deng et al., 2018). Extremely small text requires larger input scaling, which lowers speed (Deng et al., 2018). Dense or adjacent text can still produce incorrect linkages despite directional gating, reflecting what the paper describes as bottom-up error accumulation (Deng et al., 2018). Curved text is not well modeled because the proposal primitive is a quadrilateral, so only parts of strongly curved words may be covered (Deng et al., 2018).

Within the broader detector landscape, the method is positioned against anchor-based text detectors such as RRPN, $K=24$2, and TextBoxes. Its stated advantage is that geometry adaptivity emerges from corner positions rather than from hand-designed rotated anchors (Deng et al., 2018). The paper also distinguishes the method from generic corner-based detectors such as CornerNet: CornerNet predicts top-left and bottom-right corners for generic object boxes, whereas CRPN is specialized for text through four corner types, direction-aware gating, and explicit quadrilateral construction from intersecting diagonals (Deng et al., 2018). This specialization suggests that the “corner proposal” in CRPN is not merely an anchor-free replacement for proposal generation, but a text-specific geometric prior encoded directly in the proposal mechanism (Deng et al., 2018).