LPTR-AFLNet: Lightweight CLPR Network

• The paper introduces a unified neural architecture combining PTR and AFLNet that significantly improves Chinese License Plate Recognition (CLPR) accuracy in challenging, unconstrained environments.
• It employs a perspective transformation rectification module to correct distortions, enabling reliable recognition of both single-line and double-line license plates using end-to-end optimization.
• The lightweight design, enhanced LP-CA attention, and Focal CTC loss collectively boost speed and accuracy, as validated by comprehensive experiments on CCPD and LSV datasets.

LPTR-AFLNet is a lightweight, unified neural architecture for Chinese License Plate Recognition (CLPR) targeting unconstrained and complex environments. The main innovation is its integrated design: a perspective transformation rectification (PTR) module coupled with an optimized license plate recognition network, AFLNet. This composition addresses perspective distortions and efficiently recognizes both single-line and double-line license plates using an end-to-end approach amenable to deployment on resource-constrained edge devices (Xu et al., 22 Jul 2025).

1. Architecture Overview

LPTR-AFLNet comprises two tightly coupled sub-networks: the PTR Module and the AFLNet recognizer. The system accepts a roughly cropped license plate image (single-line or double-line) with an associated 4-vertex bounding box. The PTR module regresses geometric offsets to estimate the true plate corners and computes a 3×3 homography matrix to rectify the input plate to a canonical, fronto-parallel viewpoint. For double-line plates, independent rectifications are performed on the top and bottom regions, with results concatenated horizontally into a synthetic single-line format for recognition.

The rectified output proceeds to AFLNet, an improved variant of LPRNet. AFLNet processes images through a lightweight CNN backbone and extracts multi-scale features. A combination of 1×1 convolutional classifiers and average pooling produces character class scores. A novel attention mechanism (LP-CA) is applied prior to the classifier to reduce inter-class confusion. Recognition proceeds via a modified CTC (Focal CTC) loss, which addresses severe class imbalance inherent to Chinese license plate datasets.

The data flow and dependencies are:

• Plate detection → PTR (offset estimation, homography solution, warping) → AFLNet (feature extraction, LP-CA, classification, CTC decoding) → predicted sequence. Recognition loss supervision propagates through the entire pipeline, directly coupling rectification and recognition performance.

2. Perspective Transformation Rectification Module

The PTR module implements a geometric transformation to correct arbitrary in-plane and perspective distortions. Input corners $(u_i, v_i)$ are adjusted via regressed offsets $(\Delta X_i, \Delta Y_i)$ to produce the actual plate vertices $(X_i, Y_i)$. A linear $8 \times 8$ system is solved to obtain the homography matrix $H$, enabling pixel sampling from the input image to a normalized, fronto-parallel coordinate system. Mapping is carried out by applying $H^{-1}$ to target grid coordinates, with bilinear interpolation used to sample the rectified image.

For single-line plates, 8 offsets (4 corners) are regressed; for double-line plates, 12 offsets define six points (shared vertices between top and bottom regions). Each region is rectified independently prior to concatenation.

Supervision for the PTR module is indirect: no explicit geometric loss is used. Instead, the recognition loss (Focal CTC) is backpropagated through the grid sampler, influencing the homography estimation and offset regression sub-network. This weak supervision ensures that rectification is optimized for recognition accuracy.

3. AFLNet Recognition Backbone and Enhancements

AFLNet refines the LPRNet framework by introducing multi-level feature fusion, per-channel attention, and an improved loss function.

• Multi-level Feature Extraction: Four feature maps from a shallow CNN ("low," "mid," "sub-high," "high") are energy-normalized and concatenated, yielding a rich $C \times H \times W$ tensor (521 channels, 4 rows, ~18 time steps).
• Character Classifiers: 1×1 convolutional filters for 73 classes (72 characters + blank) output a $73 \times 4 \times 18$ confidence volume, which is reduced via average pooling over $H$ to logits for CTC decoding.
• Lightweight Per-Channel Attention (LP-CA): To mitigate confusion among visually similar classes, the LP-CA module applies average pooling along the height dimension, followed by per-channel convolution and sigmoid gating. The resulting attention weights are applied to each channel, modulating the fused feature representation before classification:

$$Y = X \odot \text{Copy}_y\left( \sigma\left(\text{Conv}_{1\times3}(\text{AvgPool}_y(X))\right) \right)$$

where $(\Delta X_i, \Delta Y_i)$0 denotes the sigmoid function.

• Focal CTC Loss: Given the extreme class imbalance (95.7% "WAN" plates vs 14.3% Chinese-character frequency), a "Focal CTC" loss is used:

$(\Delta X_i, \Delta Y_i)$1

with $(\Delta X_i, \Delta Y_i)$2, $(\Delta X_i, \Delta Y_i)$3, and $(\Delta X_i, \Delta Y_i)$4 denoting the greedy CTC sequence probability. This loss prioritizes difficult, infrequent cases over easy, majority-class instances.

4. Training Regimen

The end-to-end training protocol is staged:

1. AFLNet Pretraining: Recognition sub-network is trained with ground-truth rectified plates using standard CTC loss, initializing the recognizer.
2. PTR Pretraining (Weakly Supervised): AFLNet is frozen; PTR is trained end-to-end with the Focal CTC loss, with gradients flowing from recognition through to the offset regressor.
3. Joint Fine-tuning: Both PTR and AFLNet are unfrozen and jointly optimized with Focal CTC loss.

The final optimization objective is:

$(\Delta X_i, \Delta Y_i)$5

This pipeline ensures that rectification and recognition are both optimized according to recognition task performance.

5. Computational and Implementation Properties

The overall model size is 2.7 MB, comprising roughly 700K parameters (467K for the LPRNet backbone, ~200K for PTR and LP-CA). Computational demands per image are under 1 GFLOP. Benchmarks demonstrate:

• TITAN X: 2459 FPS (0.41 ms/image)
• Xeon E5-2620: 152 FPS
• i7-7700K: 108 FPS

These results significantly exceed the real-time threshold (30 FPS) for both GPU and CPU deployment, facilitating edge device and real-time video processing scenarios.

6. Experimental Findings

Evaluation on the CCPD dataset (covering diverse conditions: base, DB, FN, rotate, tilt, weather, challenge, and double-line) demonstrates that LPTR-AFLNet outperforms LPRNet across all conditions—including severe perspective tilt and double-line plates. On the LSV video-based plate dataset, LPTR-AFLNet achieves 74.10% (6-character plates) and 71.50% (7-character plates) average accuracy, with a runtime (0.41 ms/image) four times faster than MFLPR-Net.

Ablation studies show:

• PTR yields the largest single improvement over LPRNet.
• LP-CA and Focal CTC provide complementary benefits in character-level precision and rare-case handling.
• The full model (PTR + LP-CA + Focal CTC) consistently outperforms all baselines. LPTR-AFLNet reduces mis-insertions, mis-deletions, and character confusion errors by over 70% relative to plain LPRNet.

Configuration	Acc_7c (CCPD)	Acc_6c (CCPD)	Char‐Prec (CCPD)
LPRNet	96.58	97.35	99.73
+ PTR	99.00	99.23	99.88
+ LP-CA	99.39	99.68	99.92
+ Focal CTC	99.35	99.44	99.92
+ LP-CA + Focal CTC	99.46	99.56	99.92

7. Significance and Extensions

LPTR-AFLNet demonstrates that joint rectification and recognition, optimized end-to-end via recognition-driven weak supervision, yields improvements over traditional decoupled pipelines in CLPR. The lightweight design, efficient computation, and superiority on both single-line and double-line plates establish its practical applicability for edge devices and real-time video platforms. All architectural and algorithmic decisions—particularly the indirect supervision of PTR and the integration of focused loss semantics—are grounded in empirical gains on challenging datasets (Xu et al., 22 Jul 2025).