Reading Car License Plates Using Deep Convolutional Neural Networks and LSTMs (1601.05610v1)

Abstract: In this work, we tackle the problem of car license plate detection and recognition in natural scene images. Inspired by the success of deep neural networks (DNNs) in various vision applications, here we leverage DNNs to learn high-level features in a cascade framework, which lead to improved performance on both detection and recognition. Firstly, we train a $37$-class convolutional neural network (CNN) to detect all characters in an image, which results in a high recall, compared with conventional approaches such as training a binary text/non-text classifier. False positives are then eliminated by the second plate/non-plate CNN classifier. Bounding box refinement is then carried out based on the edge information of the license plates, in order to improve the intersection-over-union (IoU) ratio. The proposed cascade framework extracts license plates effectively with both high recall and precision. Last, we propose to recognize the license characters as a {sequence labelling} problem. A recurrent neural network (RNN) with long short-term memory (LSTM) is trained to recognize the sequential features extracted from the whole license plate via CNNs. The main advantage of this approach is that it is segmentation free. By exploring context information and avoiding errors caused by segmentation, the RNN method performs better than a baseline method of combining segmentation and deep CNN classification; and achieves state-of-the-art recognition accuracy.

Analysis of Car License Plate Detection and Recognition Methods Using DNNs and RNNs

This paper addresses car license plate detection and recognition by utilizing deep neural networks (DNNs), specifically deep convolutional neural networks (CNNs) and long short-term memory (LSTM) recurrent neural networks (RNNs). The goal is to enhance performance in natural scene images, which present challenges such as variability in character patterns and complex backgrounds.

Methodology

The proposed system comprises two main stages: license plate detection and character recognition. In the detection phase, the authors employ a cascade framework integrating multiple CNN classifiers. The first CNN is a 37-class classifier designed to detect characters through a sliding-window approach, producing a high recall rate. It distinguishes characters by categories, which improves feature specificity compared to binary classifiers. Subsequent model stages focus on refining bounding boxes and reducing false positives.

For character recognition, the authors propose two techniques: one involving character segmentation and another treating recognition as a sequence labelling problem. The first method uses another deeper CNN for accurate character classification, while the second employs an RNN with LSTM, treating license plates as unsegmented sequences. This sequence labelling approach notably does not require pre-segmented characters, facilitating improved recognition by incorporating contextual sequence information.

Experimental Results

The paper reports favorable numerical outcomes using the two proposed methods on multiple data sets. The CNN-based detection system demonstrates competitive precision and recall on challenging data sets such as Caltech Cars and AOLP, showcasing its robustness against complex backgrounds. The first recognition approach leverages the enhanced representational power of deep CNNs, yielding strong character and license plate recognition accuracy rates.

The sequence labelling approach utilizing LSTMs achieves significant performance improvements due to capturing temporal dependencies within sequence data. Equipped with connectionist temporal classification (CTC), the sequence-based system shows improved accuracy compared to traditional segmentation methods. The results emphasize the benefits of avoiding explicit character segmentation, as well the reliability added by LSTM models in handling sequential features.

Implications and Future Work

The implications of this work extend to the broader field of intelligent transportation systems and automatic recognition tasks involving poor-quality images and complex visual contexts. The constellation of CNNs for detection and LSTMs for recognition reveals a promising direction for enhancing LPDR performance while reducing reliance on controlled imaging environments.

Future research directions should explore efficiency improvements in detection processes, potentially investigating proposal-based approaches or optimizing computational workflows for real-time applications. The framework also invites examination of domain adaptation techniques to generalize across varying license plate styles internationally, increasing the robustness of this approach across different environments.

The thorough evaluation of proposed configurations draws attention to fine-tuning opportunities in CNN parameters and architectures for task-specific improvements, as well as investigating integration strategies for various feature extraction methods like LBP. With these considerations, the presented methodologies have potential applicability to other related recognition problems across transportation, surveillance, and document processing domains.