Vehicle and License Plate Recognition with Novel Dataset for Toll Collection (2202.05631v2)

Abstract: We propose an automatic framework for toll collection, consisting of three steps: vehicle type recognition, license plate localization, and reading. However, each of the three steps becomes non-trivial due to image variations caused by several factors. The traditional vehicle decorations on the front cause variations among vehicles of the same type. These decorations make license plate localization and recognition difficult due to severe background clutter and partial occlusions. Likewise, on most vehicles, specifically trucks, the position of the license plate is not consistent. Lastly, for license plate reading, the variations are induced by non-uniform font styles, sizes, and partially occluded letters and numbers. Our proposed framework takes advantage of both data availability and performance evaluation of the backbone deep learning architectures. We gather a novel dataset, \emph{Diverse Vehicle and License Plates Dataset (DVLPD)}, consisting of 10k images belonging to six vehicle types. Each image is then manually annotated for vehicle type, license plate, and its characters and digits. For each of the three tasks, we evaluate You Only Look Once (YOLO)v2, YOLOv3, YOLOv4, and FasterRCNN. For real-time implementation on a Raspberry Pi, we evaluate the lighter versions of YOLO named Tiny YOLOv3 and Tiny YOLOv4. The best Mean Average Precision ([email protected]) of 98.8% for vehicle type recognition, 98.5% for license plate detection, and 98.3% for license plate reading is achieved by YOLOv4, while its lighter version, i.e., Tiny YOLOv4 obtained a mAP of 97.1%, 97.4%, and 93.7% on vehicle type recognition, license plate detection, and license plate reading, respectively. The dataset and the training codes are available at https://github.com/usama-x930/VT-LPR

The paper "Vehicle and License Plate Recognition with Novel Dataset for Toll Collection" presents an image-based framework for automated toll collection that addresses a variety of challenges specific to Pakistani vehicles. The proposed framework is comprised of three main components: vehicle type recognition, license plate localization, and license plate character recognition. These components operate sequentially to identify the vehicle type and accurately read license plates from images. The framework leverages convolutional neural networks (CNNs) and evaluates multiple object detection architectures, including YOLOv2, YOLOv3, YOLOv4, Tiny YOLOv3, Tiny YOLOv4, and Faster RCNN.

Key contributions and findings of the paper include:

1. Dataset Creation: The authors introduce the Diverse Vehicle and License Plates Dataset (DVLPD), which contains 10,000 images annotated with six vehicle types. This dataset reflects real-world conditions and variations seen in Pakistani vehicles, such as decorations, non-uniform license plate positions, and variations in font styles.
2. Evaluation of Detection Architectures: The paper evaluates six object detection algorithms for their performance in vehicle type recognition, license plate detection, and character recognition tasks. YOLOv4 achieves the best overall performance with mean Average Precision ([email protected]) scores of 98.8% for vehicle type recognition, 98.5% for license plate detection, and 98.3% for license plate character recognition.
3. Challenges Addressed: The authors tackle significant challenges due to severe clutter from vehicle decorations, non-standard formats of license plates, and environmental conditions that obscure license plates. Through targeted data preprocessing and model design, they improve recognition accuracy under these conditions.
4. Real-Time Deployment: For potential implementation at toll collection facilities, the framework is designed to run on a Raspberry Pi, ensuring feasibility in low-resource environments. Tiny versions of YOLO are highlighted for their efficient performance, offering a balance between accuracy and computational load.
5. Experimental Results: On a novel test set with real-world images captured at toll plazas, the framework shows high end-to-end recognition rates, particularly excelling in recognizing less-decorated vehicles like cars and vans, while still achieving respectable accuracy for trucks, a vehicle type fraught with challenges.

The proposed solution innovatively addresses the computational and environmental challenges faced in automated toll collection, making it a practical option for regions with diverse vehicle appearances and limited digital infrastructure. Future research directions include enhancing the framework’s performance for more decorated vehicles like trucks and exploring alternative detection models to further boost accuracy, particularly under challenging conditions. The paper emphasizes improving the economic feasibility of automated toll systems without reliance on RFID tags.