Research on License Plate Recognition Method Based on Artificial

Intelligence Deep Learning

Huaiqi Jia

College of Computer Science and Technology, Jilin University, Changchun, 130012, China

Keywords: License Plate Recognition (LPR), Deep Learning, CNN.

Abstract: License Plate Recognition (LPR) is a significant technique in most of the applications related to modern traffic,

which is widely used in various cases in society like traffic control, smart parking, traffic security, and so on.

Due to the growing amount of motor vehicles and the complexity of nowadays traffic conditions, this page

uses an automatic LPR system to solve the remaining problems of conventional LPR systems, including the

lack of ability to treat datasets of extreme weather and heavy traffic. Deep learning of artificial intelligence is

widely used in the network of this page, especially the Convolutional Neural Networks (CNN) network. By

building two distinctive CNN networks, this model achieves end-to-end recognition through the application

of AI deep learning, which eventually gets rid of the need for manual intervention. At the same time, by

leveraging CNN's powerful feature extraction capabilities, the recognition accuracy of license plate characters

is significantly improved. Besides, by using MobileNet and Single Shot MultiBox Detector (SSD) networks,

the CNN network of license positioning is transplantable and light enough to be used in personal computers

without the use of a complex computing system. In contrast to traditional character template matching

recognition, the system on this page can handle license plate images in various complex conditions, including

all sorts of lightning positions, tilt, blur, etc. The network also meets the requirement of real-time application.

The results show that the enhancement can increase the LPR accuracy from 92.57% to 98.43% when blurry

or low-quality images are used.

1 INTRODUCTION

Since the license plate is a bright and intensive dark

spot of the whole image, a recognition that is capable

of capturing and condensing the characteristics of the

pictures is necessary (Habeeb, Noman, Alkahtani, et

al, 2021). However, since the rapid development of

modern society and the enormous number of cars on

the roads, the traffic conditions have become more

and more complex. Besides, the traditional LPR

methods have faced the great challenge of lacking

sufficient dataset and robustness as the low-quality

analog cameras are not capable of ensuring the

quality of images when facing extreme weather and

busy traffic conditions. To solve these problems, this

paper presents a methodology for engineering a

system to enhance the detecting and recognizing

procedures of license plates.

There are still many uncompleted aspects of the

License Plate Recognition (LPR) method. For

https://orcid.org/0009-0004-7244-9170

example, when meeting the images captured by low-

resolution cameras or extensive noises, the Optical

Character Recognition (OCR) system is not able to

recognize the plate accurately (Hamdi, Chan, & Koo,

2021).

Besides, the conventional LPR method is deeply

affected by light, shadow, dark spots, and the

background scenes. By introducing AI deep learning

to the LPR system, the system can figure deeper

characteristics, which improves the recognition

accuracy greatly (Weihong, & Jiaoyang, 2020).

What’s more, in the field of managing and mapping

complex high-dimensional functions like chromatic

images, deep learning stands for the best performance

(Abedin, Nath, Dhar, et al, 2017).

Above all, a new way of deep learning is proposed

for license plate images in this report. Besides, a

reinforcement to LPR is shown in this paper. The

contrast between the solution and the classical

methods is exhibited in this paper.

Jia and H.

Research on License Plate Recognition Method Based on Artiﬁcial Intelligence Deep Learning.

DOI: 10.5220/0013524700004619

In Proceedings of the 2nd International Conference on Data Analysis and Machine Learning (DAML 2024), pages 395-400

ISBN: 978-989-758-754-2

395

2 METHODOLOGY

2.1 Dataset Introduction

This paper uses two distinguishable datasets, one is

used for training the license distinguishing and

positioning network, and the other is used for

character segmentation and recognition. The dataset

used for license positioning contains about 2,000

photos which includes a diverse range of cars’ images

at the front side. In this way, can the research imitate

the complex traffic conditions and the limited quality

of images captured by low-quality analog cameras.

Since the problem of the low capability of

detection of current modules is about to solve, the

Chinese Car Parking Data set (CCPD) has been used

for the training of character recognition, which is the

size of 12GB, including over 30,0000 Chinese plates’

images (Khan, Ilyas, Khan, et al, 2023; Hu, Li, Li, et

al, 2020)).

Besides, this paper also used an open dataset from

GitHub. The dataset includes the following

characters: the Chinese abbreviation of 34 different

provinces, 24 English letters, and 10 mathematic

letters. The resolution of these images is 128*48, and

the image type is jpg format. Sore of original data:

CSDN: GitHub - SunlifeV/CBLPRD-330k: China-

Balanced-License-Plate-Recognition-Dataset-330k,

which is ideal for training and evaluating license plate

recognition models.

2.2 Modle/Advantage

Figure 1: The specially ordered system of the LPR network

and the steps used to translate the original license images to

recognition results (Photo/Picture credit: Original).

As shown in Figure 1, the general LPR system can be

divided into four parts: image processing, license

positioning, character segmentation, and character

recognition. As the detection and recognition of

license plates are two separate modules, the LPR

networks are always divided into two distinctive parts

(Kessentini, Besbes, Ammar, et al, 2019).

In this page, the LPR network uses two distinctive

deep-learning CNNs, which contain different data

sets for training, including a license plate location

model and a character segmentation and recognition

model. Through image processing, the original

images are processed to a resolution and size that is

easy for the model to accept. While the identified

network begins to prepare, the images then enter the

license plate positioning network. In the license plate

positioning network, the license plate area is framed

and then transmitted to the recognition network after

perspective collineation/rectification. After character

segmentation and character recognition, the model

will output the recognition result (In the form of text,

stored in a Word document).

Convolutional Layer: The convolutional layer

employs convolution operations to extract image

features, which can be mathematically expressed as:

𝑌



𝑖,𝑗



∑∑

𝑋





𝑖+𝑚,𝑗+𝑛



∗𝐾𝑚,𝑛 (1)

Where 𝑋 is the entering image, 𝐾 is the

convolution kernel, 𝑌 is the output characteristic

picture.

𝑖 refers to the line number in the output

characteristic picture (𝑌)， 𝑗 refers to the column

number in the output characteristic picture(𝑌). In

other words, (𝑖,𝑗) is a spot in the output characteristic

picture(𝑌).

𝑚 refers to the line number in the convolutional

kernel (𝐾), and 𝑛 refers to the column index in the

convolutional kernel (𝐾). Also, (𝑚,𝑛) represents all

elements of the convolution kernel.

𝑌



𝑖,𝑗



∑∑

𝑋





𝑖+𝑚,𝑗+𝑛



∗𝐾𝑚,𝑛 (2)

This equation describes how to calculate the

figure at the position of (𝑖,𝑗) in the output

characteristic picture (𝑌).This process reaches the

goal of sliding and scanning the convolutional kernel

on the original images, which will condense the

feature value of all images.

ReLU Activation Function: The ReLU activation

function is used to add nonlinearity to the network.

𝑓

(

𝑥

)

=max (0,𝑥) (3)

In this equation, 𝑥 and 𝑓

(

𝑥

)

refer to the input and

output of the ReLU activation function.

Batch Normalization: Batch Normalization is

used to accelerate the training process and improve

the stability of the whole system, which is realized

DAML 2024 - International Conference on Data Analysis and Machine Learning

396

through normalizing every single little amount of

data.

Residual Block: Residual Block is used to solve

the gradient vanishing problem of the CNN network,

which can be realized by introducing skip

connections. Mathematically represents as:

𝑌=𝐹

(

𝑋,



𝑊





)

+𝑋 (4)

Where 𝐹(𝑋,



𝑊





) represents residual function, 𝑋

and 𝑌 are the input and output.

2.3 The Convolutional Kernels Being

Used and the Concrete Function

Methodology

To realize the license plate locating, this module uses

a CNN network, which includes the MobileNet

module. Designed to classify the images, the

MobileNet is a module containing 53 CNN layers

(Jawale, William, Pawar, et al, 2023).

As a slight mobile module, MobileNet provides

the whole model with a better embedded

environment. Based on MobileNet, this program also

compiles its own unique license plate recognition

CNN network, which includes several convolutional

kernels, ReLU activation functions, batch

normalization, and residual blocks. Besides, the

network also uses SSD as a component and auxiliary.

The specific layer design is as follows:

The module begins with two convolutional

kernels, and every convolutional kernel comes with a

batch normalization layer and a ReLU activation

layer.

Following is a max pooling layer.

Then comes two convolutional kernels (every

single one is followed with a batch normalization

layer and a ReLU activation layer) and a residual

block.

Following, there is a max pooling layer.

This strategy (two convolutional kernels, one

residual block, and one max pooling layer) repeats

twice.

Finally, the last convolutional kernel is used to

output the license plate location and bounding box.

2.4 The Process of the Experiment

License plate recognition can be divided into four

steps: License Positioning, Image Processing,

Character Segmentation, and Character Recognition.

2.4.1 License Positioning

The training of the license detection module:

A predefined module structure is being used to

train the module.

Loading the training data, which includes the

images and the related tags (the position of the plate).

Using a personally defined loss function to train.

After training, save the whole module.

The establishment of the module structure:

To create a module based on CNN, the network

used residual blocks and an SPP（Spatial Pyramid

Pooling） module.

The network employs ReLU and uses batch

normalization to accelerate the training process to

avoid overfitting.

The method of identifying the edge of the plate is

used in the input images. (Lin, & Sie, (2019).

Likewise, the output of the module is a feature

map, which is used to outline the position and

bounding box of the license plate.

Using the trained module for license plate

detection:

Loading the trained module.

Preprocessing the input images, including

resizing and normalization.

Using the module to detect the license plate,

output the position and edge of the plate.Saving the

license plates as images.

2.4.2 Image Processing

The detected license plate images are resized to an

uninformed resolution (80x240).

2.4.3 Character Segmentation and

Character Recognition

The design of the CNN module includes several

convolutional layers, pooling layers, batch

normalization layers, and activation layers. The

specific design is as follows:

1)Input layers: Accept the image resolution of

80x240x3.

2)Convolutional layers: Use a 3*3 convolutional

kernel with an astride of 1 and padding set to ‘same’.

3)Batch normalization layer: Normalize each

output of all the convolutional layers.

4)Activation layer: Use the ReLU activation

function.

5)Pooling layer: Use a max pooling of 2*2 and a

stride of 2.

6)Residual blocks: Are used to deepen the internet

depth and improve the performance.

Research on License Plate Recognition Method Based on Artiﬁcial Intelligence Deep Learning

397

7) Output layer: Outputs the probability

distribution of each character in the license plate. The

outcome of the same image could vary in prediction

characters (Alghyaline, 2022). So the probability

distribution of each image is necessary.

2.5 Math Function Design

Convolutional layer:

𝑌=𝑊∗𝑋+𝑏 (5)

Where, 𝑊 is the weights of convolutional kernels

，𝑋 is the input feature map, and 𝑏 is the bias term.

Activation function (ReLU):

The function is the same as function (3).

Polling layer (max pooling):

𝑌

,

=𝑋

,

,∈

,



(6)

Where, 𝑅

,

is a block of the input feature map, 𝑌

,

is the maximum value of the map.

Batch normalization layer

𝜇







∑

𝑥







(7)

𝜎









∑

(𝑥



−𝜇



)







(8)

𝑥



=



















(9)

𝑦



=𝛾𝑥



+𝛽

(10)

Where，𝜇



and 𝜎





are the mean and variance, 𝑥





is the value after batch normalization, 𝑦



is the output

after scaling and shifting.

Loss function (Self-defined):

𝐿=𝐿



+𝐿



+𝐿



(11)

Where, 𝐿



, 𝐿



, and 𝐿



are the loss of

probability, bounding box, and category.

3 RESULTS AND DISCUSSION

In a huge amount of distinguishable environments,

the license plate positioning accuracy can rise to

98.37%, with the final recognition accuracy rising to

97.43%. In the environment of Windos11(x64), the

average recognition time of a single image of all sizes

is 2.70s.

In recent years, AI deep learning has been widely

employed in the LPR method (Gnanaprakash,

Kanthimathi, & Saranya, 2021).

Instead of using the traditional YOLO module, the

research of this page uses the combination of

MobileNets and SSD for character recognition, along

with a manually crafted CNN recognition module.

MobileNet, a lightweight neural network module

proposed by Google in 2017 specifically designed for

mobile or embedded devices, can significantly lower

the number of modules and computations when

keeping a high accuracy to get used to the limited

environment resources. The core technology of

MobileNet includes Depthwise Separable

Convolution, which decomposes the standard

convolution into Depthwise Convolution (DW) and

Pointwise Convolution (PW). Through these

methods, can the network significantly reduce the

complexity.

Compared with the widely-used LPR module

constructed by You Only Look Once (YOLO), the

method of this page has the advantages of being both

lightweight and high-performance. Its smaller size

module can save enormous storage space and

computing costs when deployed or removed into

embedded devices. Besides, it can significantly lower

the computing costs. What’s more, although the size

of the network is small, its performance is not inferior

to many large CNN networks. MobileNets is the

upgraded version designed by Google company,

which created the GoogleNet before the MobileNets.

However, the huge amount of the training data

used in the training process of GoogLeNet is a crucial

problem (Pham, 2023). Besides, since the partnership

of FPN and TOLOv3, the quality of the devices is

required while the speed is slow (Hu, Li, Li, & Wang,

2020). By contrast, MobileNets is much smaller in

size when meeting nearly the same accuracy as

Inception-v3 (PuarungrojM & Boonsirisumpun,

2018). As a result, the network created by this page

can be successfully run on personal computers

without the help of high-quality graphics cards.

However, as the complexity being sacrificed, and the

training sample are less than those of the general

YOLO module, the recognition efficiency of this

page is slightly lower than the LPR module using the

YOLO method.

DAML 2024 - International Conference on Data Analysis and Machine Learning

398

4 CONCLUSION

Through applying the CNN network of AI deep

learning to the LPR network, the module introduced

in this page achieves the end-to-end license plate

recognition system, avoids unnecessary manual

intervention, and realizes the complete license plate

recognition process. With the help of the CNN, by

contrast to the traditional character-matching license

plate recognition method, this module has greatly

improved the detection rate and robustness in

processing license plate images in several complex

conditions, like traffic jams, extreme weather, or low-

frequency cameras, etc. At the same time, the CNN

network also solves the problem of the real-time

requirement of LPR.

The License plate recognition method can be

divided into four separate parts: License plate

recognition, Image preprocessing, Character

recognition, and subsequent processing. In the

network of this page, both the license positioning and

the character recognition use the CNN network,

which also has been carried out with appropriate

training. The location module is based on the object

detection framework (the combination of MobileNet

and SSD), which outputs the bounding box and

images of the license. After the operation such as

cutting, correction, and scaling in the image

preprocessing method, the images will be then

transmitted to the character recognition network. In

the character recognition network, the module uses a

CNN network to recognize the processed images, and

finally output all the probabilities of each character.

In the subsequent processing process, the module

avoids repeated detection and determines the final

result based on the probability distribution.

In contrast with the traditional character template

matching method, since the self-learning character of

deep learning and sufficient dataset, the

generalization ability of this module has been

enhanced significantly. Besides, the module contains

an enormous image preprocessing function and image

optimization process, which can greatly expand the

approaches of resources of the character and

template, avoid the manual design, and reduce the

development workload.

Apart from the other formal AI deep learning LPR

method, the computing time of the network on this

page is highly reduced. At the same time, using the

CPU with weaker capabilities and higher resolution

images, the module reaches the goal of being

lightweight and portable, which lowers the costs

compared to the other method based on high-quality

CPU/GPU. However, due to the sacrifice of

complexity and the smaller amount of training

dataset, the accuracy of this module can not exceed

the accuracy of the LPR module using YOLO.

Nevertheless, this module also has some

disadvantages and shortcomings. The complexity of

the module remains high, which costs many

computing resources and time to train the module.

What’s more, compared with the traditional character

template matching method, since CNN training needs

a great amount of data, the recognition accuracy

closely relates to the size and quality of the training

dataset. The interpretability of the decision-making

process within deep learning models is relatively

poorer than the traditional character template

matching method.

This page designs an accurate and correct LPR

network based on AI deep learning, which shows

good performance and robustness in practical

applications. In the future, the goal of this research is

to improve the module structure and design of the

function further. At the same time, the exploration of

combining the LPR technology with other traffic

methods under many practical circumstances will be

put on the agenda, like intelligent parking, and

electronic toll collection (ETC).

REFERENCES

Abedin, M.Z., Nath, A.C., Dhar, P., Deb, K. and Hossain,

M.S., 2017. License plate recognition system based on

contour properties and deep learning model. In: 2017

IEEE Region 10 Humanitarian Technology Conference

(R10-HTC), pp. 590-593. IEEE.

Alghyaline, S., 2022. Real-time Jordanian license plate

recognition using deep learning. Journal of King Saud

University-Computer and Information Sciences, 34(6),

pp. 2601-2609.

Habeeb, D., Noman, F., Alkahtani, A.A., Alsariera, Y.A.,

Alkawsi, G., Fazea, Y. and Al-Jubari, A.M., 2021.

Deep-learning-based approach for Iraqi and Malaysian

vehicle license plate recognition. Computational

Intelligence and Neuroscience, 2021(1), 3971834.

Hamdi, A., Chan, Y.K. and Koo, V.C., 2021. A new image

enhancement and super resolution technique for license

plate recognition. Heliyon, 7(11).

Hu, X., Li, H., Li, X. and Wang, C., 2020. MobileNet-SSD

MicroScope using adaptive error correction algorithm:

real-time detection of license plates on mobile devices.

IET Intelligent Transport Systems, 14(2), pp. 110-118.

Gnanaprakash, V., Kanthimathi, N. and Saranya, N., 2021.

Automatic number plate recognition using deep

learning. In: IOP Conference Series: Materials Science

and Engineering, 1084(1), p. 012027. IOP Publishing.

Jawale, M.A., William, P., Pawar, A.B. and Marriwala, N.,

2023. Implementation of number plate detection system

Research on License Plate Recognition Method Based on Artiﬁcial Intelligence Deep Learning

399

for vehicle registration using IoT and recognition using

CNN. Measurement: Sensors, 27, 100761.

Kessentini, Y., Besbes, M.D., Ammar, S. and Chabbouh,

A., 2019. A two-stage deep neural network for multi-

norm license plate detection and recognition. Expert

Systems with Applications, 136, pp. 159-170.

Khan, M.M., Ilyas, M.U., Khan, I.R., Alshomrani, S.M. and

Rahardja, S., 2023. License plate recognition methods

employing neural networks. IEEE Access, 11, pp.

73613-73646.

Lin, C.H. and Sie, Y.S., 2019. Two-stage license plate

recognition system using deep learning. In: 2019 8th

International Conference on Innovation,

Communication and Engineering (ICICE), pp. 132-

135. IEEE.

Pham, T.A., 2023. Effective deep neural networks for

license plate detection and recognition. The Visual

Computer, 39(3), pp. 927-941.

Puarungroj, W. and Boonsirisumpun, N., 2018. Thai license

plate recognition based on deep learning. Procedia

Computer Science, 135, pp. 214-221.

Weihong, W. and Jiaoyang, T., 2020. Research on license

plate recognition algorithms based on deep learning in

complex environment. IEEE Access, 8, pp. 91661-

91675.

DAML 2024 - International Conference on Data Analysis and Machine Learning

400