Hello everyone.
In this video tutorial we will see how we can use YOLO V8 for license plate detection and recognition.
So as you can see in this output demo video, we will not only detect the license plate, but we will
also read the text on the license plate as well.
So we will do the detection of the license plate, but also we will read the text of the license plate
as well.
So in this video tutorial we will see how we can detect the license plate using the V8 and also how
we can read the text on the license plate using easy OCR so we can fine tune the YOLO V8 model for license
plate detection.
And then we will use OCR to read the text on the license plate.
So this is a complete end to end tutorial, so don't skip any part of the video and do watch the complete
video.
So let's get started.
So first of all, I will just go to the script file over here and I will explain you the complete code.
So let me move move towards the script file and then we will proceed further and I will explain you
the complete code step by step.
Okay.
So so here in front of you, you can see the CoLab notebook file, Google CoLab notebook file over here.
So before running the script, please make sure that you have selected the runtime as GPU.
Okay.
And click on Save then for this project I will be using this license plate dataset.
The dataset is publicly available in Roboflow, so I will share this dataset link with you as well.
So you just need to create an account on roboflow and it's you can assess this dataset for free.
You just don't need to pay anything to assess this dataset.
So the dataset consist of about 300 images.
We have the train validation and test images.
The author of this dataset has applied augmentation.
So after the data augmentation, the training set is of the images increased to 6 to 18 images.
Now we have after applying augmentation that augmentation, we have 618 images in the training set,
64 images in the validation set and 30 images the test set.
Okay.
So if we go to the health check as well, we can see that.
So it here we don't have the issue of class balance because if we have multi classes, then we check
whether the class is balanced or not.
So here we don't have such issue.
So we will export this dataset from Roboflow into our Google CoLab notebook to export the dataset,
click on, download the dataset, and then select the model model.
So as YOLO, V5 and YOLO V8, both were released by Ultralytics so we can select any of the model format
here for implementing YOLO V8 to train the YOLO V8 or fine tune the V8 model on display dataset, then
export the dataset in any format like you can select the V5 PyTorch format or YOLO V8 format.
So I'll select the Yolov5 PyTorch format and click on Continue.
Then in the next step, I will just copy this, all this code from here and paste into the Google CoLab
notebook.
But if you are doing in any other like I mean, if you are doing in PyCharm or if you are doing in Anaconda
Navigator or if you are doing in or doing in Visual Studio, then you just need to copy this terminal
command.
But if you are doing in the Google CoLab notebook or Jupyter Notebook, just copy this from here and
paste it in the Google CoLab notebook.
So let's move towards the code part.
So first of all, I will download the license plate automatic number, plate detection recognition required
files from the Google Drive into my Google CoLab notebook.
So these are the eight required files.
It contain the plots, files or the V8 Pre-trained model files and the Ultralytics YOLO V8 required
files over here.
So you can see over here we have all these files currently using the zip format.
So I will unzip all this file by just running this cell so it will unzip all the files and you can see
over here we have all the files in the unzipped format.
So it contains the file requirements file.
Okay.
So now I will import this required library from ipython dot display import in which.
So why we need this libraries.
If you want to display any input or output image into the Google colab notebook, then we need this
import image library over here.
Okay.
So I have unzipped this folder.
Now I need to set this folder as my current directory.
So I'm just setting this folder as my current directory.
Now in the next step I will install all the dependencies or you can say the requirements so it will
install all the required libraries that are required to run this project successfully.
If you don't install the dependencies, then when you run the training validation or the prediction
step, then you will face the error that the following library is not installed or the Hydra library
is not installed.
Seaborn library is not installed.
Matplotlib library is not installed.
So it is important to install all the required libraries.
So to install all the required libraries you just need to run the.
A setup.py file, so it will install all the required libraries that are required to run this project.
Some libraries are by default installed in the Google CoLab, but which are not installed.
You can see here these are being installed, so if you miss this step, you can then you will face this
error that the following library is not installed when you run the training script.
So you need to install.
Then you will need to install all the required libraries.
So it's always better to run this step and install all the required libraries before running the script.
Okay, so all the required libraries are being installed, so the dependencies are being installed.
So now as we are doing detection over here, so I will go to Ultralytics, YOLO V8.
So as we are doing detection, so we will go towards this folder and select set this folder as my current
directory because it contains the training validation and prediction script and I am using as I, I'm
doing detection over here.
So I will just go to the detect folder which contains all my required scripts.
Okay.
So I will just check this folder as my current directory.
So now I'm just exporting this dataset from roboflow the CoLab notebook.
So just need to go here, click on copy and just paste this over here.
Okay.
And just if you run this, it will install the dataset or it will import the dataset from Roboflow into
your Google CoLab notebook.
So the dataset will be downloaded in this detect folder.
The dataset will be downloaded in zip format, but it will automatically be unzipped when you just run
this cell.
So might take few seconds.
So now you can see that the data set is being downloaded in the zip format, but it will be automatically
unzipped here as well.
So this might take few seconds.
So let's see.
As it completes, we will then proceed further.
Okay, so dataset is downloaded in the zip format.
So it will now automatically unzip over here and we will see this folder over here in the detect folder.
So you can see the car license plate dataset over here.
It's almost done.
We have the train test validation folder.
Each folder contain the images as well as the label.
And then we have a data dot file which contains the class name.
We have only one class, which is license, and here we have the train validation test, which is path,
and this is the link of the dataset.
So if you open this link, the dataset will open.
Okay.
Okay, so it's about to download the dataset and then we will go towards the training and then we will
train our model on this dataset.
So this might take few more seconds.
So I'm just waiting.
So as it gets completes, then I will proceed further.
Okay.
So it's about to download, so.
So let me set the.
Okay, so just run these cells as well.
Okay.
So now basically we need to in the next step, the dataset is downloaded.
Now we need to train our model on this dataset, which is called a sense two dataset over here.
So this is the Python Train.py script and we are using the V8 model.
So there are five different V8 model.
So the V8 end is the smallest, but it is less accurate and it is fast, although.
But YOLO v8 X is the more most accurate and it is less fast.
So we are using a V8 medium model.
Okay.
And we are setting the default image size because if we change the image size then we might compromise
on accuracy and we are training our V8 model.
Or you can say we are fine tuning the YOLO V8 model on license plate dataset and we are training the
YOLO V8 model on license plate dataset for 120 epochs.
Okay, so let's get started and let's train the V8 model on the license plate dataset.
So it's about to start over here.
First of all, it will download the model from official repository.
It's downloading currently, so it might take some time.
Okay.
Then in the next step the training will start and the training will proceed a step by step.
It will first work for one epoch and then for the second epoch.
The training will take around a around two hours.
So as the training start, I will pause the video and when the training completes, I will be back and
I will explain you the rest of the code.
So the training has started.
It will take time.
So I'm going to pause the video and as the training completes, I will be back.
So guys, the training of the Yolo v eight model on the license plate dataset is completed and here
are the results.
We have trained our model on 120 epochs, which explained a year and we have selected the image size
as 640 as YOLO.
V eight model is officially trained on the image size of 640 by 640, so it's always better to set the
image size as 640 when you fine tune your YOLO V8 model on any custom data set.
So here we are fine tuning our YOLO V8 model on the license plate data set, and we have trained the
YOLO V8 model on the license plate data set.
And here are the results.
We have trained our V8 model on 120 epochs and the mean average precision which we obtained is zero
point with mean average precision.
With IOU, 50 is 0.916, which is 91.6% and mean average precision when varies from 50 to 95 is 0.564,
which is 56.4%.
So the mean average precision is fine and the weights are saved.
Over here, if you go to runs, detect, train and here we can see the weights file.
Here we have the best and last weights.
So the best.is updates file on which our model gives us the best results.
And the last output is the weights file on the last epoch on the 1/20 epoch.
So we will use best dot weights file for further validation of our trained YOLO V8 less model on the
license plate dataset.
So here is the model summary.
The mean average precision of the 92.6% with 50 is obtained and mean average precision.
When IOU varies from 50 to 95 is 57.6%.
So the results are what?
So all our after training, all the confusion matrix validation predictions, training, batch predictions,
precision recall curve, F1 curves are saved in this folder.
The train folder where you can see all these files.
So let me check all these files in that train folder and display all these files in the Google CoLab
notebook.
So just if you click over here so it will display all the files which we have in the train folder.
We have the confusion matrix, we have the F1 curve, we have the precision recall curve, we have the
AACR, we have the results dot CSV file.
So if I show you the result dot csv file, it contains the model evaluation or the model performance
evaluation on each epoch after each epoch.
So you can say after zero first, two, three, nine.
So if you go over here, we have ten pages of this CSV file.
So if I go to the last page, we have the on the last epoch 119 epoch, we have the box loss, the loss
DF loss, precision recall.
So we have all these values mean average precision with 50 to 95 mean average precision with 50.
So we have all these basically in the CSV, we on the model performance after each epoch is saved.
Okay, the results dot csv, the model performance after each epoch is saved.
And then we have the confusion matrix over here.
I have just displayed this confusion matrix over here into the Google CoLab notebook over here.
So Confusion Matrix basically tells us how or how our model handles different classes.
So in this case, we have only one class, which is lessons.
So we can see that 84% of the time our model detected correctly that there is a lesson split, while
16% of the times when there was a lesson played, our model was unable to identify it.
So 84% of times our model was trained YOLO model detected correctly that there is a license plate,
while 16% of the time when the license plate was there, our model was unable to detect it.
So here we have the results dot PNG, which can you can see over here I've just displayed in the Google
CoLab notebook so you can see that the Los loss box logs are decreasing, okay?
They are decreasing and the mean average precision is on the rise.
Mean average precision of 50 mean average precision when the varies from 50 to 95 is on the rise.
So it means that if we train our model on 250 or 300 epochs, the mean average precision will further
improve.
Okay, so here I have the model prediction on the validation batch.
So, uh, so basically these are the model predictions on the validation batch images.
So these images are not used for the training, So it's always better to have a look and see how our
model performs on the validation dataset, batch images.
Okay.
So here are some output examples from the validation batch images.
So you can see that our model is able to detect successfully when there is a number plate and the confidence
is very good as well.
The confidence is most cases is above 0.9 and 0.8.
Okay.
So I have just added this section to just to save your projects into the Google Drive.
And if you want to run this script after some time.
So instead of training the model again, you can directly download the weights from the Google Drive
and test your model.
So I will just place my weights into the Google Drive and I have added the link over here.
So if you want to test this, run the script on my weights, which are this batch stopped.
So you can just download those weights by just running this cell and you can just test the model on
this weights.
Okay.
So I'm just validating the custom model over here.
So on the validation batch images, so on the test batch images, sorry.
So basically these images are not used for the training.
Okay.
So it's always better to have a look and see how our results look like.
So here we are validating our model on the validation batch images.
So here are the validation batch basis.
And on these images we are validating our model.
Okay, so we are not using these images for that training and it's always better to see how our model
performs on these images.
Okay.
So now what we have done is that we have.
Okay.
Let me first.
So.
That.
Give me a minute.
So here I'm just installing PIP install easy OCR.
I'm using PIP install OCR because currently we have detected that there is a license plate like you
can see that we have detected that this is a license plate, but we need to read the read the text inside
the number plate.
Okay.
License plate.
Okay.
Basically we, we want to read the number plate like we want to read WOR516.
Okay.
So to read the text inside the number plate or the text inside the bounding box, which is over here.
So to read the text of the license plate.
Okay, so we use easy OCR to read the text inside the number plate.
There are also other methods as well.
You can use Tesseract OCR as well, but easy OCR gives best results, so we will use easy OCR to read
the text of the number plate or of the license plate.
Okay.
As you can see that this project is license plate detection and recognition.
So we need to detect the license plate and read the text of the license plate as well.
So to read the text I will use first PIP install OCR.
Okay, so it's already installed.
I've run this.
Okay, so here I'm just downloading a demo video in which I want to test my train model on the license
plate dataset and here I have the predict.py script.
So just run this script.
So it might take a few seconds for the execution.
Okay, so let's get started.
So it's about to start and we have the best weights file over here.
We are using the best weights file over here so you can see that the model is able to detect correctly
the license plate, which you can see over here in each frame.
The model is able to detect correctly that there is a license plate and the model has divided the whole
video into 631 frames and currently it's on six 373 frames.
So in this in this when we run this script, predict dot pi script, I'm basically detecting only the
license plate.
I'm not reading the text on the license plate.
Okay.
To read the text on the license plate we will do in the next phase.
So currently I am only detecting the license plate.
Okay.
So here I will just display this demo video over here.
So I have already staged this demo video over here if I run this script again.
So it will take some time.
So let me download this video and see is our model is able to detect the license plate or not.
So just let me play this video and just navigate my screen towards this video.
Okay.
So now you can see the output video.
So you can see that the model is able to check correctly that there is a license plate.
Like, okay, you can see the bounding box and the model is able to read correctly that there is a license
plate.
So in this case, the model is giving us good result.
But our main target is to read the text inside the license plate.
Our main target is to read the text on the license plate.
So let's move towards this and see how we can do this.
So for this I will download the updated predict.py file which I have incorporated the code so I will
download the updated predict.py file from the drive.
So if I click over here.
So this is the updated predict.py file.
Okay.
So if you want to.
Okay, so just let me this line present result over here.
Okay.
You just click on Save now.
Okay, so here you can see that we have created a function by the name of OCR image, and here we are
initializing the OCR and we want to read the text in English, which is and we are using the GPU is
equal to two because we are using GPU and easy OCR reader in means because we are reading the text which
is in English.
Okay.
And here I've created a function OCR image.
Okay, so here we are just choosing the coordinates of the bounding box.
So if let me just open the paint and show you what we are basically doing over here.
Okay, so just let me navigate my screen screen towards the pin.
So if you go over here.
Okay, so if, for example, this is my bounding box, okay, this is the bounding box.
Okay?
So in the bounding box, you these are my X1Y1 coordinates.
These represent the X1Y1 coordinates, and these represent the X2Y2 coordinates.
Okay, So in a bounding box, we have total four points.
One is x, 1Y1, and this is the X2Y2 coordinates.
Okay.
So using these coordinates, we will update our code to read the text.
Okay.
So you can say that x over here.
Let's write me over here.
Okay.
X here represents the x one.
Okay.
So just if I write arrow over here and y over here represents the Y one over here.
So Y represents the Y one and W represents the x two over here.
Okay.
And x represents the y two over here.
Okay, so this is how we have done.
So these are the X1Y1 coordinates and this is the x2 and Y2 coordinates.
Okay, so now what we are doing over here is that if you can see over here.
If I go over here.
Okay, So now from this complete view, reframe Now if you see over here, this is the complete frame.
I'm just selecting the area where we have the bounding box.
Okay?
So if I explain this to you.
Okay, So, okay, just let me go over here.
So now you can see the video.
So from this complete frame, this is the complete frame.
I'm just selecting the area where I have the this bounding box.
Okay.
So I'm just selecting the area where I have this bounding box.
Okay.
Over here.
So in this point, I'm just selecting the area where I have the bounding box where I have the license
plate detected.
Okay.
Then I am converting this area into grayscale.
Okay, so here I'm just converting this a area where I have the bounding box into grayscale and then
I'm using easy OCR.
I am just reading the text in the bounding box using easy OCR.
I am just reading the text in this grayscale image of the bounding box.
Okay, so.
What I am doing is using first have only detected this area where I have this license plate.
Then I have converted this area into grayscale and then I'm reading the text over here.
What I have the text over here.
You can see I'm just reading this text now using OCR, which I have inside this bounding box.
Okay.
So if I go back over here, so I'm just reading the text inside the bounding box which I have over here,
which is over.
You can see that n six, five, six is the text.
Okay, so.
Now, next up, what I'm doing is that I have already run this script so you can see that I have this
printed.
The result over here.
These are the print results.
Okay.
I have already done this script before, so these are the print results.
Okay, so what I'm doing is that you can see that, uh, this is the result.
Okay.
I have printed the result over here.
This is the result after each of the frame.
Okay, So what I'm doing is that so you can see that here I have the list over here.
So if you can see over here, let me just a bit more.
Let me just move it a bit more.
Okay.
So you can see that here.
I have a complete list over here.
Okay.
So in the list, if we see over here, this represents the sub part of the list.
Okay.
So this is one this is the first element of the list, like zero element.
This is the one element and this is the two element.
So this is basically the confidence and this is basically the zero element of the list.
Okay?
This is the zero element.
This is the first element and this is the second element.
Okay.
So I am just going this basically in result we have the list result contains the list.
So I am just going through each, uh, each, each element in the list one by one.
Okay.
So if I'm saying that if the length of the, uh, this result is equal to one.
Okay, but you can see here the length of the this result list is equal to three because this is the
first element of the list.
If we copy this over here, this is the second element of the list, and this is the third element of
the list.
But in that case, if the length of the list is one, then we will read the text in this.
Okay.
This is our basically we want to read this text of the list.
Okay.
But if the length of the result list is greater than one, so currently this length is equal to three
or.
And plus and and length of the result.
One like this is a length of this text is greater than six.
But here you can see that length is equal to four and and the confidence is greater than two.
Then we want to display this text over here, so only we want to display that number plate text we want
don't want this coordinates.
So here we have this coordinates over here, but we don't want this coordinates.
We only want to display that text on the number plate.
We have basically what these coordinates represents.
Okay, Let me tell you, uh, if you can see over here, if I create this bounding box over here, for
example, this is my bounding box, So.
If I just.
Right over here.
Okay, just let me.
Sorry.
I've just will not go on to the paint section, so if you can see over here.
Okay, so if I create a bounding box over here, for example, this is my bounding box over here.
So this represents the one.
Okay, This is the second element of the list.
This is the third element of the list, and this is the fourth element of the list.
So these are the one, two, three, four.
This is one, basically.
Okay.
So if I go back to the code again.
So basically.
If you go if you can see over here.
Okay.
So just let me go back over here so if you can see that this is the first element of the have shown
the bounding box.
This is the second element of the bounding box.
This is the third element of the bounding box.
And this is the fourth element coordinates of the bounding box.
So this is the fourth coordinate.
These are the coordinate points of the bounding box which we have drawn.
Okay, so these are the four corner points of the bounding box.
Okay?
So you can see that over here we have four corner points on the bounding box.
This is the first corner point.
This is the second corner point.
This is the third corner point and this is the fourth corner points.
Okay, So if you can see over here, these are the four corner points on the bounding box.
And this is a text we will just only display this text and this is the confidence.
Okay?
So we will call this function, which we have created over here.
Okay.
So we are saying that we only want to display that in the label section.
We want to display the text inside the bounding box only we want to read the text of the license plate.
So this will be the label.
So if we just display this demo video over here.
So now you can see that our model is able to read the text inside the number plate over here.
Okay, So let me download this video and show you the results as well.
Okay, Just let me navigate my screen over here.
Okay, now, now you can see those results, like we said.
So now you can see that our model is able to read the text inside the number plate.
Okay, So our model is successfully able to read the text inside the number plate.
So it's working completely fine and the results are very good as well.
So that's all from this video tutorial, we are able to detect the license plate.
We have trained our model to detect the license plate.
We have read the text inside the license plate as well using easy OCR.
See you in the next video tutorial.
Till then, bye bye.