Hi guys.
In this video tutorial we will see how we can implement pen and book detection with YOLO V8.
This is a complete end to end video tutorial, so I will start with gathering or collecting the dataset
to collect or gather the dataset for this project I will go to open image dataset.
Just type open the dataset over here and click on the first link over here.
Okay, so we will click on Explore.
And over here you will find type as you want to detect not segmentation.
We are not currently performing segmentation.
We are only focusing on detection.
So I will click on detection.
Okay.
So here will take on, first of all, Ben.
And this over here.
So you can see that I have a many number of images for with pen.
You can see there.
So there is a data set for the pen over here.
But you can see that like tablet is also shown in the image plus person is also shown in the image.
So I only want pen to be detected, not other things like chocolates over here, ink over here.
So I will just go to random.
I will just go to sorry, just click on pen over here.
And go to option and just uncheck this box like display box for all categories from all categories.
Only need that bounding boxes one pen.
So just unchecking it and let's see.
So here I have the bounding box with the pen.
Here I have the bounding box with the pen.
So you can see that all the other bounding boxes which are previously with cups or other objects are
not there.
So they are only, only the various pen there.
We have a bounding box so we have only bounding boxes with the pen.
You can see over here as well.
Okay.
The other thing we need is book.
So I will type book over here and just click on Enter over here.
So you can see that we have book over here.
Okay, that's fine.
We have currently a book.
This book images as well.
And we have also book images as well.
Okay, that's cool.
So we have the data set available for Penn and book on image data set or images data set.
V7 So if we want to get this data set or download this data set into our CoLab notebook, what we will
do is we will go to just open a new tab over here.
That's right.
Open image dataset.
Before.
Will get.
GitHub.
Okay.
So just open this first link over here.
So using this toolkit available in this GitHub repo, I will download the dataset from open images dataset
into my Google CoLab notebook.
Okay.
So first we need to just clone this GitHub repo into my Google CoLab notebook.
So here I have already created a Google CoLab notebook with the required script, sir.
So just run this cell.
First we are importing the library library helpers to navigate between different folders or.
Basically, we can also navigate gate between different folders files as well.
Thus we are importing glob ipython dot display.
Basically we need ipython library if you want to display or any input or output image into our google
CoLab notebook and using display dot clear dash output.
I am placing all my previous outputs.
So first of all, we need to make sure that we have selected the runtime as GPU.
So just go over here, click on runtime and select Change runtime and Select Hardware accelerator as
GPU and click on Save.
So just checking if we are using GPU memory.
So it will take a few seconds.
Okay.
So currently we have GPU memory usage.
Okay.
That's fine.
So.
First of all, I will just.
Clone this github repo over here.
Just right.
Just ctrl V over here.
We so we will clone the id v4 duplicate to download the images from the open image dataset.
Okay, now just run this cell.
So it might take a few more seconds.
So it has run now.
Let's see.
What is our present working directory is.
All we need to set the current directory as V4 toolkit so we can run this script.
Okay, so just setting the current directory as V4 toolkit.
Now we need to install all the requirements.
So here we have the requirements file.
So just click PIP, install minus requirements, minus s minus R means we are reading all the requirements.
So it might take a few seconds to install all the required libraries to required to run this script.
Okay.
So by RSA URL, IB, all these libraries are now installed.
So here I will be downloading the pen and book images dataset.
So first of all, we will run the Help command to see what different functionalities are available over
here.
Okay.
Well, you can see here we have the option of a cash limit defines how many image we want to download
for each of the class.
Okay.
And we have multi classes, 0 or 1.
So by default, it is set to zero.
Okay.
So if it is set to zero, it will download all the images into different classes.
But we basically want to have all the images in the training.
First of all, we need all the pen and book images in the training set.
Then we need all the pen and book images in the validation set.
Then we need all the pen and book images in the test set.
So we will set like multi classes as one.
We don't need images in different directories like pen images in the different directory book images
in the different directory.
We need all the pen and book images in a single directory, so we'll stick one.
Thus for that, if we see the thing which can be important for us is tied to CSP.
So if stack type as train, so it will only download the training images.
If I select it as validation, it will stack only download the validation images.
If I select it as test, it will only download the test images.
But if I stack type as csvs all it will download training, validation and test images.
So I will select Typekit csv all to download the training, validation and test images.
Okay, so here I am just running this python main.py downloader.
So I'm basically downloading the images classes for which class I am downloading the images I want to
download for the pen class and for the book class and type CSV.
We have selected all to download the train validation and test classes.
Plus we have selected multi classes as one.
So in the training directory we will have the images for the pen and books.
Okay.
If I select multi classes as zero, then we will have different folders.
Like we will have a separate folder for the pen pen class and we will have a separate folder for the
book class while selecting multi classes as one makes a.
Downloads pen and book images into a single directory while selecting multi classes as zero.
Downloads the book and pen and book images into different directories.
Okay, so here I'm setting the limit as 120.
So 120 images for the pen class will be downloaded in the training set.
120 images for the book class will be downloaded in that training set.
Similarly, 120 images for the test class will put 120 images for the pen class will be there in the
test set and 120 images for the book class will be there in the test set and 120 images for the PEN
class will be there in the validation and 120 images for the book class will be there in the validation.
So here we have to find the limit.
Now just run this cell as well.
It will download all the images.
Yeah, just right over here and click on Enter.
So it might take two seconds for the exhibition.
And just right over here again.
So this process might take some time.
7%.
10%.
13%.
15%.
So it is a bit slow process and it will take some time.
So now you can see that the data set for both the classes, pen and book has been downloaded.
You can see here it's done.
So where we can see the images, you just need to click over here and just move to dataset.
And here you can see that train.
And here you have all the images.
Okay, just give me.
So here you have all the images over here and in the label section, we have the label for each of the.
Image.
Okay, so here we have the test for the test plus images and labels as well.
Plus, here we have for the validation class images and labels as well.
You can see over here, here are the labels and here are all the images for the validation class as
well.
So now basically we need to convert this data into a zip format and download into our local computer
so that we can upload it on roboflow and convert it into the YOLO V8 PyTorch format.
So just copy this from here and just paste this.
Just remove this and just paste this over here and here.
I have written the name of my zip file and the zip file will be placed over here and you can add any
name over here as per your choice.
Okay, so this dataset will be converted into a file zip file, which name is dataset.zip.
So just run this cell.
So now we are just converting our dataset in train test and validation dataset into a zip format.
So now you can see the zip file over here, dataset.zip.
So just click on, download and download this file into your local computer.
So.
It might take a few minutes or some time to download, but I have already downloaded this dataset into
my local computer before making this tutorial.
So here you can see the dataset file and I have unzipped it and here we have the dataset.
Golden after unzipping it.
And here we have the train test and validation folders.
Okay.
So you can see over here and here is the training folder and here are the images and labels and here
is the validation folder and here are the images and labels.
So let me go to the roboflow and create an account over there and load all this dataset over there.
Okay, so just click on new tab.
Let's right here Roboflow and click Enter.
So I will go on this first link.
Okay, so if you haven't created your account, you just need you need to sign up.
Or if you have an account you just click on, need to click on sign in.
So I have, uh, so let me I have already have an account, but let me show you how you can create an
account and select the plan as public so you can export the data set from Roboflow.
So you just need to click on sign up.
Okay.
And log in or create an account.
I will create an account through my email.
Okay.
And just wait for a few minutes.
I will just click over here.
Okay.
So it's going to log in.
Okay.
So it might take a few minutes to create the account, so please bear with me.
To set up the settings and other things.
So it might take a few minutes.
Okay.
So I accept that terms and service and privacy policy continue.
Okay, then.
I just need it for academia, purpose and workspace name as.
Moin.
Okay, so this is my workspace name.
You can write any name, but just am writing my name over here.
So currently I don't want to invite other other collaborators for this project because I'm working alone
on this project.
Okay, so.
I'm using for the community purpose.
On my dataset will be available publicly, so no need a guided tour.
So just click one close and just click on create new project over here.
Okay.
So I don't need any hints.
Okay.
So what are you detecting?
Basically, you are taking pen and book.
And look just right over here.
Project name.
Pen, book detection.
This is our project name and create public project.
Click on Create Public Project.
So now a project will be created over here.
So just wait for a few seconds more until we are able to create a new project over here.
And so after creating the project, I will upload all the images which are in the training validation
and test folder into the roboflow and just we see if we need to do augmentation or not.
And I will also verify my dataset, health and other parameters to see my dataset is balanced or unbalanced.
So just creating the new creating a public project.
Please note that this dataset will be available publicly as well.
So I don't know why it's taking so much time.
So just click on Cancel and try to create a new project.
Okay.
Uh, I just writing that again cause the previous was taking so much time.
Okay, so the project is being created.
Uh, we already did that previously.
Just try to create a project.
But it was taking so much time, so just close it.
But the project has been created, which you can see here, pen book detection.
Plus one thing I want to tell you, if you go to settings and Workspaces, Muhammad.
This is my workspace name.
So please show make sure that this this workspace is on the public plan.
Okay.
So if it is on some other plan, then you will not be able to export the data set from Roboflow into
your CoLab notebook or into your terminal on a terminal.
Okay.
So just click over here.
And click on select Select Files and just go to the dataset content.
Why Db4 toolkit.
4D dataset.
First, we'll go to training, pan and book and control a upload all these files over here so it might
take few minutes or few seconds, so please bear with me.
So.
In this way.
In the similar way, you can also implement this project or any kind of object on your side as well.
Now you can see that they are not annotated because we have not uploaded the labels file yet.
So just uploading all the labels file here as well.
Okay.
So you can see that now the annotations are being done at 33, images are interpreted.
So as this process completes, all the images will be annotated.
So it might take a few seconds more.
So after this I will upload the test image test dataset images and after test dataset I will upload
the validation dataset images.
It might take some minutes or a few seconds.
Now I will go to select files and I will go to that dataset and upload the test dataset images over
here.
Control a upload.
So now all the files are being uploaded.
You can see that.
It might take some time to 1730 not integrated, so it will upload slowly, gradually, so it is a bit
time taking process.
So it's not that you just click on and it's uploaded, but it takes some time to upload over here.
So.
Please be assured that me do that, upload it now will upload the labels because you can see that they
are not annotated.
So now I'm going to see that all the images will be annotated like You can see that.
This will be getting less as the files of labels upload.
Okay, so you can see that in the first step we download the 240 images.
While in the second time we have only have 213 images because there might be some missing labels or
classes.
So there is some.
We get some less images than our target, which we have written over there.
Okay, so all the images are being uploaded.
So you can see that we have 453 images till now.
The either images are also being uploaded over here.
So.
You can see that here.
Currently we have zero images because after I upload all the images over here and then I will click
on Save and Continue.
Then these images will appear in the dataset over here and then I will check the model health and other
versions, and then I will generate the dataset after seeing that if there is any need for the pre-processing
or not, or that is there is a need for data augmentation or not.
So around 107 images are left which are not annotated correctly, but they will be annotated as we move
towards the completion of this process.
It might take some few seconds more.
So please bear with me.
Like the 378 images are integrated and 75 images are not integrated.
While.
39 images are now encrypted and 64 images are not encrypted.
Okay.
So these numbers are increasing.
You can see that.
The aim of explaining this whole process to you.
That is that so that you can implement this same project on your side as well.
So this is the main aim.
Why I'm trying to implement this project from very start.
So if you try to implement project on your side, you can do this as well.
Okay, so now there are nine widgets which are not integrated and two images which are not integrated.
So I will go to select files and then I will go to.
Dataset and click on validation and then group.
Now I'll upload all the images which are in the validation folder.
Uh, so it might take some time.
You can see that.
But they will be uploaded over here.
So after uploading these images, you can see that currently they are not annotated because I have not
uploaded the labels file in the validation folder, so I will upload the labels file in the validation
folder over here.
So now this number will start to decrease a bit.
Okay.
So it's 125 now, but it will as we move on, this number will decrease.
So it's 117 now it's 109 now.
So this number will get gradually decreased.
So please wait and watch.
Now it's 101 now, so please see and it will start to decrease like 93 now and it's 85 now.
So definitely as we move ahead, this number will decrease as the processing begins to complete.
So like 76, similar to 69.
So as we upload all the images over here, then I will click on Save and continue so that these images
are uploaded into my into this project over here.
And I can see that my dataset consists of 580 or 579 images.
Only 45 images are left, which are not annotated.
35 now.
And it's it this the figure will be actually decreased like 28 now.
And.
You're 19 now, so.
As we move ahead, this number will gradually decrease and writers never know which are not annotated.
And now it's three.
And so we can see that now all our dataset images are annotated and you can see over here and now I
will click on Save and continue to upload my dataset images into this project.
Okay, so I want to have a training.
70% of the images in the training set, 20% of the images in the validation set and 10% of the images
in the test set.
So we can also rebalance it as per our requirement.
But it is the standard practice to choose 70% of the images for the training, 20% for the validation
and 10% for the test set.
So I will select this and click on Continue.
So after click and continue.
Now all these files are being uploaded.
It will take some time, so I will be back as this files get uploaded and then we will discuss further.
Guys are on the dataset, images are uploaded.
Now you can see here our dataset consists of 579 images and here is the training data images, here
is the validation images and here is the test set images.
And if I click on all classes, you can see that we have a book and pun and mark null.
Class will be empty.
Okay, so we have images for the book class, we have images for the and class as well for training,
validation and test as well.
Okay, guys.
So now if I want to check the health of my dataset, I will click on health check and you can see that
we have total 579 images and all the images contain the annotations plus total number of bounding boxes,
or you can say annotations or labels in the in the 579 images are 2520.
So it means that we have 2520 bounding boxes in total on 579 images.
So in average, each image contains around 4 to 5 bounding boxes like you can see the 4.4 per images.
So each images contain around 4 to 5 bounding boxes.
Maybe two bounding boxes are of pen or two.
Bounding boxes are of both.
So in average, each image contain 4 to 5 bounding boxes and total we have 2520 bounding boxes on 579
images.
Okay.
And our image size is 1024 cross 768 So in the pre-processing we will rescale our image size to six
cross for cross 640 because our YOLO eight models are trained on 640 cross 640 images, so it will be
better if we consider 640 cross 640 images in our dataset for training the model.
So you can see the class balance.
So our data is a bit imbalanced because out of 2500 bounding boxes on 579 images, 295 bounding boxes
are of oak, while only 425 bounding boxes are of pen.
Okay, so we have around three times four times more bounding boxes for book and then pen.
Okay.
So here is the annotation heatmap, like in for example, this is the image.
So in this in the green or yellow region, we most usually have the bounding boxes.
Like in blue region, we don't have bounding boxes while in green and yellow region which which they
have the bounding boxes.
Now I will go to generate and I have total 579 images.
I have two classes, which is pen and book and all the images are aggregated.
No images are aggregated in the training set.
We have 405 images in the validation set.
We have 116 images in the testing set, we have 58 images.
So in the pre-processing step I will only resize the images to six cross 640 While there are other pre-processing
steps as well, like you can grayscale the images or modify the class name like I can change the book
name to the books or some other name, or you can change the pen name to ballpoint or some other name.
Okay.
So.
While you can remove the null images if you have or you can change the contrast of the images as well,
like you can make them more bright or dull as well.
Okay.
So I will only resize the images to 644 640 so if your dataset is low and you want to get good results
so you can add augmentation steps.
So using augmentation tab, you will increase the size of the dataset.
For example, if you click on 90 degree rotate.
So if I have 100 images in my dataset, so using 90 degree rotate now I will have 200 images in the
dataset because it will create a sample of images of all the images which are 90 degree rotated and
it will add them to my training dataset.
So augmentation helps us to increase the size of our dataset, plus it also make our model more resilient.
Okay, so there are a lot of augmentation steps which you can perform blood saturation you.
So in the case where you have less amount of dataset using data augmentation, you can increase the
size of your dataset.
Now I will generate this dataset, click on Generate and now my dataset will be generated.
So it might take some few seconds or few minutes.
So let's wait until our dataset gets generated and then we will export.
Okay, our dataset is generated.
So using Roboflow, we can also train our model and see what results do we get.
We can test our model on images or videos using Roboflow.
It's a no code solution.
You don't need to code anything, but I'll always like some code work.
So I will click on export and basically I want to convert into YOLO v5 PyTorch format because YOLO V8
and YOLO V5 have been has been released by ultralytics and both have the same format.
So I will select Yolov5 PyTorch format to train my YOLO V8 model and click on continue.
Now the file is zipping, so it might take some few seconds.
Okay, so I will just copy this code from here and I will just paste this code into my CoLab notebook.
So then I will train my model on YOLO model or this dataset.
So let's discuss or implement that training process of YOLO model in the next part of the tutorial.
See you there.