1
00:00:02,000 --> 00:00:03,000
All right, guys.

2
00:00:03,000 --> 00:00:12,000
So now we will see how we can train our model or potholes detection so you can see the pipeline in front

3
00:00:12,000 --> 00:00:12,000
of you.

4
00:00:12,000 --> 00:00:16,000
So first of all, we will import all these required libraries.

5
00:00:16,000 --> 00:00:24,000
So first of all, I am going to import OS, then I'm going to import import log, then I will import

6
00:00:24,000 --> 00:00:25,000
the display.

7
00:00:25,000 --> 00:00:32,000
Basically, I use this library to if I want to display any image output image or an input image into

8
00:00:32,000 --> 00:00:33,000
the Google CoLab notebook.

9
00:00:33,000 --> 00:00:37,000
Therefore we need the display library and we are just clearing the output.

10
00:00:37,000 --> 00:00:39,000
So here.

11
00:00:40,000 --> 00:00:46,000
Before running this tape, please make sure that you have selected the runtime as GPU.

12
00:00:46,000 --> 00:00:46,000
Okay.

13
00:00:46,000 --> 00:00:53,000
Please make sure that you have selected the runtime as GPU and we will check now whether we have access

14
00:00:53,000 --> 00:00:54,000
to GPU or not.

15
00:00:54,000 --> 00:00:59,000
Because if we don't have the access to GPU, the processing will become a bit slower, very slow.

16
00:01:00,000 --> 00:01:00,000
Okay.

17
00:01:00,000 --> 00:01:03,000
So we are using the GPU memory, so that's fine.

18
00:01:03,000 --> 00:01:04,000
Okay.

19
00:01:04,000 --> 00:01:11,000
So this home is basically the helping variable which helps us to redirect to different files or folder.

20
00:01:13,000 --> 00:01:16,000
But here we are installing ultralights using PIP install.

21
00:01:16,000 --> 00:01:23,000
So usually it can be implemented in two ways by installing the package which is PIP install ultralytics

22
00:01:23,000 --> 00:01:25,000
or by cloning the GitHub repo.

23
00:01:25,000 --> 00:01:31,000
But PIP install ultralytics is a very easy way, so we will use PIP install ultralytics to install the

24
00:01:31,000 --> 00:01:32,000
V8 package.

25
00:01:32,000 --> 00:01:32,000
Okay.

26
00:01:32,000 --> 00:01:35,000
So just using PIP install ultralytics.

27
00:01:35,000 --> 00:01:38,000
It will install the V8 package over here.

28
00:01:40,000 --> 00:01:42,000
But might take a few seconds, so please bear with me.

29
00:01:45,000 --> 00:01:45,000
Okay.

30
00:01:45,000 --> 00:01:46,000
So.

31
00:01:47,000 --> 00:01:51,000
I'll check if your drinks package is imported and it's working fine.

32
00:01:52,000 --> 00:01:55,000
By writing import ultralytics and ultralytics checks.

33
00:01:57,000 --> 00:02:04,000
Okay, so Internet takes 0.34 installed and set up complete and it's working verified.

34
00:02:04,000 --> 00:02:07,000
Okay, So now we'll go to over here.

35
00:02:07,000 --> 00:02:11,000
So this is the data set which we have prepared the path for data set.

36
00:02:11,000 --> 00:02:15,000
So in the last video I've shown you, like let me show you.

37
00:02:16,000 --> 00:02:21,000
This is the part whose dataset which we have generated in the last video.

38
00:02:21,000 --> 00:02:24,000
So this is the version which we generated.

39
00:02:24,000 --> 00:02:28,000
So we'll just click on this version and click on export.

40
00:02:28,000 --> 00:02:31,000
So if we export, then we'll click on continue.

41
00:02:31,000 --> 00:02:33,000
And this just copy this.

42
00:02:34,000 --> 00:02:37,000
Control C and just remove this.

43
00:02:40,000 --> 00:02:42,000
And just paste it over here.

44
00:02:42,000 --> 00:02:44,000
So what am doing is basically.

45
00:02:47,000 --> 00:02:48,000
Let me show you first one.

46
00:02:48,000 --> 00:02:55,000
So basically I'm just creating a datasets folder over here, and that means to perform this step over

47
00:02:55,000 --> 00:02:55,000
here.

48
00:02:55,000 --> 00:03:04,000
So it will create a dataset folder over here, let me show you so you can see that a dataset folder

49
00:03:04,000 --> 00:03:06,000
is created over here and it's empty.

50
00:03:06,000 --> 00:03:07,000
So.

51
00:03:08,000 --> 00:03:15,000
If you see our present working directory, if we digest, if we see our present working directory,

52
00:03:16,000 --> 00:03:18,000
it is this content.

53
00:03:18,000 --> 00:03:19,000
Okay.

54
00:03:19,000 --> 00:03:26,000
So if we want to save or if we want to after downloading this dataset, we want to save this dataset

55
00:03:26,000 --> 00:03:27,000
into this dataset folder.

56
00:03:27,000 --> 00:03:30,000
So I will set my current directory as.

57
00:03:35,000 --> 00:03:36,000
Yes.

58
00:03:36,000 --> 00:03:40,000
So now my current data directory sent is being set to this folder.

59
00:03:40,000 --> 00:03:41,000
Let me show you.

60
00:03:44,000 --> 00:03:49,000
So now you can see that my current directory is being sent set to this datasets folder.

61
00:03:49,000 --> 00:03:53,000
Now I will install dataset into this dataset folder.

62
00:03:54,000 --> 00:04:02,000
I am just installing this data set directly from Roboflow into this folder, so it might take few seconds

63
00:04:02,000 --> 00:04:04,000
to for the dataset to be downloaded.

64
00:04:06,000 --> 00:04:07,000
Like you can see here.

65
00:04:07,000 --> 00:04:10,000
So dataset is being downloaded.

66
00:04:13,000 --> 00:04:16,000
Imploding Workflow space Downloading dataset.

67
00:04:16,000 --> 00:04:18,000
Dataset usually is downloaded in the zip format.

68
00:04:18,000 --> 00:04:23,000
Then we basically adjust it according to our requirement.

69
00:04:23,000 --> 00:04:24,000
So.

70
00:04:24,000 --> 00:04:25,000
Oh, sorry, we don't adjust it.

71
00:04:25,000 --> 00:04:27,000
It's called automatically extracted.

72
00:04:28,000 --> 00:04:28,000
Okay.

73
00:04:28,000 --> 00:04:30,000
So our dataset is being downloaded.

74
00:04:30,000 --> 00:04:33,000
It's 46%, 48%.

75
00:04:34,000 --> 00:04:34,000
Okay.

76
00:04:34,000 --> 00:04:39,000
So we are just waiting for dataset download downloaded completes and then we can train YOLO V8 model

77
00:04:39,000 --> 00:04:41,000
on this custom dataset.

78
00:04:42,000 --> 00:04:43,000
Rutgers 89%.

79
00:04:43,000 --> 00:04:45,000
9599.

80
00:04:45,000 --> 00:04:47,000
Okay, so the dataset is downloaded and it's.

81
00:04:49,000 --> 00:04:49,000
Around.

82
00:04:49,000 --> 00:04:52,000
We can see this pothole dataset over here.

83
00:04:52,000 --> 00:04:57,000
Okay, so now we'll train our model on this custom dataset.

84
00:04:57,000 --> 00:04:58,000
Okay.

85
00:05:01,000 --> 00:05:04,000
And here is our data rotation, which is this.

86
00:05:04,000 --> 00:05:09,000
Okay, so let's train our V8 model on this, the same dataset.

87
00:05:09,000 --> 00:05:11,000
So first we need to go here.

88
00:05:11,000 --> 00:05:13,000
We can copy path.

89
00:05:14,000 --> 00:05:17,000
And just add this part over here.

90
00:05:17,000 --> 00:05:18,000
Okay.

91
00:05:18,000 --> 00:05:23,000
And yet here we are using Yolo v eight dot model.

92
00:05:24,000 --> 00:05:30,000
This model to fine tune our model on YOLO V8 on this custom data set.

93
00:05:30,000 --> 00:05:32,000
So we are what model we are using.

94
00:05:32,000 --> 00:05:34,000
Let me show you ultralytics.

95
00:05:35,000 --> 00:05:36,000
Your love.

96
00:05:36,000 --> 00:05:36,000
The air.

97
00:05:38,000 --> 00:05:38,000
Okay.

98
00:05:39,000 --> 00:05:41,000
Uh, just going on here.

99
00:05:41,000 --> 00:05:43,000
Okay, let me go to the GitHub repository.

100
00:05:46,000 --> 00:05:51,000
Okay, so here I am, went into the GitHub repo so you can let me show you.

101
00:05:51,000 --> 00:05:55,000
So we have YOLO V8 models from five different models of YOLO.

102
00:05:55,000 --> 00:05:57,000
YOLO, YOLO, V8'S.

103
00:05:57,000 --> 00:06:02,000
YOLO V8 Medium, YOLO V8, large and extra large.

104
00:06:02,000 --> 00:06:09,000
So although this is the this is a smallest model and this is very fast, but it is less accurate and

105
00:06:09,000 --> 00:06:15,000
YOLO v8 X is basically more accurate, but it is less fast.

106
00:06:15,000 --> 00:06:18,000
Or you can say it is a bit bit slower than YOLO v8.

107
00:06:18,000 --> 00:06:19,000
Other models.

108
00:06:19,000 --> 00:06:25,000
Okay, but it is more accurate model so we can use YOLO, yolo V8, medium older model which will be

109
00:06:25,000 --> 00:06:25,000
fine.

110
00:06:26,000 --> 00:06:27,000
So just right.

111
00:06:27,000 --> 00:06:28,000
Yolo V8 empty.

112
00:06:29,000 --> 00:06:36,000
And here I've set my dataset path and I'm training for 70 epochs and I have set my image size as 640.

113
00:06:36,000 --> 00:06:36,000
Okay.

114
00:06:37,000 --> 00:06:37,000
Okay.

115
00:06:37,000 --> 00:06:42,000
Now that's mod is equal to train and we are performing object detection.

116
00:06:42,000 --> 00:06:44,000
Let's click on training.

117
00:06:46,000 --> 00:06:50,000
Let's see if there is any error so we can fix or the training starts.

118
00:06:50,000 --> 00:06:52,000
So there is an error.

119
00:06:52,000 --> 00:06:55,000
Like data.is not found.

120
00:06:55,000 --> 00:06:56,000
Okay.

121
00:06:56,000 --> 00:06:56,000
Let me see.

122
00:06:56,000 --> 00:06:57,000
Okay.

123
00:06:57,000 --> 00:07:01,000
Basically is dash three in the path is basically creating an error.

124
00:07:01,000 --> 00:07:02,000
Let me fix it.

125
00:07:02,000 --> 00:07:05,000
Just make it this and just correct it from here.

126
00:07:05,000 --> 00:07:08,000
But we need to do other things as well.

127
00:07:08,000 --> 00:07:10,000
We need to go to data dot yml.

128
00:07:11,000 --> 00:07:15,000
And here we need to update our folder path as well.

129
00:07:15,000 --> 00:07:17,000
Just copy path.

130
00:07:18,000 --> 00:07:19,000
And just paste it over here.

131
00:07:20,000 --> 00:07:22,000
Just go here.

132
00:07:22,000 --> 00:07:27,000
Just copy this path and just paste this over here.

133
00:07:29,000 --> 00:07:31,000
Just an aghori here as well.

134
00:07:31,000 --> 00:07:34,000
And just copy a path and just.

135
00:07:36,000 --> 00:07:45,000
Click over here and control V Okay, so we have the only class name as Porthole and all others are fine.

136
00:07:45,000 --> 00:07:45,000
Okay.

137
00:07:46,000 --> 00:07:52,000
So just now I hope that there will no error and the training will work fine.

138
00:07:52,000 --> 00:07:55,000
But let's see if there is any other error.

139
00:07:55,000 --> 00:07:56,000
We will definitely correct it.

140
00:08:00,000 --> 00:08:06,000
Okay, so the training is about to start, so it might take some time.

141
00:08:06,000 --> 00:08:07,000
So.

142
00:08:08,000 --> 00:08:14,000
So we have 1340 images in the training set and in the vector validation, we have 133 images.

143
00:08:14,000 --> 00:08:15,000
Okay.

144
00:08:15,000 --> 00:08:17,000
So it might take some time.

145
00:08:17,000 --> 00:08:22,000
The training has started, but let's see, how much time does it takes?

146
00:08:22,000 --> 00:08:28,000
Okay, so I'm training my model for the 70 epochs and it will take some time to train.

147
00:08:28,000 --> 00:08:34,000
So I will be back as the training completes and then we will start from there and validate and test

148
00:08:34,000 --> 00:08:39,000
our model on some images as well as on videos and see how our model performs.

149
00:08:39,000 --> 00:08:44,000
So see, see you as the training completes and then we will validate our model.

150
00:08:52,000 --> 00:08:54,000
So guys, the training has completed.

151
00:08:54,000 --> 00:08:57,000
So we have trained our model on 70 epochs.

152
00:08:57,000 --> 00:09:01,000
So it took more than an hour for the training to complete.

153
00:09:01,000 --> 00:09:07,000
So we have got the last weird spy on the which is basically the last epoch and the best weights file

154
00:09:07,000 --> 00:09:11,000
on which our model has given us the best performance or results.

155
00:09:11,000 --> 00:09:18,000
So here in the weights section, we can see the best best weights file and last dot means the last weights

156
00:09:18,000 --> 00:09:20,000
file like on the 17th epoch.

157
00:09:20,000 --> 00:09:22,000
So here is the model summary.

158
00:09:23,000 --> 00:09:31,000
We have got the mean average precision with 50 as 72% represented by 0.72 and the mean average precision

159
00:09:31,000 --> 00:09:36,000
when varies from 50 to 95 is 0.408.

160
00:09:36,000 --> 00:09:41,000
So the results or the performance of our model is very satisfying.

161
00:09:41,000 --> 00:09:43,000
Or you can say that it's giving good results.

162
00:09:44,000 --> 00:09:46,000
So in that runs detect train folder.

163
00:09:46,000 --> 00:09:54,000
So in all our results, the confusion matrix, recall curves, F1 curves, precision curve validation,

164
00:09:54,000 --> 00:09:58,000
batch results are saved in this runs detect train folder.

165
00:09:58,000 --> 00:10:00,000
So am just checking here.

166
00:10:00,000 --> 00:10:04,000
The different files I have in the past detect train folder.

167
00:10:04,000 --> 00:10:06,000
So just just running this.

168
00:10:06,000 --> 00:10:09,000
It will show you that we have a precision curve over here.

169
00:10:09,000 --> 00:10:15,000
F1 curve over here, precision recall curve over here and results dot csv which contain the performance

170
00:10:15,000 --> 00:10:17,000
of the model on each epoch.

171
00:10:17,000 --> 00:10:22,000
Then we have results dot png over here in the results dot PNG file.

172
00:10:23,000 --> 00:10:30,000
Basically we have the training and validation loss graphs and here we have the prediction on them of

173
00:10:30,000 --> 00:10:36,000
the model on validation batches and their labels and the prediction of the models on the train batches.

174
00:10:36,000 --> 00:10:39,000
So let us first describe the confusion matrix.

175
00:10:39,000 --> 00:10:45,000
So Confusion Matrix is basically a chart that shows how our model handles different classes.

176
00:10:45,000 --> 00:10:49,000
So we can see here pothole and this is the prediction.

177
00:10:49,000 --> 00:10:56,000
So 68% of the time our model correctly detected that there is a pothole when there is a pothole, while

178
00:10:56,000 --> 00:11:00,000
32% of time when there was a pothole.

179
00:11:00,000 --> 00:11:03,000
But our model was trained to detect the pothole.

180
00:11:03,000 --> 00:11:10,000
So we can say that a 68% of the time our model correctly detected that there is a pothole, while 32%

181
00:11:10,000 --> 00:11:14,000
of the time when the pothole is there, but the model is not able to detect it.

182
00:11:15,000 --> 00:11:15,000
Okay.

183
00:11:17,000 --> 00:11:20,000
So here is the graph of the training and validation loss.

184
00:11:20,000 --> 00:11:25,000
So in the training and validation loss, the box loss and the task loss is most important.

185
00:11:25,000 --> 00:11:28,000
The others don't matter much.

186
00:11:28,000 --> 00:11:34,000
So the overall behavior of the model is very convincing and we can see that the model is converging.

187
00:11:34,000 --> 00:11:38,000
Then you can see that the loss is decreasing in the downward direction.

188
00:11:38,000 --> 00:11:43,000
While the precision and recall curve is moving upward is improving.

189
00:11:43,000 --> 00:11:50,000
So if we train our model, higher epochs like 150 epochs or 200 epochs, we will get more better results.

190
00:11:50,000 --> 00:11:50,000
Okay.

191
00:11:50,000 --> 00:11:54,000
You can see that the loss is clearly decreasing and the precision is increasing.

192
00:11:56,000 --> 00:11:59,000
So here are the model prediction results on the validation batch.

193
00:11:59,000 --> 00:12:03,000
So these images are basically not used for training.

194
00:12:03,000 --> 00:12:09,000
So it is always better to take a look like how our model is behaving on the validation dataset.

195
00:12:09,000 --> 00:12:12,000
Like you can see here, the model performance is very convincing.

196
00:12:12,000 --> 00:12:14,000
It is able to detect the potholes.

197
00:12:14,000 --> 00:12:18,000
Okay, so now we will validate our custom model.

198
00:12:18,000 --> 00:12:24,000
So here we are taking the best weights file and using them to validate our model.

199
00:12:24,000 --> 00:12:29,000
Similarly as before, we are using the command line interface to do that.

200
00:12:29,000 --> 00:12:35,000
So the only difference is that previously we write mod is equal to train, but here we are writing mod

201
00:12:35,000 --> 00:12:41,000
is equal to validation because we are validating our model on the validation images.

202
00:12:41,000 --> 00:12:42,000
So.

203
00:12:43,000 --> 00:12:44,000
So validation.

204
00:12:44,000 --> 00:12:50,000
So this script, basically this cell is using the test dataset images which are not used before.

205
00:12:50,000 --> 00:12:55,000
So you can just run this and to validate the custom order.

206
00:12:55,000 --> 00:12:58,000
So it might take a few minutes, I think.

207
00:12:58,000 --> 00:13:04,000
I have not given the break path, so just go over here and where you have the best rates file, just

208
00:13:04,000 --> 00:13:07,000
click on copy path and just.

209
00:13:09,000 --> 00:13:12,000
Control V and just remove this.

210
00:13:12,000 --> 00:13:13,000
Okay.

211
00:13:13,000 --> 00:13:17,000
And I have already set the data dot yml file.

212
00:13:17,000 --> 00:13:18,000
Okay, that's fine.

213
00:13:18,000 --> 00:13:19,000
And just run this cell now.

214
00:13:20,000 --> 00:13:23,000
So here we are, validating our custom model.

215
00:13:23,000 --> 00:13:26,000
So it might take about few minutes.

216
00:13:26,000 --> 00:13:28,000
So let's wait and see.

217
00:13:28,000 --> 00:13:34,000
I will also add this CoLab file in the attachment so you can run this script on your site as well.

218
00:13:37,000 --> 00:13:40,000
So we have validated our custom order.

219
00:13:41,000 --> 00:13:43,000
So we have validated our custom order.

220
00:13:43,000 --> 00:13:50,000
So we are getting a mean average precision with 50 is the 72% and mean average precision when varies

221
00:13:50,000 --> 00:13:53,000
from 50 to 95 is 0.40%.

222
00:13:53,000 --> 00:13:57,000
7% means 40%, which is a good and fine.

223
00:13:57,000 --> 00:14:04,000
So now we will run our model on the test dataset images which are over here.

224
00:14:06,000 --> 00:14:07,000
Like you can see.

225
00:14:07,000 --> 00:14:14,000
So these are the test data set images on this will run the model and see what those do I get.

226
00:14:14,000 --> 00:14:18,000
So I have set the path already and just run this cell.

227
00:14:19,000 --> 00:14:26,000
So here I'm doing prediction on the test dataset images and let's see what results do we get.

228
00:14:26,000 --> 00:14:27,000
Okay.

229
00:14:30,000 --> 00:14:31,000
We take my take a few seconds.

230
00:14:31,000 --> 00:14:31,000
Okay.

231
00:14:31,000 --> 00:14:33,000
Let's display these images.

232
00:14:34,000 --> 00:14:35,000
We're here.

233
00:14:38,000 --> 00:14:38,000
Okay.

234
00:14:38,000 --> 00:14:41,000
Award is basically file pod image.

235
00:14:41,000 --> 00:14:42,000
Pod.

236
00:14:43,000 --> 00:14:44,000
Everyone.

237
00:14:44,000 --> 00:14:46,000
Let me see what it's giving this error.

238
00:14:50,000 --> 00:14:55,000
But there was an issue in the script, so I just replaced it with the predicted single image.

239
00:14:55,000 --> 00:14:57,000
And here is our results.

240
00:14:57,000 --> 00:15:00,000
So you can see that the model is perfectly detecting that there is a pothole.

241
00:15:01,000 --> 00:15:08,000
Now what we will be testing our model on a demo video, so just downloading the demo video from a drive.

242
00:15:09,000 --> 00:15:13,000
Okay, so now we will test our model on this demo video.

243
00:15:14,000 --> 00:15:17,000
So it might take a few seconds for this script to run.

244
00:15:17,000 --> 00:15:23,000
So let's wait until this is done and then we will see what results do we get.

245
00:15:24,000 --> 00:15:26,000
Okay, So our model is detecting potholes.

246
00:15:26,000 --> 00:15:36,000
Like you can see that for one one and okay, so let's see where our results are saving runs detect,

247
00:15:36,000 --> 00:15:37,000
predict two.

248
00:15:38,000 --> 00:15:41,000
Okay, so let's display this output demo video.

249
00:15:42,000 --> 00:15:46,000
The demo video name is demo dot mp4.

250
00:15:46,000 --> 00:15:50,000
So okay, just check what results we are getting.

251
00:15:51,000 --> 00:15:53,000
Either they are satisfying or not.

252
00:15:53,000 --> 00:15:57,000
So I'm just displaying the output demo video over here.

253
00:15:57,000 --> 00:15:59,000
Then I will show it to you as well.

254
00:16:06,000 --> 00:16:08,000
Getting the an output demo video.

255
00:16:08,000 --> 00:16:11,000
Let me download it and see what results do we get.

256
00:16:12,000 --> 00:16:14,000
So just downloading it.

257
00:16:14,000 --> 00:16:15,000
And displayed.

258
00:16:15,000 --> 00:16:15,000
Okay.

259
00:16:15,000 --> 00:16:19,000
So you can see that the model is detecting perfectly that there are potholes.

260
00:16:20,000 --> 00:16:20,000
Okay.

261
00:16:20,000 --> 00:16:25,000
So it's is also detecting and here it's also detecting a pothole.

262
00:16:25,000 --> 00:16:28,000
So the results are overall fine.

263
00:16:28,000 --> 00:16:31,000
Like you can see that the model is working very perfectly.

264
00:16:32,000 --> 00:16:32,000
Okay.

265
00:16:33,000 --> 00:16:37,000
So let's see if we can test on some other demo video as well.

266
00:16:37,000 --> 00:16:38,000
Okay.

267
00:16:38,000 --> 00:16:44,000
So currently I've only have this demo video, so I hope you have learned a lot from this lecture.

268
00:16:44,000 --> 00:16:47,000
See you all in the next lecture.

269
00:16:47,000 --> 00:16:48,000
Till then, bye bye.