1
00:00:00,240 --> 00:00:07,590
Hello, everyone, and welcome to this new session in which will treat Model Virgin with one TB.

2
00:00:07,620 --> 00:00:14,250
Now, in the previous session we had looked at this had versioning in which we had created an augmented

3
00:00:14,250 --> 00:00:20,940
version of our data, which happens to be this version right here from previous versions like a pre

4
00:00:20,940 --> 00:00:26,910
processed version, another pre processed version and the original data.

5
00:00:27,120 --> 00:00:36,930
And so from here we now going to see how to implement model versioning using this augmented data and

6
00:00:36,930 --> 00:00:39,420
the untrained model version.

7
00:00:39,420 --> 00:00:44,640
So we have this version of the model which is untrained and the augmented data which are going to be

8
00:00:44,640 --> 00:00:49,100
passed in this run to produce our trained model version.

9
00:00:49,110 --> 00:00:56,310
In the previous session, we saw how to implement data set versioning with one DB artifacts we left

10
00:00:56,310 --> 00:01:01,980
from this original raw data right here onto this pre processed data.

11
00:01:01,980 --> 00:01:09,420
We created a trained validation and test sets and then finally we had this augmented data right here

12
00:01:09,630 --> 00:01:16,200
and now we are going to see how to make use of this augmented data in training our model.

13
00:01:16,200 --> 00:01:24,270
But while training this model, we are going to implement model versioning and the two model versions

14
00:01:24,270 --> 00:01:30,300
will have here will be the model before training and the model after training.

15
00:01:30,300 --> 00:01:35,700
And so we're making use of this sequential model which we had created previously.

16
00:01:35,700 --> 00:01:42,690
And the good news here is the way we implement model versioning is quite similar to the way we implement

17
00:01:42,690 --> 00:01:43,940
dataset versioning.

18
00:01:43,950 --> 00:01:48,330
We'll start by copying out this code and then we'll simply modify that.

19
00:01:48,330 --> 00:01:54,840
In here we have this one DB model versioning, let's piece that out and then instead of augment data,

20
00:01:54,840 --> 00:01:57,750
we are going to log our model.

21
00:01:57,750 --> 00:02:06,780
So here we have log model, which is in fact this net model right here we have the log model want to

22
00:02:06,780 --> 00:02:07,470
be in needs.

23
00:02:07,470 --> 00:02:14,670
Our project entity specified the artifacts to be used here is known, so we're not going to make use

24
00:02:14,670 --> 00:02:19,350
of any artifact to lock this model artifact right here.

25
00:02:19,350 --> 00:02:22,530
And so we that we have this take of this augmented data.

26
00:02:22,530 --> 00:02:33,210
We have our model let's call model or untrained, untrained model, and then we specify that it is this

27
00:02:33,210 --> 00:02:37,410
untrained model, it's of type model.

28
00:02:37,410 --> 00:02:46,530
So we have that type model, the initial version of our net model.

29
00:02:46,860 --> 00:02:47,850
There we go.

30
00:02:47,850 --> 00:02:52,560
Now we have that done this artifact file here.

31
00:02:52,560 --> 00:02:54,150
We're going to make use of this.

32
00:02:54,240 --> 00:02:56,220
We're not making use of any previous artifacts.

33
00:02:56,220 --> 00:02:59,070
We just log in this model initially.

34
00:02:59,070 --> 00:03:02,550
And then here, we'll take this off.

35
00:03:02,670 --> 00:03:08,310
We're not going to make use of this just before writing in our model.

36
00:03:08,310 --> 00:03:09,840
We're going to save that model.

37
00:03:09,840 --> 00:03:12,510
So here we have this model to be saved.

38
00:03:12,510 --> 00:03:14,580
We call it's all in that model, actually.

39
00:03:14,580 --> 00:03:19,320
So we have learned model that safe and then we'll give it a file name.

40
00:03:19,320 --> 00:03:23,010
So let's call it Loonette dot h five and that's it.

41
00:03:23,010 --> 00:03:24,900
So we've saved this model.

42
00:03:25,320 --> 00:03:28,470
Let's have this file name to be Loonette H five.

43
00:03:28,470 --> 00:03:34,830
So let's define this Loonette H five file here and take this off.

44
00:03:35,130 --> 00:03:37,530
So we have now our file name.

45
00:03:37,830 --> 00:03:43,650
Now, since we've already saved this file, instead of going through this process here, I've created

46
00:03:43,650 --> 00:03:45,120
a new file and all of that.

47
00:03:45,120 --> 00:03:47,610
We're just going to add a file.

48
00:03:47,610 --> 00:03:55,830
So here we have our untrained model dot at file, and then we'll specify the file name.

49
00:03:55,980 --> 00:03:57,480
So that's all we need to do.

50
00:03:57,480 --> 00:04:04,530
And then we're going to also do one TB dot, save that file based on the file name and that's it.

51
00:04:04,530 --> 00:04:10,620
So let's take off this part here and then we're going to log our artifact, which happens to be our

52
00:04:10,620 --> 00:04:12,390
untrained model.

53
00:04:12,390 --> 00:04:15,230
So we have your untrained model, and that's fine.

54
00:04:15,240 --> 00:04:17,180
Now we could add in some merit.

55
00:04:17,180 --> 00:04:24,150
Editor So let's have meta data, which equals our configuration, which we've defined already in here.

56
00:04:24,180 --> 00:04:25,500
Let's modify this name.

57
00:04:25,500 --> 00:04:28,530
This is our untrained model.

58
00:04:28,530 --> 00:04:31,410
So your untrained model and that's fine.

59
00:04:31,410 --> 00:04:34,170
So with this, we could now run this here.

60
00:04:35,570 --> 00:04:42,590
We have this successful run right here, and then we could now go back to one to be getting back here.

61
00:04:42,590 --> 00:04:46,280
We have this under our artifacts, we have the model.

62
00:04:46,430 --> 00:04:52,580
So yeah, we could click on this model and then we specified the untrained model.

63
00:04:52,580 --> 00:04:56,120
So let's click on this and we have this overview.

64
00:04:56,150 --> 00:05:02,360
You see, we have that API metadata, you see the metadata we've locked in, we have the files, you

65
00:05:02,360 --> 00:05:07,990
see the LINET model, and then we have, as usual, our grab view, which we could explode.

66
00:05:08,000 --> 00:05:12,560
Notice that it's quite simple because for now it isn't linked to our data set.

67
00:05:12,560 --> 00:05:19,880
So all we have is just this run which produces this untrained model or this initialized model.

68
00:05:20,620 --> 00:05:21,610
From this point.

69
00:05:21,610 --> 00:05:27,430
We'll go ahead to train our model and lock that trained version of the model.

70
00:05:27,460 --> 00:05:33,460
So again, here we have this artifact, which we are going to use in training, the model which happens

71
00:05:33,460 --> 00:05:35,560
to be our augmented data set.

72
00:05:35,560 --> 00:05:38,980
Recall that we had logged this augmented data previously.

73
00:05:38,980 --> 00:05:40,600
We could have this overview from here.

74
00:05:40,600 --> 00:05:45,400
We have this augmented data set and then it's subtype pre processed data.

75
00:05:45,400 --> 00:05:46,180
We've seen that already.

76
00:05:46,180 --> 00:05:53,470
We download the data and then we create this new artifact which is going to be our trained sequential

77
00:05:53,470 --> 00:05:54,340
model.

78
00:05:54,340 --> 00:06:01,900
We call this trained sequential model subtype model description, a trained version of our model, as

79
00:06:01,900 --> 00:06:02,710
simple as that.

80
00:06:02,710 --> 00:06:03,490
So that is it.

81
00:06:03,520 --> 00:06:10,870
We could also pass in the metal data, just as we had seen here with this log model where we pass in

82
00:06:10,870 --> 00:06:11,980
the configuration.

83
00:06:11,980 --> 00:06:16,900
So metadata equals our configuration, and that's fine.

84
00:06:17,170 --> 00:06:22,000
Getting back to this augmented data method, we have seen previously, we're going to follow this same

85
00:06:22,000 --> 00:06:22,420
pattern.

86
00:06:22,420 --> 00:06:26,590
That is, we're going to download the data as we've done already.

87
00:06:26,590 --> 00:06:30,910
We download the data and then we have this artifact file here.

88
00:06:30,910 --> 00:06:37,330
The artifact file we're using was a trained data set because we needed to take this and convert it into

89
00:06:37,480 --> 00:06:38,770
augmented data set.

90
00:06:38,770 --> 00:06:43,670
And then we had this processing right here to produce dataset X and dataset Y.

91
00:06:43,690 --> 00:06:50,310
Now we're going to do this exact procedure for training our data on the augmented data.

92
00:06:50,320 --> 00:06:57,970
So let's paste this out right here and then your AR artifact file will be this data set.

93
00:06:57,970 --> 00:07:06,460
So here we have our data set and we get this right from your let's get back to our data set files and

94
00:07:06,460 --> 00:07:07,050
there we go.

95
00:07:07,060 --> 00:07:11,470
We see we have this ARG data set Z, which we start previously.

96
00:07:11,470 --> 00:07:16,720
And so once we are going to download this, we're going to have this file which is going to be found

97
00:07:16,720 --> 00:07:20,620
in this folder, which is also found in this artifacts folder.

98
00:07:20,620 --> 00:07:24,340
So we have artifacts, augmented data set version zero, which is from this.

99
00:07:24,340 --> 00:07:28,510
This means that if we want to use the version one, we just simply have to specify your version of one

100
00:07:28,510 --> 00:07:29,580
and so on and so forth.

101
00:07:29,590 --> 00:07:35,680
So that said, we have our artifact file data set and then we go to this processing to obtain dataset

102
00:07:35,680 --> 00:07:44,230
X and dataset Y, and then from here we'll go to this series of steps to convert this data into a TensorFlow

103
00:07:44,230 --> 00:07:45,880
data set type data.

104
00:07:45,880 --> 00:07:52,870
So here we have the x, dx, y, this dataset X, we convert to tensor, so we convert this dataset

105
00:07:52,870 --> 00:07:55,630
X to a tensor and they said Y to a tensor.

106
00:07:55,640 --> 00:08:03,310
Then from here we apply this from tensor slices method, which takes the X and D Y, That's our dataset

107
00:08:03,310 --> 00:08:08,760
X, and this said Y to create this new dataset D.

108
00:08:08,770 --> 00:08:15,190
Now we go to the same procedures we have seen already as we shuffle, we batch, we pre fetch and then

109
00:08:15,190 --> 00:08:17,170
we have our training data set.

110
00:08:17,200 --> 00:08:23,260
Now note that you could repeat this same process for the validation, but here we just do this for the

111
00:08:23,260 --> 00:08:27,970
training and you could take it as an exercise to do the same for the validation.

112
00:08:27,970 --> 00:08:32,080
Note that for validation we didn't make use of the augmented dataset.

113
00:08:32,080 --> 00:08:34,920
We instead made use of the validation dataset itself.

114
00:08:34,930 --> 00:08:36,730
Let's get to the graph review so you could see that.

115
00:08:36,730 --> 00:08:37,330
Clara.

116
00:08:38,150 --> 00:08:44,090
And as you could see here, we have this data which is pre processed pre processed data to train data,

117
00:08:44,090 --> 00:08:45,530
set pre process data.

118
00:08:45,530 --> 00:08:48,080
The validation that the separate process data test data set.

119
00:08:48,080 --> 00:08:51,890
And then from here we had the augmented data, which is this one.

120
00:08:51,890 --> 00:08:58,940
And so when you want to carry out validation, you're not going to be making use of this training data

121
00:08:58,940 --> 00:08:59,600
obviously.

122
00:08:59,600 --> 00:09:02,000
So we're going to make use of this one instead.

123
00:09:02,000 --> 00:09:09,650
And so if you are to put in the validation in your that's convert the validation data into a TensorFlow

124
00:09:09,650 --> 00:09:18,620
dataset type data and then use it for training and or evaluation, then you would have to make sure

125
00:09:18,620 --> 00:09:20,270
you modify this year.

126
00:09:20,690 --> 00:09:23,690
So that's that we continue with this process.

127
00:09:23,690 --> 00:09:29,960
We have the trained data which we just created is now a tensorflow dataset type data.

128
00:09:30,080 --> 00:09:33,860
And so training can now go on smoothly.

129
00:09:33,870 --> 00:09:40,670
So you see here we have the metrics we've used to see in this already compiling and then we feed the

130
00:09:40,670 --> 00:09:41,420
model.

131
00:09:41,420 --> 00:09:43,970
So yeah, we're now going to include the validation.

132
00:09:44,630 --> 00:09:50,060
We have that ready, we're going to do the training and after training our model would have this.

133
00:09:50,060 --> 00:09:55,340
So let's, let's simply just have that net model that fit.

134
00:09:55,340 --> 00:10:02,540
And then after this we will have similar to what we had seen already here in this log model method,

135
00:10:02,540 --> 00:10:10,430
the file which has been saved and then saved to one TB, and then the artifact which has been locked.

136
00:10:10,430 --> 00:10:15,310
So let's copy this out and then get back here and paste this.

137
00:10:15,320 --> 00:10:16,940
Now let's change this name.

138
00:10:16,940 --> 00:10:17,570
Let's see.

139
00:10:17,570 --> 00:10:20,780
Loonette trained, Loonette trained.

140
00:10:20,780 --> 00:10:24,650
That's our file name on train, model dot and file.

141
00:10:24,650 --> 00:10:26,060
It's actually.

142
00:10:26,450 --> 00:10:29,690
Yeah, we have this trained sequential model.

143
00:10:29,690 --> 00:10:38,390
So let's have this as trained sequential model.

144
00:10:38,390 --> 00:10:39,140
There we go.

145
00:10:39,140 --> 00:10:41,810
We have the train sequential model or file file.

146
00:10:41,810 --> 00:10:48,500
Name one, DBC file name, run the log artifact, the trained sequential model.

147
00:10:48,500 --> 00:10:51,590
Trained sequential model.

148
00:10:51,620 --> 00:10:52,520
Okay, So that's it.

149
00:10:52,520 --> 00:10:54,380
So we have this set.

150
00:10:54,380 --> 00:10:57,560
We could now run this method and then see what we get.

151
00:10:57,560 --> 00:11:03,440
We obtain this error where we've been told the name Loonette models and defined now, as you had seen

152
00:11:03,440 --> 00:11:11,240
previously with this log model method, we made use of the Loonette model, which we had defined right

153
00:11:11,240 --> 00:11:11,570
here.

154
00:11:11,570 --> 00:11:18,050
So we defined this Loonette model as using the sequential API, and that's what we had.

155
00:11:18,050 --> 00:11:22,280
And this led us to having this year where we had a run.

156
00:11:22,280 --> 00:11:28,070
So initially we had this run which produced our untrained model.

157
00:11:28,070 --> 00:11:32,930
So we have this year on trained model, let's call it, um, untrained model.

158
00:11:32,930 --> 00:11:39,830
And since in our case we're trying to build our train model is going to be reasonable for us to make

159
00:11:39,830 --> 00:11:47,150
use of this untrained model which we have stored already in one DB since we're doing model versioning.

160
00:11:47,150 --> 00:11:53,990
And so instead of us making use of this learned model, we are going to make use of this version of

161
00:11:53,990 --> 00:11:57,440
the untrained model, which will start already in one DB.

162
00:11:57,440 --> 00:12:04,070
So from here we're going to run another process which now will lead us to this trained model.

163
00:12:04,070 --> 00:12:07,640
But then to run this process, we'll also need theater.

164
00:12:07,640 --> 00:12:14,990
So as you could see, you have the data that's augmented editor From here we have the trained data.

165
00:12:14,990 --> 00:12:18,680
We have different order notes which came before this.

166
00:12:18,680 --> 00:12:25,760
So we call our weights in before having the untrained augmented data, which is this one year, and

167
00:12:25,760 --> 00:12:33,590
then this augmented theater, together with this untrained model, will produce our trained model.

168
00:12:33,590 --> 00:12:37,580
And once we have this trained model, we can now use it for prediction.

169
00:12:37,580 --> 00:12:44,900
And so coming back again, if we had to use this Loonette model, which was defined here in the note

170
00:12:44,900 --> 00:12:53,570
book, would have the this this data set that will have our data which will pass into the run, and

171
00:12:53,570 --> 00:12:57,230
then we'll have our outputted train model.

172
00:12:57,530 --> 00:13:05,180
And then on the other side we'll just have this output at like we have this run and then we have this

173
00:13:05,180 --> 00:13:06,710
output, that untrained model.

174
00:13:06,710 --> 00:13:13,940
So we have the untrained model and the trained model which should exist like two separate entities in

175
00:13:13,940 --> 00:13:15,200
our global graph.

176
00:13:15,200 --> 00:13:18,860
Whereas what we want to do is to be able to link this to up.

177
00:13:18,860 --> 00:13:22,880
And so that's why we are not going to make use of this linear model which was defined here.

178
00:13:22,880 --> 00:13:27,710
And this is why we're going to make use of the model which has already been stored in one DB.

179
00:13:27,740 --> 00:13:34,520
That said, getting back to the code, we're going to copy this out here and then in the place of this

180
00:13:34,520 --> 00:13:37,280
Loonette model, as in.

181
00:13:37,690 --> 00:13:40,350
Of using the internet model with defined with defined.

182
00:13:40,360 --> 00:13:40,930
Year.

183
00:13:40,960 --> 00:13:44,950
We are now going to make use of our artifact which was stored already.

184
00:13:44,950 --> 00:13:46,870
So let's let's have this here.

185
00:13:46,870 --> 00:13:54,430
So let's have that pasted here and then we will make use of this artifact, which is in fact called.

186
00:13:55,390 --> 00:13:57,040
The untrained model.

187
00:13:57,040 --> 00:14:04,240
So you see, we have this year let's copy this out, we copy that out, and then we paste this here.

188
00:14:04,450 --> 00:14:08,320
Let's paste this here and there we go.

189
00:14:08,650 --> 00:14:16,630
Just after downloading the model, the next step will be to have this artifact file, which is simply

190
00:14:16,990 --> 00:14:20,910
our untrained model, which we should have done, which we should have downloaded.

191
00:14:20,920 --> 00:14:27,790
So in that case, we're going to have this year artifacts starting the artifacts, directory artifacts.

192
00:14:27,790 --> 00:14:34,900
And then in here we take this all from years, copy this and paste out your untrained model.

193
00:14:34,900 --> 00:14:39,270
And then to be more specific, we could get the file name from your.

194
00:14:39,280 --> 00:14:44,100
So here you see, we have the file name you could get back here overview and then you have files little

195
00:14:44,140 --> 00:14:45,430
net each five.

196
00:14:45,430 --> 00:14:47,800
So this is a file we're going to download.

197
00:14:47,800 --> 00:14:49,660
And yours is it your.

198
00:14:49,660 --> 00:14:50,710
So we have that.

199
00:14:50,710 --> 00:14:53,980
So now we have the artifact file which we want to download.

200
00:14:53,980 --> 00:15:00,580
We have this path and then now we could make use of tensor flows, load model, method.

201
00:15:00,580 --> 00:15:09,790
So here we have now the net model, which is equal to the cross models and load model which takes in

202
00:15:09,790 --> 00:15:12,160
the artifact file.

203
00:15:12,430 --> 00:15:13,300
There we go.

204
00:15:13,300 --> 00:15:20,980
So now we have this little net model which is not gotten from this definition here, but which is gotten

205
00:15:20,980 --> 00:15:25,470
from a previous version of our model, which has been stored in one DB.

206
00:15:25,480 --> 00:15:29,080
So let's now run this and see what we get.

207
00:15:29,320 --> 00:15:36,400
After training for over three E books, your other results we get and then we get to one DB.

208
00:15:36,430 --> 00:15:41,430
You see here we have the model trained sequential model and then the all train model.

209
00:15:41,440 --> 00:15:45,130
So let's click on this train sequential model loading artifacts.

210
00:15:45,400 --> 00:15:46,570
There we go.

211
00:15:46,570 --> 00:15:52,300
We could look at this API metadata which we loaded and the file.

212
00:15:52,300 --> 00:15:58,930
So here we have Linet trained and then we also have our graph view explode in this view, here's what

213
00:15:58,930 --> 00:15:59,650
we get.

214
00:15:59,680 --> 00:16:02,560
Now, let's focus on this part here.

215
00:16:02,740 --> 00:16:08,110
And as you can see, we have this run which produces this untrained model.

216
00:16:08,110 --> 00:16:08,530
Here.

217
00:16:08,530 --> 00:16:15,850
We have this untrained model and then this untrained model, together with our augmented data set,

218
00:16:15,880 --> 00:16:23,110
been passed through this run, now produce our trained sequential model.

219
00:16:23,110 --> 00:16:29,320
And that's it for the section in which we've seen how to implement model versioning with one DB.