WEBVTT 00:00.510 --> 00:02.850 -: Hello and welcome to this Python tutorial. 00:02.850 --> 00:05.670 All right, so we have very exciting tutorials ahead of us. 00:05.670 --> 00:06.510 We are gonna start 00:06.510 --> 00:09.060 by creating the architecture of the neural network. 00:09.060 --> 00:11.130 That is, we will make the neural network 00:11.130 --> 00:13.590 that will be at the heart of our AI, 00:13.590 --> 00:16.980 and that will return the action to play at each time team. 00:16.980 --> 00:18.120 So let's do this. 00:18.120 --> 00:21.810 So since we want our neural network to be an object 00:21.810 --> 00:23.190 we're gonna make a class, 00:23.190 --> 00:25.440 and that's because it's much more convenient. 00:25.440 --> 00:28.500 A class is the model of something we want to build. 00:28.500 --> 00:30.270 We want to build a neural network 00:30.270 --> 00:32.670 and we need to make some kind of instructions 00:32.670 --> 00:35.100 which will all be contained in a class. 00:35.100 --> 00:37.950 And in this class, we're gonna make two functions. 00:37.950 --> 00:39.870 First, the init function 00:39.870 --> 00:41.760 which is the function that comes up all the time 00:41.760 --> 00:42.780 when making a class, 00:42.780 --> 00:46.620 and that basically defines the variable of your object, 00:46.620 --> 00:48.120 that is the neural network, 00:48.120 --> 00:50.010 the variables attached to the object 00:50.010 --> 00:52.830 as opposed to the global variables. 00:52.830 --> 00:54.630 And so this is in this in it function 00:54.630 --> 00:57.390 that we'll define the architecture of the neural network. 00:57.390 --> 00:58.920 Defining the input layer, 00:58.920 --> 01:01.260 which will be composed of five input neurons 01:01.260 --> 01:03.060 because we have five dimensions 01:03.060 --> 01:05.850 for the encoded vector of input state. 01:05.850 --> 01:08.100 Then we will define some hidden layers, 01:08.100 --> 01:10.380 maybe we'll start with one hidden layer, 01:10.380 --> 01:11.370 and then you will be welcome 01:11.370 --> 01:14.160 to try some other architectures of the neural network. 01:14.160 --> 01:17.370 And then of course, we will end up with the output layer 01:17.370 --> 01:19.950 that will contain the possible actions 01:19.950 --> 01:22.170 that we can play at each time. 01:22.170 --> 01:24.840 So that's exactly what we'll do in this in it function. 01:24.840 --> 01:26.850 And then we will make another function 01:26.850 --> 01:28.260 still inside the class, 01:28.260 --> 01:30.180 which will be the forward function, 01:30.180 --> 01:32.880 and that will be the function that will activate 01:32.880 --> 01:34.860 the neurons in the neural network, 01:34.860 --> 01:37.440 that will activate the signals. 01:37.440 --> 01:40.200 And so we will use a rectify activation function 01:40.200 --> 01:41.790 because of course, we're dealing 01:41.790 --> 01:44.160 with a purely non-linear problem, 01:44.160 --> 01:47.160 and this rectified function breaks the linearity. 01:47.160 --> 01:49.620 But mostly we are making this forward function 01:49.620 --> 01:51.150 to return the Q-values 01:51.150 --> 01:53.520 which are the outputs of the neural network. 01:53.520 --> 01:56.160 But we have one Q-value for each action, 01:56.160 --> 01:59.160 and later on we will be returning the final action 01:59.160 --> 02:01.950 by either taking the max of the Q-values 02:01.950 --> 02:03.990 or using a softmax method. 02:03.990 --> 02:05.400 We will see that afterwards. 02:05.400 --> 02:07.350 So in this tutorial we're gonna start 02:07.350 --> 02:09.240 by implementing the init function, 02:09.240 --> 02:10.140 and then the next one 02:10.140 --> 02:12.840 we will be implementing the forward function. 02:12.840 --> 02:13.860 So let's do this. 02:13.860 --> 02:17.190 First we need to introduce our class. 02:17.190 --> 02:20.850 So we start with class, then we give a name to our class. 02:20.850 --> 02:23.640 Well, we can call it network. 02:23.640 --> 02:26.010 And then in this network class, 02:26.010 --> 02:28.590 I'm going to use an object programming technique 02:28.590 --> 02:30.300 which is called inheritance. 02:30.300 --> 02:32.040 And that is just to inherit 02:32.040 --> 02:35.190 from all the tools of a parent class. 02:35.190 --> 02:38.250 So our network class that we're about to make 02:38.250 --> 02:43.250 is a child class of a larger class which is an nn.Module. 02:44.430 --> 02:47.760 So that's just to inherit from all the tools 02:47.760 --> 02:50.220 of this module class which are of course the tools 02:50.220 --> 02:51.900 to implement a neural network. 02:51.900 --> 02:54.390 So that's a very powerful and efficient trick 02:54.390 --> 02:56.550 in object oriented programming. 02:56.550 --> 03:00.270 That's called inheritance and right now we are inheriting 03:00.270 --> 03:02.760 from this module parent class. 03:02.760 --> 03:06.120 All right, and now we're ready to go inside the class. 03:06.120 --> 03:08.520 So I'm pressing, enter, twice actually 03:08.520 --> 03:10.470 because we'll be making two functions. 03:10.470 --> 03:14.340 And we're starting with the init function. 03:14.340 --> 03:17.070 So the init function, we have to name it this way 03:17.070 --> 03:20.640 with two on course, then init, 03:20.640 --> 03:22.110 and then again, two underscores, 03:22.110 --> 03:24.180 that's just python syntax. 03:24.180 --> 03:25.890 That is just how we have to do it. 03:25.890 --> 03:28.680 And then we need to input the arguments. 03:28.680 --> 03:30.330 So we have three arguments. 03:30.330 --> 03:33.300 The first one is a compulsory argument 03:33.300 --> 03:35.010 that is actually self. 03:35.010 --> 03:37.410 And self, there is no mystery about it 03:37.410 --> 03:41.160 that refers to the object that will be created 03:41.160 --> 03:43.080 from this class that we're about to make. 03:43.080 --> 03:45.780 We're making this class, it's like some instructions, 03:45.780 --> 03:49.140 some model of this neural network we want to build. 03:49.140 --> 03:50.460 And then once the class is ready 03:50.460 --> 03:53.610 we can make as many neural networks as we want. 03:53.610 --> 03:55.350 And each of these neural networks 03:55.350 --> 03:57.810 will be some object of this class. 03:57.810 --> 04:02.190 And since we'll be using the object for some other purposes 04:02.190 --> 04:05.880 we need to spot what are the variables of the object. 04:05.880 --> 04:10.110 And to spot this, we are using this self here to specify 04:10.110 --> 04:12.060 that we're referring to the object. 04:12.060 --> 04:15.090 So whenever I want to use a variable from my object 04:15.090 --> 04:18.270 I will use self before the variable to specify 04:18.270 --> 04:20.463 that this is variable of the object. 04:21.330 --> 04:22.920 All right, so that's the first argument. 04:22.920 --> 04:24.510 And then we have two other arguments, 04:24.510 --> 04:27.180 which are of course the number of input neurons 04:27.180 --> 04:30.540 and the number of output neurons. 04:30.540 --> 04:31.890 So the number of input neurons, 04:31.890 --> 04:34.293 we're gonna call it input size. 04:35.640 --> 04:37.500 And that's actually five 04:37.500 --> 04:40.710 because our input vectors have five dimensions, 04:40.710 --> 04:45.120 the three signals, plus orientation, plus minus orientation 04:45.120 --> 04:48.150 that's our vectors of encoded values 04:48.150 --> 04:51.180 that describe one state of the environment. 04:51.180 --> 04:54.000 These five values are enough to describe a state 04:54.000 --> 04:54.960 of the environment. 04:54.960 --> 04:58.380 We could have thought of less values or more values 04:58.380 --> 04:59.550 but that's what I tried. 04:59.550 --> 05:00.600 And it actually makes sense 05:00.600 --> 05:03.360 because we actually need one signal from the left 05:03.360 --> 05:05.760 one in front of us, and one at the right. 05:05.760 --> 05:07.080 When we're driving a car. 05:07.080 --> 05:09.720 We could have gone for 360 signal, 05:09.720 --> 05:12.300 like the signals at the top of the Google cars 05:12.300 --> 05:15.990 but we can totally self drive with three centers. 05:15.990 --> 05:18.510 And then we have this orientation and minus orientation 05:18.510 --> 05:22.500 to keep track of the goal that we're trying to reach. 05:22.500 --> 05:24.780 And then we have, of course, 05:24.780 --> 05:27.150 the output neurons of our neural network, 05:27.150 --> 05:28.920 which correspond to the actions. 05:28.920 --> 05:30.750 And we have three possible actions, 05:30.750 --> 05:32.850 going left going straight, ordering right, 05:32.850 --> 05:36.450 and therefore I'm gonna call it nb-action. 05:36.450 --> 05:38.204 And there will be three of them. 05:38.204 --> 05:39.037 All right. 05:39.037 --> 05:41.580 But so far we only have to give names to the input 05:41.580 --> 05:43.500 and then we'll use these variables 05:43.500 --> 05:46.233 to do the computations inside the neural network. 05:47.070 --> 05:47.903 All right. 05:47.903 --> 05:52.380 Then I'm gonna start by using another pie torch trick. 05:52.380 --> 05:54.660 This trick is the super function. 05:54.660 --> 05:59.370 That's a function that actually inherits from the nn.Module. 05:59.370 --> 06:01.080 So that's why we had to use inheritance 06:01.080 --> 06:02.730 to inherit of the module tool. 06:02.730 --> 06:04.500 This is the first tool we use. 06:04.500 --> 06:07.590 And so basically we are only using this super trick, 06:07.590 --> 06:11.520 this super function, to be able to use the tools of module. 06:11.520 --> 06:13.650 So that's much more efficient. 06:13.650 --> 06:14.940 And inside this super function, 06:14.940 --> 06:18.630 I just need to specify first the network. 06:18.630 --> 06:21.360 So that's our network channel class 06:21.360 --> 06:24.780 because this is inheriting from the module parent class 06:24.780 --> 06:27.360 and then our object. 06:27.360 --> 06:29.580 And then I'm just adding dot 06:29.580 --> 06:33.000 and our init function like this, 06:33.000 --> 06:35.520 exactly how we named it. 06:35.520 --> 06:36.353 All right. 06:36.353 --> 06:37.186 So that's just a trick. 06:37.186 --> 06:40.650 That's just to use all the tools of nn.Module. 06:40.650 --> 06:43.050 Then we can move on to the next step, 06:43.050 --> 06:46.530 which is to specify the input layer. 06:46.530 --> 06:50.280 So basically what I have to do is introduce a new variable 06:50.280 --> 06:52.980 that will be attached to the object 06:52.980 --> 06:57.150 and this variable will contain the number of input neurons. 06:57.150 --> 07:00.420 So not to be confused with input size. 07:00.420 --> 07:05.160 Input size is the argument of the init function 07:05.160 --> 07:06.900 but that's not the variable that is attached 07:06.900 --> 07:08.220 to the object yet. 07:08.220 --> 07:10.500 The variable that is attached to the object, 07:10.500 --> 07:12.090 well, as I just mentioned, 07:12.090 --> 07:14.700 we need to specify that it is instead attached 07:14.700 --> 07:15.533 to the object. 07:15.533 --> 07:16.797 So we use a self., 07:18.180 --> 07:20.610 and now we give a name to this first variable 07:20.610 --> 07:22.140 attached to the object. 07:22.140 --> 07:24.870 And so we can simply give the same name as the input. 07:24.870 --> 07:27.330 We can call it input size, 07:27.330 --> 07:30.510 and we will say it is equal to the argument 07:30.510 --> 07:33.660 of the unit function that is input size. 07:33.660 --> 07:34.493 All right. 07:34.493 --> 07:37.110 So each time I'm creating an object from the network class 07:37.110 --> 07:39.150 and I'm specifying the input size. 07:39.150 --> 07:41.160 Like for example, I'm inputting five, 07:41.160 --> 07:42.570 there will be a five here 07:42.570 --> 07:44.432 and therefore the input size variable 07:44.432 --> 07:47.700 of our object will have the value of five 07:47.700 --> 07:50.430 because this input size here will be five, 07:50.430 --> 07:51.750 and therefore our neural network 07:51.750 --> 07:55.470 will have five input neurons in the input layer. 07:55.470 --> 07:56.303 All right. 07:56.303 --> 07:57.360 And then that's the same 07:57.360 --> 08:02.190 for the other variable that we want to attach to our object. 08:02.190 --> 08:03.600 And as you might have guessed, 08:03.600 --> 08:06.600 this is going to be a variable for the number 08:06.600 --> 08:08.310 of output neurons. 08:08.310 --> 08:11.520 And so same, we take our object self, 08:11.520 --> 08:12.510 and then we give a name 08:12.510 --> 08:14.507 to this second variable of the object. 08:14.507 --> 08:18.150 We are gonna call it nb-action. 08:18.150 --> 08:20.910 And this will be equal to this argument here 08:20.910 --> 08:22.140 giving the number of actions. 08:22.140 --> 08:24.510 That is the number of output neurons. 08:24.510 --> 08:28.380 And so we set it equal to nb-action. 08:28.380 --> 08:30.870 So actually nb-action will be equal to three. 08:30.870 --> 08:34.860 Therefore, the variable nb-action attached to our object, 08:34.860 --> 08:38.190 to our network, will get the value of three. 08:38.190 --> 08:39.810 Actually, we can see a warning here. 08:39.810 --> 08:42.450 It says undefined name nn. 08:42.450 --> 08:46.320 Well, that's because here we use the nn shortcut 08:46.320 --> 08:49.440 and we need to use the shortcut here, nn 08:49.440 --> 08:51.300 for our torch.nn.Module, 08:51.300 --> 08:52.830 and then it'll disappear. 08:52.830 --> 08:53.670 Here we go. 08:53.670 --> 08:54.570 Perfect. 08:54.570 --> 08:56.010 Right now we have have no warnings. 08:56.010 --> 08:59.100 All the warnings here are just to specify 08:59.100 --> 09:02.100 that what we import is not yet used, but that's okay. 09:02.100 --> 09:04.620 We'll be using them afterwards. 09:04.620 --> 09:05.453 All right. 09:05.453 --> 09:08.850 Then we have another two variables we want to define 09:08.850 --> 09:10.170 for object. 09:10.170 --> 09:12.630 And this will be the full connections, 09:12.630 --> 09:15.360 the full connections between the different layers 09:15.360 --> 09:16.770 of our neural network. 09:16.770 --> 09:19.470 So since right now we want to make a neural network composed 09:19.470 --> 09:21.150 of only one hidden layer 09:21.150 --> 09:23.550 where there will be two full connections. 09:23.550 --> 09:25.260 There will be one first full connection 09:25.260 --> 09:27.960 between the input layer and the hidden layer. 09:27.960 --> 09:29.460 And one second, full connection 09:29.460 --> 09:32.460 between the hidden layer and the output layer. 09:32.460 --> 09:34.860 So let's start with the first full connection. 09:34.860 --> 09:36.247 We're gonna call it fc1. 09:38.070 --> 09:40.440 And again, I use self here to specify 09:40.440 --> 09:43.830 that fc1 is the variable of my object. 09:43.830 --> 09:47.610 So self.fc1, which will be equal to, 09:47.610 --> 09:50.880 and now we use the nn.Module, 09:50.880 --> 09:54.300 and we're gonna use a function called linear. 09:54.300 --> 09:57.270 And that's exactly to make this full connection 09:57.270 --> 09:59.790 between the neurons of the input layer 09:59.790 --> 10:02.130 to the neurons of the hidden layer. 10:02.130 --> 10:04.140 And what do I mean by full connection? 10:04.140 --> 10:06.600 That means that all the neurons of the input layer 10:06.600 --> 10:08.790 will all be connected to all the neurons 10:08.790 --> 10:10.170 of the hidden layer. 10:10.170 --> 10:11.520 And so to make this full connection, 10:11.520 --> 10:13.710 we use this linear function 10:13.710 --> 10:16.170 to which we need to input some arguments. 10:16.170 --> 10:19.860 And as you can see, these arguments are in_features. 10:19.860 --> 10:21.990 So that is the number of neurons 10:21.990 --> 10:24.060 of the first layer we want to connect. 10:24.060 --> 10:26.310 Then the out_features that is the number of neurons 10:26.310 --> 10:28.500 of the second layer we want to connect. 10:28.500 --> 10:31.110 That is the layer at the right, that is the hidden layer. 10:31.110 --> 10:32.460 And bias equal true. 10:32.460 --> 10:35.280 So bias equal true, we will keep the default value, 10:35.280 --> 10:37.860 that's in order to have a bias 10:37.860 --> 10:39.900 and not only some weights attached to the neurons. 10:39.900 --> 10:43.560 We'll have the weights and one bias for each layer. 10:43.560 --> 10:46.230 And so, well, let's see what we need to input. 10:46.230 --> 10:49.260 So the first argument in_features is the number 10:49.260 --> 10:51.990 of input neurons in the input layer. 10:51.990 --> 10:53.040 And so what is it? 10:53.040 --> 10:55.080 Well, that's actually input size. 10:55.080 --> 10:57.510 That's the argument of our init function. 10:57.510 --> 11:00.210 Which layer will be equal to five? 11:00.210 --> 11:04.140 The three signals, orientation and minus orientation. 11:04.140 --> 11:05.130 So here we go. 11:05.130 --> 11:06.780 We input the first argument, 11:06.780 --> 11:08.883 input size. 11:09.870 --> 11:12.990 And then the second argument is out_features. 11:12.990 --> 11:15.720 That is the number of neurons we want to have 11:15.720 --> 11:17.130 in the second layer. 11:17.130 --> 11:18.990 The second layer that will be fully connected 11:18.990 --> 11:20.430 to the first layer. 11:20.430 --> 11:21.750 And so now the question is 11:21.750 --> 11:25.200 how many neurons do we want in this hidden layer? 11:25.200 --> 11:27.420 Well, I did a lot of parameter tuning. 11:27.420 --> 11:29.160 I did a lot of experimenting. 11:29.160 --> 11:30.630 That's what we do in AI, 11:30.630 --> 11:32.820 or that's what we do in deep learning in general. 11:32.820 --> 11:35.010 We do a lot of experimenting to see 11:35.010 --> 11:37.110 what would be the best neural network 11:37.110 --> 11:39.150 for our specific problem. 11:39.150 --> 11:40.890 And so I tried many values 11:40.890 --> 11:45.150 and I ended up choosing 30, 30 neurons in a hidden layer. 11:45.150 --> 11:46.740 And you will see that with this number, 11:46.740 --> 11:48.300 we will get some pretty good results. 11:48.300 --> 11:50.790 But then feel free to change the architecture 11:50.790 --> 11:51.623 of the neural network. 11:51.623 --> 11:53.160 Feel free to play with it. 11:53.160 --> 11:55.410 You can not only change the number of hidden neurons 11:55.410 --> 11:59.160 in the hidden layer, but also you can add some more layers 11:59.160 --> 12:01.320 so that maybe you get an even better car. 12:01.320 --> 12:04.860 But 30 hidden neurons will get us a good neural network 12:04.860 --> 12:06.000 and a good car. 12:06.000 --> 12:07.500 So that's what we go for. 12:07.500 --> 12:08.430 And there we go. 12:08.430 --> 12:11.430 We have our first full connection ready. 12:11.430 --> 12:14.640 With this linear function, we make the full connection 12:14.640 --> 12:17.310 between the input layer and the hidden layer. 12:17.310 --> 12:21.270 And now time to make the second full connection, 12:21.270 --> 12:22.230 that is the full connection 12:22.230 --> 12:25.470 between the hidden layer and the output layer. 12:25.470 --> 12:26.700 So there we go. 12:26.700 --> 12:31.470 We're gonna call this second full connection fc2. 12:31.470 --> 12:32.340 There we go. 12:32.340 --> 12:34.980 And still, this is a variable from our object. 12:34.980 --> 12:36.630 So I'm using self here. 12:36.630 --> 12:38.430 And then again, we use, 12:38.430 --> 12:40.620 well, actually we can copy this 12:40.620 --> 12:42.810 because we're gonna use the nn.Module 12:42.810 --> 12:45.540 and then the linear function. 12:45.540 --> 12:48.360 But then we need to change the arguments of course. 12:48.360 --> 12:49.890 First, that's the same. 12:49.890 --> 12:52.920 First is the number of neurons we want to have 12:52.920 --> 12:55.170 in the first layer of the full connection. 12:55.170 --> 12:59.610 So that's the hidden layer, and therefore that is 30. 12:59.610 --> 13:01.620 And then second argument is the number 13:01.620 --> 13:04.830 of neurons in the second layer of the full connection. 13:04.830 --> 13:07.200 And that corresponds to the output layer. 13:07.200 --> 13:10.710 And the output layer has nb-action neurons, 13:10.710 --> 13:12.030 which later will be three 13:12.030 --> 13:14.100 because we have three possible actions. 13:14.100 --> 13:17.040 But so far we have to use the names we defined. 13:17.040 --> 13:19.830 That is the names of the argument of the init function 13:19.830 --> 13:23.790 and therefore we input here nb-action. 13:23.790 --> 13:24.900 And there we go. 13:24.900 --> 13:27.870 First of all, our two full connections is ready. 13:27.870 --> 13:31.380 And second of all, our init function is ready. 13:31.380 --> 13:34.380 So that's what will initialize our object 13:34.380 --> 13:37.110 whenever we create an object from the network class. 13:37.110 --> 13:39.360 And so as soon as we create an object, 13:39.360 --> 13:42.390 well, all these variables, the four variables here, 13:42.390 --> 13:47.160 input size, nb-action, fc1 and fc2 will be defined. 13:47.160 --> 13:49.260 And that's how we'll get the architecture 13:49.260 --> 13:52.140 of our neural network for each object that we create. 13:52.140 --> 13:54.960 Each object will correspond to neural network 13:54.960 --> 13:57.840 of five input neurons, 30 hidden neurons, 13:57.840 --> 14:00.450 and three output neurons. 14:00.450 --> 14:01.410 So there we go. 14:01.410 --> 14:03.630 We are done with this first init function, 14:03.630 --> 14:06.090 and now we can move on to the second function, 14:06.090 --> 14:07.830 which is the forward function. 14:07.830 --> 14:10.200 And that will be used to activate the neurons 14:10.200 --> 14:14.040 in the neural network using rectifier activation function 14:14.040 --> 14:17.130 and mostly to eventually return the Q-values 14:17.130 --> 14:19.560 which are the outputs of our neural network. 14:19.560 --> 14:21.810 So I can't wait to do this in the next tutorial. 14:21.810 --> 14:23.673 And until then, enjoy AI.