At this stage.
We now have the video IDs.
What we need is a function that relates these video IDs to the final seven variables of interest, which
we have discussed already, and that you see here.
From the Russian analogy.
We know that we will use the videos resource to get the video data.
And if we had to go to the API explorer to get the seven variables of interest, we'll need to use the
content details, statistics and snippet for the path parameter that you see here.
The video IDs we found in the previous lecture.
We can specify here in this ID folder.
Here I am just defining one video, but looking at the documentation, it states that we can define
it as a comma separated list of video IDs with data type string.
An important note on this is that for this concatenated string of video IDs, same as for the maximum
results in a page, we can only have a maximum of 50 IDs in this video ID string.
So knowing this, we can start by defining a helper function to split the video ID list into batches.
Let's define it right now in VS code.
Let's go down here and let's call it batch list which takes as input the video IDs list and also the
batch size which we will set it equal to the max results when we come to implement it inside the function.
So the batch list video ID list and also takes batch size.
From here we just define a for loop that will loop through the list and batches of size batch size.
And we each time want to eat the batched video list.
So we define a for loop for video ID in the range going from zero to the length of the whole video ID
list, which we found in the previous function, taking also into consideration the batch size.
We will also yield the video idealists.
In batches.
So starting from video ID all the way to the batch size length.
Note how here we write here, then not return.
If we specify return, the loop will stop in the first iteration, which is not what we want.
So now that we have defined this function, let's momentarily go back to the API explorer so that I
can show you the Response from what we have defined already.
Again, here I will use just one video ID.
Let's go down and we untick the old tool and press execute.
And the response we get the variables of interest.
These being the snippet.
This is the snippet key.
And we want the published at.
And the title of the video going down to the content details which is this key over here.
From here we want the duration of the video.
And if we had to scroll further down we also want the statistics.
And for this we know that we want the views, the likes and the comment count.
Now that we know what we are after, let's copy the URL from the API explorer and paste it into our
script.
And we take this back to VS code.
Let's just paste it here for now.
We will obviously have to do some changes to this URL.
But for now we can leave it as it is.
Moving on.
Let's now define the main function by giving it a meaningful name like extract.
Video data, which takes the video ID list from the previous function as argument.
So we take the argument as video IDs.
We can also define an empty list.
Let's call it extract the data that will store the seven variables we are after.
For all the videos that are in Mr. Beast's channel extracted data, and we define it as an empty list.
Before we move on, let's try and give a bit more structure to this main function.
Let's put the helper function inside.
Let's put it below the extracted data definition.
And we will also use the URL.
Somewhere here.
Now that we have modified the structure a bit, let's encapsulate the code in a try and accept clause.
So we do try Same as before.
Accept requests dot exceptions dot request exception as a and reraise the error should there be one.
The next step is to loop through the whole list in batches at each batch, defining the concatenated
video ID string using the Python join method, which we will then be use to build the URL.
So let's do that right now and define a for loop for batch in batch lists, which we have defined already.
This will use the video IDs and also the max results.
Now we join the video IDs into a comma separated string as per the docs.
Comma separated using the join method in Python.
Now at this point we need the URL.
So let's define the URL here and paste it here.
Obviously we also need to do some cleaning.
So in this case the part stays as is.
But the ID we need to change.
This will be the video ID string.
Here we also need to define an f string.
Put this here.
And same as always we also change the API key.
Same as we have done for the previous functions.
We then get the response.
Check for errors and if there are no errors, get the response data in JSON format.
So let's copy this from the previous functions.
So this.
We don't need it anymore.
The batch list since we are defining it in the main function.
Now similar to what we have done in the previous function, we can also use the get method to get the
values under the main items key for video ID snippets, content details and statistics, with the empty
list as a value to return if the specified key does not exist.
Let's write this right now.
So for item in data dot, get get from the items.
If there is no key that we are after, we define an empty list.
That would be the default.
And also we start defining for video ID we will take it from the ID key snippet will be the snippets
key.
Content details will be in the content details key.
And finally this stakes.
Will be in the statistics game.
From here, it's only a matter of creating a dictionary that will contain all the data we are interested
in.
Let's write that right now.
So we define the dictionary.
Let's call it video data.
We can define a dictionary that will take the variables from this loop.
So we can define the video id.
Which will write the key's first title published at.
Duration.
View count.
And just to be sure that we are all on the same page.
These seven variables in this dictionary are the final data we want for each Mr. Beast YouTube video.
So let me complete this dictionary.
So as you are seeing for statistics variables, we are using the Get methods as.
While I was testing, I found that there are some instances where not all the videos have views, likes
and comments visible.
So we need to take this into account in our case.
And that is what we are saying for these three variables.
If for a specific video, either the views, likes or comments counts are not available, then replace
this value with none.
So finally we just append the data to the extracted data variable as we had defined at the beginning
and return it.
So extract the data dot append.
Getting the video data which we just defined here.
And we simply return the final variable.
One very important thing to notice and I was about to make this mistake is the indentation of our code.
Here you see that the video data dictionary is outside our for loop like this.
We only get data for the first 50 videos.
So to get the data for all the videos, we need to indent the video data dictionary and the appending
to the main variable inside the for loop.
So we do that like this.
Now the function is complete.
If we run this function, once we loop through all the videos we will get a large dictionary.
Let's showcase this by going in the double underscore name equals double underscore main part of the
script and make some changes.
So let's first make a change of defining the variable for video IDs function.
And from here we can call the function that we just defined.
And as an input.
Getting the video IDs from the previous function.
For purpose of the demonstration, I will also include a print statement to showcase the outputs of
the run in the terminal.
Let's press run.
This might take some time since we are looping through a number of videos.
And there you have it.
This is the video data we will be populating in our data warehouse.
Same as I have done in the previous lecture.
I will remove the print, save and commit this change to GitHub and I will see you in the final part
of this four part lecture series.