In the last lesson,
we got started using pandas.
We installed the library,
and we imported it to read a CSV file.
And then, we used that file to get hold of a column,
which has been automatically identified as soon as it read
this data.csv.
So, what exactly are we dealing with here?
Well, one of the most useful things I find
is to do type checks on any of the objects
that I'm working with from a new novel library.
For example, we can use the type method to check
what exactly is the data type() of this data
that we're getting back from pandas.
So if I go ahead and print this one, and comment out
this second line, then you can see what we're getting
is what's called a pandas DataFrame object.
In the Package overview, they talk about
the two primary data structures of pandas.
Series and a DataFrame.
A DataFrame is kind of
the equivalent of your whole table here.
So, every single sheet inside an Excel file
or inside a Google Sheet file
would be considered a DataFrame in pandas.
Now, what about the second part here,
where we've gotten hold of one of the columns
in our DataFrame?
If I do a type check on this object, and I hit Run,
then you can see, this is a pandas Series object.
The Series is the other super important concept in pandas.
And the Series is basically equivalent to a list.
It's kind of like a single column in your table.
So, the temperature column would be a Series,
the condition column would be a Series,
and the day column would also be a Series.
So, once you've grokked this idea
that the whole table is basically a DataFrame in pandas,
and every single column is a Series,
kind of like a list in pandas,
then you're pretty much half of the way there
to understanding how this library works.
Now if we go over to the API reference,
you can see that
this is basically a list of all of the things
that you can do with pandas.
And it is a long list of things.
But let's take a look at those two core classes,
the DataFrame and the Series.
So if we go to the DataFrame, you can see
that it has things on how to construct a new DataFrame,
how to get hold of the index,
how to get hold of the column labels,
and there's a whole bunch of attributes
that you can tap into as well as many methods.
For example, if we take a look at this section
on Serialization / IO / conversion,
you can see that you can actually convert a DataFrame
to various different file types.
You can convert it to an Excel file,
you can convert it to HTML.
You can also convert it to a dictionary.
So if we click on this,
this takes us to the actual documentation
on how you would use this method.
And if you look at the basic Python documentation
versus panda's documentation,
you'll see why this is so much better.
It lists out all of the possible parameters.
It gives you examples on how you can use
each of the methods, and it's also got some related methods
that it thinks that you might want to take a look at.
So, let's use this method.
And what we're going to do is, I'm going to get my data,
and I'm going to call to_dict,
which is going to convert it into a dictionary.
So, let's call that data_dict = data.to_dict().
And then if we print out our new data_dictionary,
you can see that pandas has taken our table,
and taken each column of the table
to create a separate dictionary for each of the columns.
So, we've got day, temperature, and condition,
and we can now work with this
as if it were a real dictionary.
Now, if we take a look at the Series data type.
And you take a look at the Conversion section
for this type of data,
then you can see that you can actually convert
each of the Series to a list, if you want to be able
to work with it just as you would with any other list.
So, we saw that we can get our data,
and then get the temperature column.
And this, when we printed out the type, was a Series.
So, we can get the Series, and then call .to_list().
And this will turn this data Series into a Python list.
So, let's call that temp_list.
And let's go ahead and print it out.
This would be a list of all the temperatures,
and this is now converted into the raw Python data type.
So, we can do all of the things
that we can do to a Python list,
like for example, we could check its length.
So, here comes another challenge for you.
Given what we've spoken about so far,
can you figure out if you can work out
the average temperature in our column of temperatures?
So remember, you can always Google if you don't remember
how to calculate the average of something.
So, pause the video and give that a go.
All right, so we know that we can get
a list of all of the temperatures in that list,
and we know that Python has a built-in method called sum().
So, then we can get the sum of all of the temperatures
in our list of temperatures,
and then we can divide it by the length of the temp_list.
The sum of all the values divided
by the number of values gives us the average,
which is also known as the mean.
And so if we print this out, then you can see,
the average temperature of the week was 17.4.
Now, an alternative way of solving this challenge is,
maybe you took a look through this list of methods,
and you might have come across some of the computations
and statistics that you can do with your Series in pandas.
Now, one of those methods is the mean.
So, you can actually get rid of all of this excess work,
and take our data Series,
which is basically the column of data
under the heading temp, and simply call .mean on it.
And now if I print this out, you'll see
it's the same result as before
without a lot of the extra work.
In addition to the mean, you can get other things
like the median or the mode,
and a whole bunch of other things just by calling
the right method on the data Series.
So, here's a challenge for you.
I want you to get hold of the maximum value
from this column of temperatures
by using one of the data Series methods.
Pause the video, have a look at the documentation,
and see if you can complete this challenge.
All right, to get the maximum value,
we're probably going to need this max method.
So, we call this method in the same way as we did before,
which is get hold of the data Series.
So, our entire table is stored in data,
and then we can get the column
under the heading temperature.
So, this is now a data Series,
and then we can call that method max on it.
And if we print it out,
then you can see what we're getting is 24.
So, the highest temperature was 24.
So, you've seen that when we're working with pandas,
it's really easy to get hold of data in a particular column,
all we have to do is just take the DataFrame,
use a set of square brackets,
and then specify the name of the column,
which it takes by default as the first row of the data.
So, the day column, the temp, the condition.
So if I want to get hold of all of the conditions,
then I would say data["condition"].
And if I go ahead and print this out,
you can see it gets hold of all of the weather conditions,
and selects that column to print out.
Now, an alternative way to using
the square bracket notation, where you have to be careful
about the string you use here,
it has to match, the name of the column exactly.
Another way that you can work with the columns
is simply by calling data.condition.
And the fact that this code is valid at all means that
pandas behind the scenes, has taken each of these columns
and each of the headings,
and converted those headings into attributes.
So, we can say data.condition or data.day.
And if I print that out, you can see
it's actually going to be
exactly the same as doing it like this.
So, it's up to you which method you want to use
to select the columns.
But be aware that if your column name has a capital C,
for example here, then your key has to be a capital C.
And also, your attribute has to be a capital C.
So effectively, when you're using a DataFrame like this,
it's almost like you're treating it as a dictionary.
And you're pulling out each column by the key.
Now, when you are using the DataFrame like this,
then you're kind of treating it more like an object.
You're saying data.attribute,
and you get hold of the data in that column.
So, I'm going to restore everything to lowercase
because I find it easier to read the code.
But the next thing I want to show you is a little bit harder,
which is how do you get data,
which are in the rows of our DataFrame.
If I wanted to get hold of the entire row of data
for where the day is equal to Monday,
then the way that I would do that in pandas, is firstly,
get a hold of my entire data table,
and then, inside that data table, get hold of the column
that I want to search through.
So, I'm going to search through the day column,
so I can use data.day or data["day"],
both will work the same.
But once I've got the column, then I can say, well,
where inside that column, I want to check
for the row where the value is equal to Monday.
This is basically going to return my row that I want.
So, I'm going to print this out, and I'm going to
comment out all of the previous code other than the place
where we created our DataFrame.
And then, I'm going to run my code.
And you can see it's pulled out that correct row
where the day is equal to Monday.
And it's given me all of the rest of the data for that row.
So, here's a challenge for you.
Can you figure out how to pull out the row of data
from our weather data where the temperature
was at the maximum?
So, which row of data had the highest temperature
in the week?
Pause the video and give that a go.
So, we know that we can get the maximum temperature
in the temperature column just by using this code.
Now, you can either use this method
where you say data["temp"],
or you can use the attribute .temp,
which is the code that I prefer.
I don't like writing a lot of strings if I can avoid it.
In this case, we're checking to see which row inside
our column of temperatures
is equal to the maximum temperature.
We would say data.temp == data.temp.max().
And then, we're going to get our data,
and access the row that fits that criteria.
Now if I print out this row,
then you can see it was the row for Sunday,
where the temperature was 24 and the condition was sunny.
Essentially, when we get our DataFrame
and then we use some square brackets,
and inside those square brackets, if we only put
the name of our column, day, temp, or condition,
then we would get the entire column.
But if we filter that column by a condition,
say when a particular column is equal to a particular value,
then we actually get hold of the row instead.
Now, once you've gotten hold of the data in the row,
you can actually go one step further.
Because we know that the row contains lots of data, right?
What if we wanted that particular row's temperature
or that particular row's condition?
Well, let's say that we create a variable
called monday, which is equal to our DataFrame,
and then searching through that DataFrame,
where the [data.day == "Monday"].
So, now with this row, monday,
we can then tap into the values under different columns
by using the same kind of way that we got data
in the entire column over here.
So, we can say monday.condition.
And if I print this out and just comment these other bits,
then you can see that I get the actual condition
for that particular day, which happens to be Sunny.
Now, here's a challenge.
I want you to get Monday's temperature,
but because my temperatures are in Celsius,
I want you to convert it into Fahrenheit.
So, pause the video,
and see if you can complete that challenge.
All right, we know we can get hold of Monday's temperature
by saying monday...and then the name of that column
in the row, which happens to be temp, T-E-M-P.
And then, we get the first value in the series
at index zero.
And then if we want to convert Celsius to Fahrenheit,
all we have to do is to multiply the Celsius
by 9 over 5, and then add 32.
So, it'll be Monday temperature
multiplied by 9 divided by 5,
and then add 32.
So, monday_temp_F, and then we can print this value out.
So now when I run it, we get 53.6.
And if I put Monday's temperature, 12,
into this Google converter, I get the same value, 53.6.
Now, the final thing I want to show you is how you create
a DataFrame from scratch.
So, in our case, we created our DataFrame
by reading from our CSV file,
but what if you wanted to create a DataFrame
just from some data that you're generating in Python?
Let's say that I have this dictionary of values,
I've got some students, and these are their names
held in a list, and then each of them has a score.
And the scores correspond to the students,
so 76 is Amy's score, James scored 56,
and Angela scored 65.
Now, how would we create a DataFrame from this dictionary?
It's really simple.
We call our pandas library,
and we get hold of the DataFrame class,
and then we initialize that class with some data.
And the data, in our case, is just going to be
our data dictionary.
And now, if I go ahead and save this as our data,
so I'm going to comment out what we had previously,
and then print out this data,
then you can see, I've now created a table
using the values from that dictionary.
Now, we can go even further than this.
When we've created a DataFrame, we can actually get
that DataFrame to be converted to a CSV file.
And this to_csv() method takes only one required input,
which is the path that you want to save this file.
So, let's just create a new file,
which we'll call ("new_data.csv").
And when I run this code, then watch over here,
you'll see a new CSV file being created from thin air,
and all of our data has been added to that CSV file.
So, we're just getting a glimpse
into how powerful this panda library can be.
And we're only really using it to read CSV data
and write CSV data.
Which is a common format
that you'll see being manipulated using Python.
In later lessons,
we're going to dive deeper into data analysis,
and we're going to be looking not only at pandas,
but we're also going to be looking at NumPy,
Matplotlib, and other libraries that make it easier for us
to work with large chunks of data.
In the next lesson,
we're going to be putting what we've learned to use
by analyzing some squirrel data from Central Park.
So for all of that and more, I'll see on the next lesson.