Welcome everyone to this section of the course on visualizing data.
Visualizing data is a key aspect of data science.
It's very important to be able to convey the information that you've worked on or have studied to others,
especially people who may not actually have your full technical knowledge or understanding of how to
view the raw data or perform a statistical analysis.
You should think of data visualizations as bridging a gap between the practitioners of data science
and key decision makers or leadership.
As we continue learning about data visualization throughout this section, you should always keep in
mind what is the information that I want to share or the story I am trying to tell?
And how does this specific visualization help in conveying that information to others?
You should think of data visualization as just another form of communication.
Now, data visualization is especially crucial in organizations because often decisions are made based
on final visualization or interpretation of the data.
Keep in mind that typically when discussing data science, we're using that to actually make a decision
or perform a specific action.
The end result is typically not a data visualization, but the actual decision made using that data
visualization.
And again, remember that the purpose of data science at an enterprise level, again, is to use it
to make key decisions and improve products or services.
Sometimes beginners get confused and think that data visualization itself is the final end product,
but that's typically not going to be the case.
It's a tool for communicating what decisions should be made.
So again, think of data visualization as an interface between those who work directly with data sources
and those who make higher level strategic decisions in a business.
A key part of data visualization is understanding what type of plot, chart or visualization to use,
which is the purpose of this section of the course.
Keep in mind, you should also be careful with visualizations.
Often information doesn't actually need a pretty visual to be useful.
Sometimes you really just need to report simple metrics like the average or mean, and in that case
the number itself is probably useful enough and you don't need to make some sort of beautiful visualization.
Remember, we're not really artists here.
We're not trying to make the prettiest visualization of all time.
We're trying to make the clearest communication factor possible.
To understand which data visualization to use.
Let's have a quick tour of the different data visualizations categories we're going to be covering in
this section, including scatter plots, line plots, distribution plots and categorical plots.
For each of these plot types.
In this introduction lecture, we're going to slowly construct the plot step by step, and we're also
briefly going to mention some more advanced variations for certain plot types.
So let's begin with the scatterplot, which is very likely a plot or visualization that you've seen
before.
A typical scatterplot is going to represent two dimensions, essentially two features of your data.
For example, the height versus weight of a group of people.
So we have two dimensions height and weight or two features, height and weight.
Now, a scatterplot can actually reveal relationships between two data features.
So think of that as the key indicator of using a scatterplot when you want to reveal the relationship
visually between two features.
So imagine the following data set.
Here we have what's known as the tips data set.
We see a total bill.
A tip, the sex of the client that actually visited the restaurant, whether or not they were a smoker,
the day they visited, the time, whether it was lunch or dinner and the overall party size.
So we have lots of features in this data set to work with.
Now, you may be wondering, for example, is there a relationship between the size of the tip left
and the total bill?
So now we're going to analyze two specific features, and we can do this visually with a scatterplot.
So we end up graphing along one axis.
A feature such as the x axis can be total bill, and then we add in the other feature along the y axis,
such as the tip.
And then for every data point, you actually start plotting them as a dot.
And keep in mind you can use other symbols besides a circular dot.
So, for example, we have one point with a total bill of a little less than $10 and a tip size that
was around $5.
Then we keep going through the data set marking down the points of total bill versus tip.
And you do this for the entire data set.
And here we can now visually see a general trend that as your total bill increases, typically your
tip is also going to increase, which intuitively makes sense.
But obviously there's going to be data features where the relationship is unclear and it's a lot easier
to digest, so to speak, in a visual format.
So again, the tip tends to increase as the total bill increases.
And that's the sort of story you can tell with a scatterplot.
Keep in mind that the relationship may also show something like the inverse, where one feature tends
to decrease as another feature increases.
So what are the considerations for scatter plots?
Well, you could add a trend line, so you could actually use a simple linear regression model that
we're going to learn about later on in this course.
And you could plot that on top of the scatter points to really set home the story that there is some
sort of relationship here between these two features.
You can also add transparency for stacked scatter points.
So take a look at the area inside the circle.
You'll notice that right now there's actually a lot of dots that are stacked right on top of each other.
And for really large data sets, it may be unclear how many dots are actually stacked on top.
So what you can do is add a little bit of transparency for those dots.
That means dots that are stacked on top of each other will appear darker than a singular data point
or dot.
So here we're now adding some transparency, which allows you to see where the data points are actually
stacked on top of each other.
So again, if you have a lot of data points, use transparency to indicate where your data is clustered.
You could also add more variables.
So previously we showed you using scatter plots for two features that is total bill and tip.
But remember, we also had the sex of the person visiting the restaurant.
So in that case, we can add another feature using something like color, such as orange for male and
blue for female.
And now you can add in even more features into your visualization.
Remember, you should only do this if that's important to the final decision being made.
And you can do this again with shapes.
So now you can see we're actually putting in a lot of information visually into this plot.
We're not just plotting out total bill versus tip, but we're also showing the sex of the person through
color and then using shape to indicate whether or not they were a smoker.
So notice we have smoker.
Yes.
As a circle and a cross or X for non smoker.
Keep in mind, this can get very busy very quickly.
So you typically want to try to show the minimum amount of features in order to get your story across.
You shouldn't just add features for the sake of it.
Now let's move on to line plots.
Sometimes we already know there should be a continuous relationship between points along an axes.
This is where we can use a simple line to indicate a known relationship between points along a data
feature.
So for example, in our previous example of using the total bill versus tips, it does not make sense
to draw a line between the points because remember, the actual visitors to the restaurant are all separate
from each other.
There's no relationship between a person that showed up on Sunday for dinner versus another person that
showed up on Saturday for lunch.
You shouldn't be drawing a line between those two dots.
So again, here, the line connecting total bills of separate parties doesn't actually represent anything.
So a line plot would be incorrect on charting total bill versus tip.
So here a line is misleading and this is an example of when you should not use a line plot because it's
trying to indicate some known relationship between each party.
Bill, You should only be using line plots when you for sure know that there is a relationship before
you actually plotted out the data.
So a trend line would have been the correct way to address a need to show a linear fit.
And we saw the trend line earlier.
That is the trend line on top of your scatterplot.
This is not what we mean with a line plot.
A line plot is going to assume prior knowledge of some sort of relationship in the data.
Here we are not assuming some prior knowledge of a relationship between total bill and tip.
It's only after we plotted the points that we decide to add a trend line.
This is, again, not the line plot that we're talking about.
So it is the line plot that we're discussing here.
And when is the line appropriate?
When we already know for certain that there is some sort of continuous relationship between data points
along a feature.
A very common example is timestamps.
So we know for certain there are continuous times in between each timestamp to point.
So imagine that you have the following data set.
You have a year, a month, and then the passengers that happen to fly may be in the thousands for that
particular month of that particular year.
So the way to read this is that in January of 1949, you had 112,000 passengers fly on an airplane.
And then in February of 1949, you had 118,000 passengers fly on an airplane and so on and so on.
Notice that we already know the way times and dates work.
So I know there's a relationship between January to February, to March, to April, to May, etc.,
throughout the years.
So it makes sense to draw a line between the actual time stamps, since I already know that's a continuous
linear relationship.
So we know flights occurred between years and if you wanted to, you could just add a dot there for
the passengers total per year or per month or over the entire time series.
But as I mentioned, you know that flights occurred throughout the days between those months.
And, you know, there's a linear relationship or continuous relationship as you move along the time
axes, in which case you can use a line plot.
The line indicates that you have that continuous knowledge that, you know, time moves forward and
you know that there is some sort of relationship time wise along an axis between the data point in January
to February, to March to April and so on.
So this indicates that, you know, that there is some sort of continuous relationship or knowledge
for a particular data feature and that there is a linkage between the separate data points, the link
here being that they're continuous on some sort of time series.
A line plot also makes it easy to stack features.
So if you want it to, you could separate out all those data points by month and then you can actually
stack them via different colors.
So you can see over the years how many travelers were there in January versus June or July versus November
and December.
So if we showed you this plot and you quickly did an analysis, what's the story being told here?
Well, you'll notice it looks like these cooler colors, like the greens and the blues that are actually
related to the summer months tend to be higher than the rest of the months.
And that makes sense because people would tend to travel more during the summer.
And we can now see a lime plot indicating a story here that you expect higher travel during the summer
months regardless of the year.
That was always the case from 1950 all the way to 1960.
And the overall trend is that passengers keep growing over year over year.
So we're telling actually two stories here.
The passengers are growing year over year, but also summer months are always popular regardless of
the year you're looking at.
Now let's discuss distribution plots.
Recall our discussions on measurements of dispersion of data such as variance or standard deviation.
Distribution plots allow us to visualize the dispersion of data across a feature or variable.
One of the most common ways to do this is through a plot known as a histogram.
Let's revisit our tips data set.
So let's imagine we wanted to know what is the distribution of total bill amounts?
Is everything super wide ranging from $1 to 1000?
Or do most bills tend to have the same amount somewhere between ten and $20?
How could we actually answer this?
Well, you could just report back a standard deviation or variance along that feature, but that doesn't
really tell you the whole story because then you may not need to include the mean.
So what is the visualization we could use to show the distribution of a feature?
And again, I could answer this through some statistical metrics, like telling you the mean total bill
is $18.79.
If the standard deviation is $8.09 or $0.90.
Minimum values.
Maximum values.
But this is sometimes a little confusing and it's hard to actually internalize.
So instead of using this information, let's try a visualization.
So a histogram is going to count the number of occurrences of data points within a range along the x
axis.
Then it will create a bar of the height of the count of the occurrences within that range in the x axis
where the height moves along the y axis.
You should always know that the x axis feature is continuous and this has to be the case for a histogram.
Again for a distribution plot.
The x axis feature should be continuous.
So we end up getting a distribution or histogram that looks like this.
So we have the total bill on the x axis.
And remember, total bill is continuous because you go from $20 to $21.22 and then there's also cents
in between those.
So $20.01, $20.02 etc., etc..
And you can also change the bin size in order to get a different visualization around the distribution.
It depends on how granular you want to be on the actual width of the bin for the x axis feature.
Notice that the y axis here is just a count of the number of occurrences that fell between that particular
width of the bin.
So if we were to take a specific look at one of these, for example, this highest bar, we're taking
a look at dollar total bill values that are somewhere between, I would say $12 and maybe $16.
So again, take a look at the x axis width here of this actual bin.
So we're taking a look at some of those values in between ten and 20.
It doesn't go from 10 to 20.
It goes from somewhere in between that.
So maybe somewhere around $12, so somewhere around 16 or $17.
And then we count how many total bill instances were between those two values.
And then it ends up being something like a little less than 70, and you're doing that for all the bins.
And that's how a histogram works.
And again, remember, you can change that bin size to try to get a more granular feel on your data.
So here we have wider bin sizes, which means we're going to have less vertical stacking, so to speak,
but you could diminish that bin size.
So look at smaller pieces along the x axis and gets higher granular visualization.
Again, it's still showing the same data, but you're being a little more specific on the ranges along
those bins and it's up to you to decide what's the best bin size to choose and what's the story you're
trying to tell with the distribution plot.
Now, keep in mind, there's actually many more types of plots that can display distribution.
There is a box and whisker plot, and there's also a CD known as a kernel density estimation plot.
By far the most common you're probably going to encounter, though, is the histogram.
Now let's talk about categorical plots.
Categorical plots simply display some metric per category.
For example, you can have a mean value per category or a count per category.
There are many variations of these types of plots, but one of the most common is the simple bar plot.
Be careful not to confuse the bar plot for the histogram we just saw earlier.
They may appear similar, but you should carefully look at the x axis feature for a bar chart or bar
plot that is categorical.
Then the feature along that x axis should be categorical.
Recall for the histogram, it was a continuous feature.
So again, taking a look back at this tips data set that we've been playing around with, maybe we're
trying to answer a question like what is the total expenditure per day?
So the sum of the total bill per day, per sundae, per Monday, per Tuesday, etc..
So for example, it was a total amount of revenue or expenditure per each category of day.
Notice.
Day is not really continuous in the same way because if we're thinking in terms of weekdays, there's
not really a weekday between Monday and Tuesday or Tuesday and Wednesday.
So we can think of each of these days as a category.
And then we want some metric per category, such as the total expenditure or the sum of the total bill
per day.
So if we were to do this for the data set and keep in mind, this dataset actually only has four days
Thursday, Friday, Saturday and Sunday, we can get a bar plot or a categorical plot.
And again, the whole purpose here is to show some sort of metric per day.
And what's nice about the visualization is it makes it a lot easier to compare different categories
to each other.
So here we can really quickly tell that Saturday and Sunday tend to be the highest expenditure days
and surprisingly Friday is actually quite low.
And here we can quickly see the total sums per day in a visual format, allowing me to quickly compare
days to each other.
I could have given you the same metrics just in terms of numbers per day, but with a visualization,
it's very easy to see.
The story here.
Saturday and Sunday are very popular then followed by Thursday and Friday surprisingly low.
Keep in mind, you can always add in more features.
So for example, I could separate this out via another category, and now I have a category per category.
So essentially a metric per category per category, a metric per day per sex, male or female.
And you can keep adding more features via color or design.
But again, we really want to make sure that we're showing the simplest visualization possible in order
to get the information we're interested across.
And again, remember that there are a lot more types of visualizations than the ones shown here on our
quick little tour.
But always keep in mind what is the information I want to share or the story I'm trying to tell, and
how does this specific visualization help in conveying that information to others?
Okay, let's take a much deeper dive into data visualizations throughout the next lectures.