The last couple of plot types we want to talk about are violin plots and kernel density estimation plots.
And both of these are based on the histogram that we already looked at earlier.
So the process of kernel density estimation or CD is what allows us to produce a smooth curve out of
a histogram.
So as an example, if we take this histogram that we were working with earlier, when we first studied
histograms, we remind ourselves that this is a distribution essentially, and kernel density estimation
allows us to create a distribution curve or a density curve out of this histogram, for instance, something
that maybe looks like this.
So essentially what we're doing here is just creating a smooth curve out of the original data set that
we started with.
It lets us visualize the shape of the data, the shape of the distribution without looking at all the
individual bars in this discrete histogram.
So in terms of terminology here, we might say that we use kernel density estimation to create the kernel
density estimate.
Where the kernel density estimate is this curve, we might also call it the distribution curve, the
density curve, the kernel density plot or the CD plot.
But no matter what we call it, we'll use this CD plot as part of the violin plot as well.
Now, it's important to say that today we'll basically always use software or calculators to create
file in plots and CD plots.
We don't really do this process by hand, but we do want to understand the fundamentals of what's going
on behind this.
So what we really want to know about the CD plot is first, that we can dictate how smooth the plot
is.
And that smoothness is based on what we call the bandwidth of the kernel function.
So think of the kernel function as something maybe that looks like this.
And this kernel function is essentially looking at all of the data points contained under the curve
of the kernel function.
And we could think about the data points underneath this function based on the data points that are
making up the histogram behind it.
So it basically looks at those data points and based on the number of data points and how close they
are to the center of the kernel function gives us a height for the kernel density estimate or gives
us a height for this density curve.
We can use kernel functions of different shapes so we can use a shape like this one.
You'll also see a kernel density function that is a normal distribution like this.
We can use a kernel density function that's a uniform distribution like this one or even a triangular
distribution like this one.
And when it comes to the kernel density function, we really just want to think about its width and
its height.
So if we have this kernel density function, it has a certain width to it.
We call that the bandwidth and it has a certain height to it.
We call that the amplitude.
So this kernel density function here has a smaller bandwidth and a smaller amplitude.
Then this kernel density function here, because this larger one here is both wider and taller.
And the fact that it's wider means it has a larger bandwidth.
And the fact that it's taller means it has a larger amplitude bandwidth.
The width of the kernel density function is what dictates how smooth the kernel density plot will appear.
So a lower bandwidth or a narrower kernel density function indicates a less smooth curve.
So if we were starting with this original kernel density estimation plot and it's based on some kernel
density function, if we were to decrease the bandwidth or narrow the width of that kernel density function,
then maybe our kernel density estimation plot changes from looking something like this to looking something
like this.
See how it's much less smooth?
We see a lot more variation throughout the CD plot.
If we increase the bandwidth again, then we go back to the smoother curve.
So if we were going to summarize that, we could see here a table where as we move from left to right,
the amplitude of the kernel density function increases.
In other words, in each row here, as we move from left to right, we can see that the height of the
triangular distribution here, assuming we're using a triangular distribution, that the height is increasing,
height is increasing, height is increasing.
And as we move from top to bottom, the bandwidth of the kernel density function is increasing.
So even though we're keeping height roughly the same here in this first column, as we move from top
to bottom, the bandwidth increases the width of that kernel density function increases.
So in the top left here, we have the kernel density function with the smallest bandwidth and amplitude.
And in the lower right we have the kernel density function with the largest bandwidth and amplitude.
In other words, this is what's going on behind the scenes when we use software.
So when we use software to build a CD plot, all we're doing is entering the data that would have created
the histogram in the first place.
So all of the data points that make up our histogram and then we give the software some value for bandwidth
and.
Amplitude of the kernel density function.
And that creates for us the CD plot based on those values.
And in short, it's a way to turn a discrete histogram distribution here into a smooth distribution
curve or a smooth density curve.
Now, keeping the shape of this curve in mind, this CD plot here in red, let's look at how the shape
of this plot here can transition into a violin plot.
So to move toward violin plots, what we've done here is taken the CD plot for this histogram and we've
created its mirror image.
We've also moved this horizontal axis down here to the bottom.
And now if we take away the histogram, what we have is a violin plot for this same data, which means
that a violin plot is really just a more visual way of showing us the distribution of the data along
some scale.
So violin plots are also really similar to box plots or box and whisker plots in the sense that we always
have this sort of external axis that helps us position where the violin plot actually is according to
this external scale.
So we have this external scale, we have the violin plot, which is just a symmetric picture of that
distribution that was created through the process of kernel density estimation.
And then for violin plots, we usually add additional data, and this is where we see that violin plots
actually are very similar to box plots.
So this red outline here is a visual representation of the data distribution.
But then we add something very similar to a box plot inside of this violin plot.
So this box here is just like the box from the box plot in the sense that this left edge here is the
first quartile or the 25th percentile.
The right edge of the box is the third quartile or the 75th percentile.
And then we also indicate the median, which is this blue dot.
Here we indicate the median, which is the second quartile, also the median.
And therefore, of course, we can identify the interquartile range as the value here at Q three minus
the value here at Q one.
So it looks like according to this external scale, if we say that the third quartile is maybe 70 and
the first quartile is maybe 20, then the interquartile range would be 70 -20 or 50.
So from this box plot, we have the first, second and third quartiles, which means we also have the
median, because the median is equivalent to the second quartile.
That's two different ways of saying the same thing.
And then we usually also plot this line behind the box.
And this line is not equivalent to the whiskers of the box plot.
Usually it represents a 95% confidence interval and we'll talk about confidence intervals later in the
course.
But the idea is that 95% of our data points lie within this 95% confidence interval.
We will also sometimes include the mean as part of our violent plot as well.
Not always, but sometimes.
So this extra dot here is supposed to represent the mean.
That's something we can optionally choose to include in our violin plot if we want to.
So sometimes we'll see violin plots sketched out this way with this box plot in the center and the 95%
confidence interval.
But sometimes we'll see violin plots with just three lines running here, perpendicular to the plot
that represent the first, second and third quartiles.
So if we sketched in those lines behind this information, we might show them like this.
You can see here at the first, second and third quartile.
And if we chose to go that direction, then our plot would just look something like this, where we
have this line for the medium, this dashed line for the first quartile and this dashed line for the
third quartile.
It gives us a lot of the same information, but in a simpler form.
And then the other thing that we want to say is that we can always display violin plots horizontally
like we did here or vertically like this.
Now, the histogram itself behind the kernel density estimation plot and therefore behind this violin
plot histograms, we almost always display sitting on a horizontal axis with the bars extending up vertically.
So a horizontal violin plot like this one kind of matches that format.
But it's very common to also see violin plots rotated counterclockwise by 90 degrees and shown vertically
like this.
Everything's exactly the same.
It's just that we're showing the distribution vertically.
So it's as if we take a horizontal violin plot like this, grab the right edge of it this way, and
rotate it counterclockwise by 90 degrees just a quarter turn to turn it on its end and create a vertical
plot here, whether we show the.
Island plot.
Horizontally or vertically, we always run the axis for scale parallel to the body of the violin.
So here we have a horizontal plot.
Our axis for scale is horizontal.
Parallel to the plot here, because our violin plots are vertical.
This is our axis for scale, and so it runs vertically parallel to the violins.
Now, violin plots, we can say, have an advantage over the traditional box and whisker plot in the
sense that they give a great visual picture of the distribution as sort of the body of this violin.
But that also makes violin plots more visually busy and sometimes harder to interpret depending on what
it is we're trying to show.
As always, when we're trying to choose a chart type, we should just be thinking about the kind of
chart that we should use in order to best represent or most clearly communicate the data that we're
trying to display.
So maybe depending on the data that we have, it's not super helpful or important to show this visual
representation of the distribution.
We might, in that case, just choose to use simple box plots instead.
But maybe showing this visual representation of the distribution is particularly helpful because we
can more clearly see the similarities or differences across distributions.
We just need to think about what makes the most sense for the data that we're working with in the results
that we're trying to communicate.
If we have multiple data sets like we're showing here in this graph, we have data set A, data set
B, and data set C, It can be particularly helpful to arrange the order of the corresponding violin
plots based on the value of the median.
So in this comparison plot here, we can see that the violins are arranged from smallest median on the
left to largest meeting on the right, unless there's some other reason to arrange the data in a different
order.
For instance, maybe data set B here represents the month of January, data set C represents February,
and data set A represents March, and may be this chronology is important to us such that it would make
more sense to show the violin plot for data set B and then C and then A so that we can see change over
time.
That version might make more sense.
Or if we don't have this kind of ordering or something like this doesn't matter, then maybe arranging
them this way as a, B, C in order of increasing median does a better job at communicating the differences
between the data sets.
So that's the idea behind creating CD plots and violin plots.
And now that we understand how to create all these different types of charts and graphs, in the next
lecture, we'll look at common plot pitfalls or things that we should make sure to avoid when we're
creating these different types of charts and graphs.