We've already talked about the idea of a population and a sample where a sample is a subset of the population,
and we're trying to take a sample from the population that is representative of that population.
But now we want to talk about this idea of comparing information about the sample to information about
the population, for instance, the population mean versus the sample mean.
So the first thing we want to say is that when we talk about information that's related to the population,
we call those the parameters of the population.
So the population has the parameters of size mean and standard deviation given by capital nn mu and
sigma respectively.
And in comparison, if we talk about these same characteristics or the same set of information that
relates to the sample, we call those values statistics.
So characteristics of the sample are statistics, characteristics of the population are parameters.
And we already looked at these values before the size of the population is given by capital N, whereas
the size of the sample is given by lowercase M, we indicate the mean of the population and sample with
MU and x bar respectively, and the standard deviations of population and sample are given by Sigma
and S, And the whole idea behind the field of statistics is that we are sampling, we've talked about
sampling already.
We are sampling from the population.
We are collecting statistics from that sample.
For instance, we're looking at the mean and the standard deviation of the sample.
And our goal, our hope is to be able to use these statistics to make inferences about the corresponding
parameters from the population.
In other words, we want to be able to look at the sample mean and use the sample mean to get what we
hope will be an accurate estimate of the population.
Mean because after all, it's these population parameters that we're really interested in.
We want to know maybe the mean and standard deviation for the entire population.
And if we could actually calculate that value specifically by collecting data or surveying the entire
population, we certainly would.
We're only sampling because it's very difficult or impossible to collect data from the entire population.
So the best we can do is sample gather statistics and then use these statistics to make inferences about
the corresponding parameters.
The problem is that if we just collect one sample from the population and we compute a mean and a standard
deviation for that single sample, the values we find for sample mean and sample standard deviation
may or may not be good estimates of the mean and standard deviation of the population.
In other words, if the sample is very representative of the population, if it does a good job representing
the population, then our statistics might be good estimates of the parameters, but we might just happen
to pick a sample that is not very representative of the population or does a bad job representing the
population, in which case our sample statistics are going to do a bad job estimating their corresponding
population parameters.
Now to try to prevent this problem where the statistics we find are bad estimates of the corresponding
parameters, we can think about the idea of taking many samples instead of just one sample.
You can imagine that if we take just one sample, that sample will have its own mean and maybe we call
that sample mean x sub one like this because it's the sample mean for the first sample and the standard
deviation of that first sample will call s sub one, but then we might take another sample.
And that sample mean we might write like this X to with that standard deviation for that second sample
being as two.
And maybe we take a third sample and we compute the mean and standard deviation for the third sample.
And obviously we could continue on and eventually if we keep sampling over and over and over collecting
many, many samples, we end up with a whole set of sample means x1x2x3x4x5 and on and on through many
samples.
So we have this whole set of sample means and it turns out, and this might feel somewhat intuitive,
that as we take many, many sample means more of those sample means will turn out to be closer to the
population mean mu and fewer of those sample means will turn out to be further away from the population.
Mean mu and in fact that set of sample means will form its own probability distribution around the population.
Mean mu, and that probability distribution of sample means will almost always be a normal distribution.
This normal distribution has its own mean and if you think about it, we could call that mean the mean
of all the sample means right.
This is the probability distribution of the sample means the means of all those different samples that
we took and we could calculate a.
I mean, of all of these sample means.
So this probability distribution has its mean right at the center here as the mean of all these sample
means and that mean of sample means.
If we put it in maybe right here we call mu sub x bar because x bar is how we indicate a sample mean.
And so this is the mean of the sample means.
And it turns out that this probability distribution is centered at or has its own mean the mean of the
sample means and that this mean of sample means will be equal to.
And this is the amazing part the population mean and this is in fact the conclusion of the central limit
theorem which is an amazing conclusion.
It tells us that if we continue to take samples and we find a sample mean for each one of those samples
and so we get the sample means x1x2x3.
If we plot those sample means along a number line, then it turns out that more of these sample means
are going to cluster around the population mean.
So we might calculate the first sample mean and find that its value is let's say right here.
And then we calculate a second sample mean and we find out that its value is here.
We calculate a third sample mean and we find out that its value is here.
And as we continue calculating more sample means taking more samples and finding the mean of each sample.
We see that those means start to cluster more heavily around this one value.
Here.
Of course, there's some all over this distribution, but most of them are clustered in the center here
at this value.
And then fewer of them out to the right here and fewer of them out to the left here.
But most of them are clustered around the mean of the sample means, which is going to be equal to the
population mean.
And so this is amazing here because what we're saying is that we have a population and we don't know
the population mean we have no idea what it is.
But with enough sampling, we can create this distribution here.
And this distribution is almost always going to be a normal curve, a technically normal curve, that
bell shaped curve that is symmetrical with its mean.
Here at the center, we call this distribution the sampling distribution of the sample mean or SDS.
M Which makes sense because we're creating a distribution by sampling, and this is the distribution
of all of the sample means.
So it's the sampling distribution of the sample mean.
And the central limit theorem tells us that this distribution will be a normal curve.
So what this is really allowing us to do is turn a population that is non normal, that is not normal
into a distribution that follows a normal curve here.
And once we have this normal distribution, we can convert it into the standard, normal distribution
and use Z scores to answer all kinds of probability questions related to this distribution and in that
way make educated guesses about how accurate our statistics are as estimates of their corresponding
population parameters.
Now, a couple of things we should say before we just move on from the idea that we have a normal curve
here for the sampling distribution of the sample mean.
The first is that if the original population is normally distributed, then the sampling distribution
of the sample mean will also be normally distributed regardless of the sample size that we use.
In other words, if the population is normal, then we don't really have to worry about the size n of
our sample.
We know that the sampling distribution of the sample mean will also be a normal distribution.
It'll be a normal curve.
But if the population is not normally distributed, or if we just don't know whether or not it's normally
distributed, then the sampling distribution of the sample mean is only guaranteed to be normal if we
use a sample size of at least 30.
So n has to be greater than or equal to 30 in order to ensure that this curve is a normal curve.
So if at all possible, it's good practice to use a sample size of 30 or greater to ensure that we're
working with the normal curve here.
If we don't know whether or not the population itself is normal, which will often be the case, we
often will not know for sure whether or not the population is normally distributed.
So now that we have a sampling distribution of the sample mean, we recognize that this is a normal
curve.
And so we can talk about the mean and standard deviation of this specific distribution.
Specifically, we can say that the mean of this distribution, which we said we can think about as the
mean of the sample means is equal to the population mean.
So that's our first point there.
Then the variance of this distribution is going to be equal to sample variance divided by sample size.
So the variance of the sampling distribution of the sample mean will be equal to the sample variance
of the single sample that we took.
Divided by the sample size for that single sample.
And therefore, remember, that standard deviation is always the square root of variance.
So as you might expect, if we take the square root of SE squared divided by MN, the square root of
SE squared is just SW, and the square root of N is the square root of MN.
So the standard deviation of the sampling distribution of the sample mean is se divided by square root
of MN or the standard deviation of our single sample, divided by the square root of the sample size.
And we write that standard deviation as the standard deviation with x bar here, because it's the standard
deviation of the sampling distribution of sample means.
And specifically, this value is important enough that we give it a special name.
We call it the standard error.
In other words, the standard error is another name for the standard deviation of sample means or the
standard deviation of the sampling.
Distribution of the sample means.
Now, of course, just like with any other standard deviation value we've talked about before, when
this standard deviation is larger, when the standard error is larger, it tells us that the sample
means in the sampling distribution of the sample mean are more spread out away from the mean, which
means that any one particular sample mean is less likely to be an accurate representation of the true
population mean.
Whereas when standard error is smaller, when this standard deviation is smaller, it means that all
these sample means are more tightly clustered around this mean here, which means that any one sample
mean that we take is more likely to be an accurate representation of the actual population.
Mean.
In other words, standard error gives us an idea of how likely it is that any given sample mean is an
accurate representation of the true population mean.
So ideally, what we want is a small standard error because the smaller we can get the value of standard
error, the more likely it is that our sample mean for our one single sample that we took from the population,
the more likely it is that that sample mean is a good representation of the population mean, which
is ultimately what we want because we want to know population mean.
So we're hoping that when we take a sample the mean of that sample will be close to the population mean
and the smaller the value of standard error, the more likely that is to be true, the more likely it
is that the mean of the single sample we took is close to the population mean.
So then the idea becomes how do we decrease the value of standard error?
How do we get a really small standard error?
Well, when you have a fraction like this one, we have a sample standard deviation divided by square
root of sample size.
When you have a fraction, there are two ways to decrease the value of the entire fraction.
We can either decrease the value of the numerator, which in this case would mean decreasing sample
standard deviation and or we can increase the size of the denominator, which in this case means increasing
the size of the sample.
Now, there's not a whole lot we can do about the sample standard deviation, because the value of sample
standard deviation is just a result of the data that we collect from the sample.
It just turns out to be whatever value it is based on the sample that we get.
What we can do is increase the sample size and that should make sense to us at an intuitive level.
The larger our sample, the more accurate our statistics are going to be as representations of the parameters.
If our total population, let's say, is 1000 people and we take a sample of just ten people, intuitively
we know that that sample could turn out to be really inaccurate.
It may not be representative of the population at all, but if we have a population of 1000 people and
our sample is, let's say 700 people, we drastically increase the sample size, being able to collect
information from 700 of the 1000 total people, a 700 person sample of a 1000 person population.
We're probably going to get a pretty accurate idea of what that population looks like with a sample
that big.
So as we increase the sample size, we probably got a more accurate sample mean and therefore a smaller
sample standard deviation.
And so our standard error is smaller and it's more likely that our sample mean is an accurate representation
of the population mean.
So the conclusion there is that taking a larger sample is going to help improve the accuracy of the
sample mean as a reflection of the population mean.
But of course taking a larger sample might mean that we have to spend more time or money on our sampling
process.
It might be more difficult to take a larger sample.
So in the real world, we're always balancing these things.
Maybe we want to take as big of a sample as possible, but we're limited by our manpower or money,
our resources, and so the sample we take can only be so big.
Now, the other thing we want to say about these formulas for variance and standard deviation is that
depending on the size.
With our population and the size of our sample, we may need to apply what's called the finite population
correction factor, which looks like this.
So the finite population correction factor.
FPC for short, we'll talk about when we need to apply it, but when we do need to apply it, the formula
for variance changes to this formula.
Here we keep the SE squared over n, but we have to multiply by capital n minus lowercase n.
In other words, population size minus sample size, divided by population size minus one.
And then the formula for standard error or standard deviation of the sample means is the same as divided
by square root.
N.
But then we have to multiply that by square root of this population size minus sample size, divided
by population, size minus one.
Now, when do we have to apply the finite population correction factor?
Well, we have to do it when we are sampling without replacement and or when we're sampling from more
than 5% of a finite population.
So here's what that means.
Remember previously we talked about taking a simple random sample.
Well, whenever we're sampling, ideally we're sampling with replacement, which means that in theory,
the same person can be picked multiple times for our sample.
So, for instance, to take a simple example, let's say our population is all the people who live in
our neighborhood, which maybe is 1500 people.
If we're sampling with replacement, it means that we randomly choose one person in the neighborhood
and we record them as part of our sample, but then we sort of throw them back into the pot and we pick
another person to be included in our sample.
We throw them back in the pot and then we pick a third person, throw them back in the pot, pick a
fourth person, until eventually we have our full sample.
What that means is that because every time we choose to include someone in our sample and then we sort
of put them back in the population, in theory, we could put that same person multiple times for our
sample.
Two times, three times, four times, really, because it's random chance we could pick them every
time for our sample.
That's the idea of sampling with replacement, The same subject object person, whatever it is in the
population can be chosen multiple times for the sample, but sometimes we won't be able to sample with
replacement.
Maybe we're working for a political campaign and we're asking people as they leave their voting location
about how they voted and we're collecting that information for our sample.
Well, obviously that person's only going to leave the voting location one time.
We're going to ask them our questions on the way out the door and then we're never going to see them
again.
In that case, we're sampling without replacement.
It's not possible for us to pick the same person for our sample multiple times.
And so we would be sampling without replacement.
If that's how we're collecting our sample, then we would need to make sure that we include this finite
population correction factor in the formulas for variance and standard deviation.
For this sampling distribution a sample means.
And then the other sampling condition when we apply the FPC is when we are sampling for more than 5%
of essentially a finite population.
So again, going back to our neighborhood example, when our population is the 500 people who live in
our neighborhood, 5% of 4500 is 75 people.
So if we are taking a sample that is 75 people or larger, then we would want to apply this finite population
correction factor.
In that case as well.
We only apply the correction factor in these conditions because outside of these conditions, it turns
out that this finite population correction factor, this value here, turns out to be very, very close
to one.
And when its value is close to one, obviously it's not going to affect the value of variance or standard
deviation.
And so we don't have to include it.
The value of the correction factor drifts away from the value one under these conditions and therefore
does start to have an effect on variance in standard deviation, which is why we have to include it
under these particular conditions.
So that's the idea behind the central limit theorem.
And it's critical because what it allows us to do is take one sample from our population and with this
justification of the central limit theorem telling us that the sampling distribution of sample means
is normal, a normally distributed curve around the mean of sample means, which is equal to the population,
mean What we'll be able to do with that conclusion a little later on is from the one sample.
Calculate the sample mean and sample standard deviation for that one sample.
And then because we have this normal curve and we know how to use Z scores to answer probability questions
about data that's normally distributed, we'll be able to make a statement about how likely it is we'll
be able to give a specific percentage about the likelihood that this sample mean the mean of our one
sample that we took falls within.
Some interval around the population mean.
In other words, we might find a sample mean let's say our sample mean is 14 and then we'll learn later
on about how to calculate an interval around this sample mean.
But let's say our interval is one and one half units to each side of this sample mean, which takes
us to 12.5 on the left and to 15.5 on the right.
And so we end up with this interval of 12.5 to 15.5 and we might be able to make a statement along the
lines of we are 95% confident that the population mean falls somewhere in the interval 12.5 to 15.5,
which of course is a very powerful kind of conclusion.
We can say how confident we are or how likely it is that the actual real population mean will fall within
some particular interval, even though we have no idea what the actual population mean is.
And all we did was take one single sample from that population and calculate a sample mean of 14.
But using just that one data point, just that one statistic, one sample mean because of the power
of and the conclusion of the central limit theorem as it relates to the sampling distribution of the
sample mean we can make a statement about our confidence level that the population mean falls within
some particular interval around the sample mean that we calculated.