Hello, everyone.
So let's start working on a case study.
And for this, we will be using a commonly used dataset, which is how how is price prediction?
So let us import the basic libraries, which we will need.
So we have important non-pay.
US, you, my.
And we have also important the data set which we will in needed for the latest, Jabe Schabel, the
data.
So we can simply say to you don't want to get the data sheet.
So here you can see there are fourteen hundred and sixty rows and 81 columns, which we have.
So let's have a look at the data first so we can safely thought or tend to view the data.
But first, we can also set some properties in pandas so that we can view all the data points, at least
all the columns.
This will allow us to view all the columns and once we have said it to anyone.
Now we will fund this.
And you can see we are able to view all the data points at once.
So what does the status quo?
So this is a data regarding house prices and different parameters, which actually are present for estimating
that house price.
So this is the column sale price, which is the price of the house.
Further, we have different properties regarding the house to be rented.
This is simply an IED, which cannot which does not hold any specific information.
It is just a unique identifier for each and every house.
Then we have a subclass and a zoning.
What is the log frontage?
What is the area?
What is the street?
Which alleys on the sheep, Dequan?
Door of the land?
The utilities which are available?
What is the neighborhood is what is the condition of the house?
And several other details are present.
You can also view delete a dictionary which is available with the data and using that.
Additionally, you can easily find out what all the information is presented, this data.
So usually what we do is we actually go through the entire data dictionary to understand what type of
data we have.
And for that, we look at the data frame itself and we try to identify what type of columns are present
and different attributes of a particular dataset.
So for that, letters, of course, have a look at the names of the data so we can simply say they,
Don, don't them?
And you can see the columns of the data.
And for the letters, also have a look at different.
You can see different type of information so we can say data to inform full view.
What type of data is present?
So you can see IDs, integer and subclasses and data that are fourteen point values also.
And there are a lot of categorical columns which are present.
You can see these are object.
These are all categorical columns which have datatype as an object for there.
You can also see the number of null values.
So you can see that there are one four six zero, not null values here.
You can see in the column there are a lot of null values present.
Here again, in five place, there is a lot of null values present.
So accordingly, you can decide which column you want to keep.
Usually I will not keep any column which is having more than, let's say, 50 percent or more than 20
percent, no data.
So it's a big no.
So we remove these columns.
Flameless.
Q You will see things, miscellaneous features.
You can remove all of these because these will not really impact will not hold much information or do
what you can do is you can have a view of the mean median more than important metrics of the numeric
variables.
So for that, what you can simply do is you can see a detente or that's great.
And here you can see the count, the mean, the standard deviation, the minimum value, the maximum
value.
Then you can have a look at the twenty five percentile books and ten, seventy five, PopTech, whatever.
Do they split and then you can accordingly have a look at that.
You can see this seems to be a categorical value just Kadence.
Here we have zero one, two, three.
So maximum three bathrooms are present here.
This is how but maximum to have but sufficient so that you have all the information which is present
here.
You can see the maximum price of our house is seven point five thousand.
Well, if you see the minimum value, is this here?
This is the minimum value.
And the maximum value is very much different from the seventy five percent in value.
So here you can easily see that the values are here.
We have a lot of outliers, but we will be able to identify and analyze that later.
Let's go further.
And let's see the different sale prices and let us actually have a lot of the distribution of the sales
prices.
So for that, what we can do is we can simply say it's this thought, this song, which is the distribution
block, and inside this, you can give data and in data.
I want to have a look at the sales price column so I can put appeal and order.
I can simply say not not sure.
And
now here you can easily see that this is the distribution.
Initially, the data looks nicely distributed, but later you can see there are a lot of points which
are lying towards the right thing.
So here you can see that data is skewed in nature.
For though, you can easily find out the mean median mode.
Although we would we have seen the mean of these values, you can still find out the median, the mood,
the skewness, the Cardoso's.
So let's do that first.
So here I have retrieved all the important metrics that is minimax mean, median Morde, standard deviation
readings, cutlasses and skewness.
So you can see it is highly lepta political protecting nature.
It is highly skewed.
So you can easily see these other data points, which we have.
You see, this is the main value.
This is the max value.
This is the mean value.
And this is the median value.
You can easily see that the mean and the median value is almost similar in nature.
So there is not much difference because here there are very few data points which are present in the
team.
So this is what we have got from this.
Now, further, what we can do is we can try to find out the sample and population mean.
Let's try to identify something from this particular data.
Let's see if a sample will be similar to the population itself.
So let's try to find that out.
So for this, I can fix a.
So let's say in Bidoun flying now.
So it seemed.
So here I am giving some seats, 23.
And now let's take some sample values.
So let's take oh, how many data points out there?
So the shape is one four six zero.
So let's take 500 data points from this.
So we can simply see something.
Each is equal to impede or random
for.
It's his price, and out of this, I want to get 500 values so I can see size is equal to 500.
So this is my south village.
And for this, I want to get the mean to what is to me with this, I can just simply pectus and find
out the meaning of this.
So here you can see that the sample price.
It's nothing but one eight one six seven five.
Now, if we look at the population mean we can basically find all the means we hear already we have
it.
So this will see something mean the third one.
So this is the population mean and this is the sample mean.
So you can see these are clearly very close to each other.
Now, next, what we will be doing is we will try to find out a little more about it.
Now, let us have a look at the few neighborhoods which are present.
So for that, we can have a look at these numbers, the work.
These are the names.
So these are the different neighborhoods which are present.
Now let's see for Edward if the prices in Edwards is similar to the prices in the entire
place.
So what we do is we can simply try to find the comparison between both of them and we can compare this.
So for this, we will be performing the sectors because we are comparing a sample to the population
and we want to find out if the sample is different from the population.
We want to compare a particular neighborhood to the entire population, which we have.
So let's do that.
Before that, we will first need to import the library for the test so we can simply says from.
From stats Mondelēz about the stats.
With stand.
Because it is so wildly important, said that we will be using this particular test to find out if the
population is similar to the neighborhood or not, so we can save said.
Well, and you do have that as an output from this test and say that this we will have to pass the population
as well as the sample, so we can simply say X one is equal to data.
They
would be done a very good is equivalent to it punch.
Now, from this, I want to get these sales price so and sees his price hill.
Now, apart from this, I need to provide the entirety of space for the population so I can simply say
values is equal to Peter still space.
Let us again have a look at the documentation to be sure of it so we can see.
His leftist.
So you can see it takes value
and the value has to be they mean, so we'll see.
I mean.
Now let's run this and also let us bring these Etoile and Biegun.
So you can see that does did well is minus 12 and the P-value is very small, which means that the neighborhood,
Edward, is significantly different from the other neighborhoods.
Now, let us perform the same thing for another neighborhood so we can fix the entire thing from here
and we can replace it with another neighborhood name.
Let's take Boystown for now.
And it's this.
So here again, the value is very small.
So you can easily see that the mean for the location, Edward, and the entire population is different.
And similarly, the mean for the location or town is also significantly different from the entire population.
So this is the data which we have.
So we are just trying to see that the means would be different for both for the sample that we have
picked for a particular neighborhood and for the entire population.
Now, when we look at this data, we can easily see that the neighborhood is having a very important
impact on the teams faces.
So if there is one neighborhood, there would be different type of sales places, which would be pleasant
in another neighborhood.
There would be some other type of sales places would be which would be pleasant.
Now, for the weekend here, we have compared to the population.
We can also compare the city's prices and different factors amongst the neighborhoods.
Is that because these will be two different detours?
The one would be, for example, neighborhood and there would be for Vye Neighborhood.
So here we are comparing two different samples.
So based on two different samples, we can try to find out if those two samples are somewhat similar
to each other or not.
But that is also another thing which we can do.
But whenever we are dealing with two different samples and not comparing with the population, that
is the time when the other tests come into which that is like the disguise criticized.
Those tests will come into picture.
When we are talking about population, we will try to solve it using the Zetas because their test gives
a population into consideration.
Now, let's try to identify how many houses would I be able to get if I had a certain amount of money.
So let's first find out the details about the neighborhood input.
So here, let me get the data for Atwood and for the same.
I will find out what the mean value and standard deviation.
So the means
to me
and it would be the standard deviation.
OK.
Now, let us import that package from
before.
That's usually what we do is we imake on imports at the top of the file so that if someone else is looking
at our code, they'll be able to find out more likely to be imported.
So usually in take this example of your coffee.
So that is find out the Z score.
So here, as we know, that code is equal to the value.
So let's say I want to find out how many houses I will be able to get if I see something that's 50000.
Then I looked the mean from it
and I divide it from the standard deviation.
And I want to find out later the percentage.
So the percentage can be from Goldman Sachs thought, ma'am, thought C, the F CVS will give you the
probability, the percentage probability from the Z score.
So we put into that scoring in.
So here there is ninety nine point seven, five percent probability that you will get a house in Everwood
if you are paying 250 thousand.
Now let's decrease it a little bit.
So let's say sort of 250 thousand.
I'm willing to pay only 180 thousand.
So let's see how much is the probability.
So there is a B eight point four, six percentage that I will be able to find a good house for myself.
Now, let's do the same and check for some other neighborhood.
So let's have a look at.
So I
see there are very few houses there, so let's check what is to start with that particular place.
So let's first check for 250 and drive this.
So here you can see that is eighty seven point two.
It was intense.
And I'll get a house in science.
So this means that the amount would have been higher.
The range would have been higher.
So let's try to look at the mean for this particular location.
So, I mean, is a one eight six triple five that get the maximum value?
See, here is a higher value.
So let's put.
So even after paying a much higher amount in comparison to input, there is a comparatively lower probability
that I'll be able to get a house here.
So as we see Ed Wood and Soyer have comparatively much difference in the max value and the values which
I'm getting into that school, let's try to find out from the samples of both if these would be actually
different from each other or are they are similar in nature.
So if I take a sample from somewhere and if I did a sample from Edwards, let's try to find out if the
those prices are significantly different in Edwards.
And.
So let's let's try that.
So for that, first, we need to get the data for both the locations.
So let's get it inside a piece.
So we say E is equal to the data for soy
and the beans equal to the data for October.
Now, let's do a thesis on this independently so that.
So the the other schools before independent.
And we have kids in here.
And we give amethysts zero and equal.
Explain that.
And it's from this.
You can see that the P value is very low.
Hence, we can see that these the prices in Soyer and in gold are significantly different.
Now, let's actually try to find out that with how much amount I will be able to be 90 percent sure
that I will get a house in, let's say, in Englewood or or down or any other location.
So let's try to find that out.
So let's do it below this one also.
So, yes, let's use these same means and standard deviations.
So now let's check foreswear.
So let's see how in how much amount in what interval.
I will be almost 90 percent sure to find out that I be able to get their house.
And.
So for that, I can simply do with the fact that they don't know what I mean.
So I guess we need to provide how much was did we want to show it with?
How much probability we want to be sure with.
So let's say 90.
Then we need to provide the details.
So let's say we're looking for societies.
So let's get new places for societies.
Then we need to provide the means.
And the skills would be the standard deviation.
Now, we can simply run this.
And these are the different values which we are getting.
We have to provide the lens here, so we say, and then
this will give us the DNA P-value.
So here.
You can see the blower and a button below.
So we need to have at least two, seven, nine typifying to be sure that there is a 90 percent probability
of getting them or getting the house here.
So let's look at some other
neighborhood.
So let's check for Edwards.
Let's run this entire thing.
Put it Curtice.
So we need to get assistance.
The experts that strung this post so far, 90 percent probability, we need to have around
200000.
Let's get further and let's try to look at it so at some other way.
So we actually saw that when we are comparing the neighborhood for approval for the actual sale price
of the entire data, it is significantly different.
Same is the case of Georgetown.
So for both, it is significantly different.
And we found that using these mechanisms now is at best is usually used when we are comparing with the
of the population values.
And then we don't really have these samples in hand.
But when we're dealing with samples, then in that case, in sort of using this as a test, we should
have used even to repeat this.
So let's apply the same.
Let's try to find out if the neighborhood sample for neighborhood Edwards or the neighbor down or it
is significantly different from the entire population means.
So let's try to do that.
Let's try to find that out.
So let's take this from here.
Let's get this news.
So now we are applying the one somebody asked.
So let's get the scores and the bailout deal.
And this we will be getting from these steps are the best ideas for one step, not just for one sample.
Then we get the data.
So the data would be just the same from here.
Let's check for a good.
And this will be the population us.
Now, this is people me.
And here we have the data, so instead of providing the entire population, we'll have to provide the
sample here.
So let's take a sample of, say, 60 values, taking a sample of 60 values.
Now
let's get the Peace Corps and the P-value.
So here you can see again, the P value is significantly low.
Hence, we can easily see the same thing, that the neighborhood advert is significantly different from
the population on the basis of one sample fetus.
Same thing we can do repetitively for different neighborhoods.
Let's pick a different neighborhood.
So let's see.
For now, for proper iliakis this and insert over 60 Nesquik just to be here.
What is the actual.
That's fifty one so they can be put to you.
So here you can see here the P value is zero point zero two five three.
So this is not much different.
But yes, it is marginally at the exact point.
So it completely depend on the P value, which you are looking at.
So if you're looking at, let's say, one percent, then it is significantly different.
If you are looking at 2.5 or 2.5 percent, then it is almost that.
So this is completely up to you how you look at it.
And again, if you're looking at, let's say, five percent P value, then it is way below the five
percent above the five percent P value.
Then you cannot see that it is significantly different.
So that's how you can easily evaluate using the one sample doing that.
So it completely depends on the significance level that you're looking at.
Now, let's look at this particular location.
You can do your own analysis.
I'm just giving you an idea of how you would perform an analysis, how you would do that even in sight.
Now, let's take an example of, say, 60 years.
Here you can see this is very clearly visible, that that is a significant difference between the amount
of opposition and quality growth.
So that's how you get an insight photofit.
Now, for there, we can have look at different now.
We have been comparing a particular neighborhood with the entire population.
Now, in case we want to compare one neighborhood with another neighborhood, here, you can see this
is how we do it from the independent Betis to compare to different neighborhoods.
So this is only the which we have under the best umbrella.
We have the B, the independent the best.
We have the one sample, the test.
So you can perform these features like this.
Then apart from that, we will have prepared people.
So then we have the bare details.
It has to be done when we have some particular data, which is the same sample is being tested for in
another time frame.
So in that case, in this particular data, we don't have that example, but you can surely use that
in any other case from any other dataset.
So that is completely up to you.
But just to give you a particular insight, let me try to give you an example from this particular data.
So so let's say be down and get the value for each day.
Now, the t the values in it.
So these are the different values.
Now, let me plot these for you in the same plot.
So this might be a little time to generate the plot.
So here you can see the plot for the year boot.
Now, when you see this, you can see that there are few houses which were built earlier and major lead
in on houses which are very neat here.
So let's try to compare the houses which were built.
Nineteen sixty and the houses which have occurred after nineteen sixty.
So let's do that and compare using the independent dataset.
So let's see A and B and let it be.
Now let's get this condition.
So.
Let's get just as much from
now, we need to compare the year before,
and let's say we're looking for a less than equal to 1960, the same space for that.
And the ones which are based off the 1960s included in 1960.
These are the two which we wanted to compare.
Now, for the next to the datasource that don't be based on the school that might be four independent.
And we provide in the and we see this equal to zero and b C for is equal to two.
Now let's run this.
So here you can see the P value is very, very, very low.
So which means that there is a very highly significant difference between the sales prices of the housebreak
before and after 1960.
Now, let's do the same comparison on the houses, which are right after, let's see, 1980.
Let's compare on the basis of 1980.
So here again, the prices are very different.
Now, let us strive for 1980.
Same thing.
So here you can see the result now.
Let us move to the Chi Square umbrella test, so let us check if dolus different line slopes are independent,
so let us perform a test to check if knowing a particular line condole will have some impact.
So for that, we will have to divide the science place into different criteria.
So we'll go divide those whose plays into high, medium and luteal spaces, and then we will compare
the different line contours with each other.
So let's do that.
So for that, let me simply create.
Let me, first of all, import the stats by we already had these tags.
So that is perfectly fine.
So let us write a very small function, which will basically compute the frequency for guys square test.
And then we can perform Chi Square test using those frequencies.
So let's just write the function and just write that function for you and then explain it to you.
So here is the function that is very simple in nature.
So I have defined this function, compute frequency, chi square.
So what it does is it will simply take the two columns which you are trying to compare.
So I am providing the X and Y here.
So in actual, we are comparing willAnd, Gondor and basis.
So now let's get this.
Now what we do is we just create the crosstabs of these frequencies.
And using that particular crosstabs, we will simply apply the guys who are best for contingency on
top of it.
I'm using these guys.
What is the contingency?
We will be able to find out if Limed Gondwe actually has an impact on this, use phrases or not.
So let's simply run this now.
So here I have done this so you can see it simply created the cross tab.
And then I simply printed be crossed out for you.
This is the cross tab which was generated.
So you can see for our line one door, there are four line Condor's being key achiness, low level.
And then there seems places divided into high, medium and low using the feed.
You could you could basically isor function, which cuts the data or you can divide the data into equal
quarters.
So when I say three, when I see I've got this in the freezer, it creates three equal regions for all
three equally distributed categories for me, based on the numerical values presented the basis.
So here I have the guys with this to start testing.
So it comes out to be twenty six and the P value comes out to be a very low value.
It is almost zero here.
So this shows that there is a very significant difference between the scenes faces with respect to the
front line contours and their independent dimension.
So this is what we get you.
We get to know that there is a lot of independents in the land, on doors and their corresponding sets,
prices for this.
If we look at this particular P value, we get to know that when you're performing decries, grittiest,
and we have this very small P value.
So it also appears the relationship between the land control and business price.
Now, when you see the land, Korngold is having a very high impact on the scenes places.
Right.
When the not the line control is level, the sales prices are comparatively higher.
When the land condor is low.
You can see the sales prices are very low.
So this shows that there is a very important dependency of sales prices on the land Gondo.
So although Dilan condos internally are independent in nature, the sales prices are highly dependent
on the long line control.
So basically, non control is a very important feature to identify the sales places.
This is this is what you will find out using the guys with this, that if the sales price is dependent
on the line or not.
So when I say that the P value is very low, that means that the sales price is dependent on the line
control.
Very high now.
It simply shows that the two variables, sales price and loan line control, they are not independent.
They are highly dependent because price is very highly dependent on the line.
Quantock.
Now, let us perform the one the analysis for radians, which is an over.
Now it is the user domain, whether there are any statistically significant differences between the
means of three or more independent or unpredicted groups.
So Raegan, let check for the same thing.
Will check for Dilan Condole.
So Dilan Condor's bank is low and level.
They are significantly different from each other.
There are different categories.
So these are independent and related groups.
So we can actually check if the sales prices for these are different internally or not.
So we did see this seem to be able to see that the see they had an impact on two places.
Significant difference expectancy's prices.
But we were not able to compare them.
So let's compare them internally themselves.
So let's check again.
Now, here, each group which we will be taking considering has to be having a different participant
and each group should have a different take.
So when we are checking at this year's prices, we are looking at two prices of these different locations.
So their houses will be different for each location.
So the condition is perfect.
Now, next, we will be checking out basically what we are trying to find out.
It is when we perform the one we know, what is the one win or best will actually tell us if these are
different or not.
So we will be comparing multiple categories here and finding out which one is actually significantly
different from all of the categories or or less similar or not.
So if even one of the the category is significantly different, there will be a significant value which
will be coming out.
So first, let's applaud the group for this.
And we also applaud the book's plot for the theme.
I'm grouping the data by Land Cawdor and getting the sales places.
And later, imploringly land control over this is prices.
So let's try this.
So here you can see I'm getting the land condors and their respective count mean, standard deviation
and the maximum value.
There you can see the mean for bank is one four three one zero four here.
Twenty three, twenty nine, eighteen.
Here the maximum values are not much different here, but for level the max value is quite high.
Those 75 percent beilis somewhat similar for all for 50 percent nihil is again higher for the low one.
So these are the details which we are giving from these.
And total data is also different for bank we have for bank hills.
And Lord, we have smaller number of houses.
Microlevel, we have higher number of houses.
So now let's look further.
So we'll run the slides for if one way ANOVA.
So here when we run this, we are checking.
We are running it.
We are comparing it for these places of land, condo level, for fighting for the low and for height.
Now, let's run this.
So when we run this, we get the value as a very high, very low value, which means that at least one
of the classes is significantly different from all under three.
So at least one of them is different.
So what you can do is you can compare these you can compare these with one another and then find out
what I even further does here.
And you can see that be countered slightly different.
So let me take a sample of B five from each one of these.
So let's take a sample of thirty five from each one of these.
So let's get this again.
And they could go something
like this.
They've done this for.
Here you see still the P-value is coming out to be very low, which simply means that there is at least
one category which is having a significant difference from all of those three categories.
Now, this actually calls for the further analysis.
So what you will be doing is really comparing level with bank level with low level the 10th.
And then you will find out which one is actually different so that you can get an idea what you will
be doing.
And you can simply do that using this.
So that does not involve you can easily find that out.
So this is one Vilanova.
And similarly, you can try this out for much different multiple variables.
So I have considered like one good.
You can pick up any of this activity and then walk on it and then test.