The next model, which we will be discussing about is K nearest neighbors.
But the name you can understand that this is about nearest neighbors.
So here we will be considering the data point and from the point we will find out the nearest neighbors
of that particular data point.
And based on the nearest neighbors, we will decide what to us should the particular point should be
assigned to or what value should the value of the particular data point be getting.
So what value should be predicted out of that particular point?
So let us go.
So here is the number of neighbors, so let's say we have this particular data and we have assigned
to be three.
So in this particular data as gays three and we want to find out the class for this particular study,
then we will consider the majority point that this this three, hence this red star is actually belonging
to these four people.
Plus that is Class B.
Now, if you consider me as six, then it will fall into this particular region, that is the nearest
six volumes.
Now, when we are considering the nearest six in this scenario, the majority vote is of these yellow
classes that this class.
So now we can see that how the nearest the number of the key value decides what should be the value
of the point, which we are looking for.
Now, the small value of K passes over everything that is due to learning the localized five.
It is not able to find the general bargain.
So that is the reason why, because this point is looking for a very small number of key.
That is why it is not able to identify that this point is actually belonging to Class E.
Next is when we have a very large give and then it will become highly generalized.
I mean, Mr. Complex bought it.
So let's say that the nearest key value is actually a very high value system.
So in that case, the nearest ten point would be one, two, three, four, five, six, seven, eight,
nine and 10.
That is all of the values.
And then we take a majority vote on the glasses actually have five five point.
So we will not really be able to analyze what this actually means.
Hence, we need to find out a value of care, which is not very large and not very small.
So we have to make sure that the value of K is acceptable.
So for this, what we usually do is we find out the results for different key values.
We will be finding out the results.
Work is equal to one, is equal to two, is equal to 10 or 20 based on the amount of data that we have.
And then we will compare that which particular value gives the best result, and that is the key which
we will be selecting.
Now, let us look at a few properties of Ken.
So human can be used for both classification as well as regression, predictive problems.
Now, when you want to classify so in case of classification, we will take the nearest key and take
a majority vote of the point around it and then decide which class to the point below.
When we are working with regression in case of regression, we will be considering these points and
taking a weighted average of the values.
A weighted average will allow to regularise the values because some values might be close and some points
can be fired up.
So a weighted average or weighted average on the basis of distance will allow to modulate the value
in case of finding out the regression values.
It is mainly used for classification, predictive problems.
Why is it mainly used for classification, predictive problems?
It is mainly used for classification problems because it can consider that it is a distance based in
Wertham and it might ignore a few buttons because it is looking for the nearest point.
So it is looking in a circular region around.
Now, it is called a lazy learning algorithm devised by the lazy learning algorithm because there is
no specialized training phase.
In this gordito, it uses all the data for training classification, so it will not need explicit training
period, when we are actually making a prediction, we can simply provide all the data points and these
data points.
It says we act as the training.
What if the removal of your data point, then it will be classified differently.
And if the other data points, then it reclassified different.
So that is why it is called the lazy learning called.
Then it is known the Matrix learning algorithm, that is, it does not assume anything about the underlying
data.
And it finds the it's based on closely matching data points that it will consider the closest data point
for this particular criteria.
And because the closeness is measured by the distance, which is the reason why we will be scaling the
data, so we will scale the data so that.
The distance will be equated for different types of features, so because there could be a feature named
age and they could be another feature amount now age will be ranging from, let's say, zero to hundred
and the amount might range in lacson.
So in that case, we need to have skinny.
Now, let us discuss a few pros, so in order to uncomplicated and easy to apply in an atom is uncomplicated.
It is not complicated in nature.
It is very easy to apply.
So you can simply see if we have the data points and we know the distances, then we can easily apply
the algorithm by hand and find out what the point should be.
Then there are only two metrics to provide the algorithm value and the distance metrics, so.
All we need for this algorithm is the value of work and the distance between the points so that we can
just decide on the basis of that how many points we need to consider and we find the nearest point and
then make our decision.
Next walks with any number of classes, not just binary classification.
So in this case, we might have any number of classes and based on that, we can just find out the distance
and the majority number of classes and then we can be able to go.
We can simply use this algorithm for five classes or ten classes.
Then it is easy to add new data to the bottom because it is a lazy loner, so it does not need any specific
training so we can easily apply more data to bellbottom and then we will be able to analyze them according.
Now, let us just discuss about a few points, so it is computationally expensive, right, because
it will be finding out the distance between all points because it is a distance, basically, and it
will have to find distance between all the points and then it will have to find the nearest distance
of all points from this particular point, and then it will make their decision.
That is why it is computationally expensive.
Next is it is having a high memory storage requirement because it will have to save all the distances,
that is why it will have a high memory storage, then it is hard to work with categorical features.
So let's say we have certain categorical features.
Now we have categories.
We have values like age and amount.
In that case, it is very easy to find out the distances.
But let's say we have classes.
One is gender and another class is.
Let's say if someone is married or not, then there will be a very difficult to find out.
What is the distance between the what?
Next prediction is for a big number of features, that is for a big number of features, it will have
to find out the distance based on all the features being considered.
So the dimensions of the distance which will be found out, would be very high, which is why it will
be a little difficult to work with these and also.
The sensitivity towards the scaling of data is very important and also there might be a presence of
irrelevant features which might tweak the distances.
So let's say we have certain data points like is an amount and we do not scale them properly, then
they might get biased towards the amount and we might not really consider the changes which are caused
in the age of the person.
So that is why it is very important to make sure that the data is.
So this is about an.
Next, we would learn how we can implement in.
And always remember, Kevin is also always learning algorithm, they know what everybody seems to have
learned are all supervised learning algorithm.
That is another algorithm named Kamins, which is an unsupervised learning algorithm, which we will
be learning very soon.
So people usually get confused between the nearest neighbors and the key means algorithm.
So we will be discussing the comparison and we will be visiting again and again while we will be discussing
gaming's so that there is no confusion between both of the constant.