Hi.
Now that we have covered all the supervised learning algorithms and learned how we can actually create
a pipeline out of them, let us have a look at the next wave of machine learning, which is called unsupervised
learning.
Then now we have learned about supervised learning algorithms.
Those algorithms allowed us to make predictions about the data.
So in those models, we basically gave the input value and the output value to the machine to learn
from it so that when another input data will be given to it, it will be able to make predictions and
find out the output values again.
But in case of unsupervised learning, we don't need that output value.
We don't need those values now because we don't want to predict the anything.
We just want to group things together.
We just want to form clusters here.
So let us have a look at this.
So in case of unsupervised learning, I mean, it is finding patterns in the data.
So supervised learning, needed output or target values to be provided as it wanted to predict the specific
value or label, such as guide dog prices, what kind of fruit dessert?
Is it a mango or is it a banana?
So it wanted those details because it actually had to predict that.
So we used to give our entire weather data to the machine so that it can predict what temperature will
be tomorrow or if or if I should play or not.
So these kind of decision we had to make these kind of output values we had to provide.
But now unsupervised learning does not require any outward values because it does not want to predict
any labels.
It only requires input values and it wants to group the items.
So we even give several input data here.
And based on this input data, the machine will try to find different patterns and segregate these elements
into different groups.
It will just group them together and it will not really know if this is an apple or this is a banana.
So it will not know what is present in the information.
It will only know how to segregate these on the basis of the pattern which is present.
Now, why do we need unsupervised learning?
We need unsupervised learning to create more focused marketing company.
So let's say I have a new product coming out and I want to find out which people will actually be interested
in the coffee product.
So for that, I will have to find out different groups of people.
I'm their likings and I will have to segregate people on the basis of their likings and based on their
likings or based on their daily activities.
I can see that these kind of people will actually be interested in coffee or these kind of people will
be interested in.
So we will find different thighbones and different characteristics about the people or regarding the
product so that we can categorize them differently.
So let's see.
I have some details about the products.
Let us see.
I have some products and I get to know that something is in store level or not in-store label if something
is a hardware or a software.
If what memory space something needs, so based on all these days, I can actually classify, I can
actually cluster things together into different types of products.
I can get a cluster or group different type of products together and create a bunch of them.
So we can find clusters of data, pardon's in groups, similar items together, we can do product categorization.
And next thing is anomaly detection.
What anomaly detection helps us in achieving is that is legacy.
We have a daily transaction and we have a monthly transaction of Safian thousand rupees.
Now, if somehow that is one month, when I make a transaction of one like rupees, then it creates
a red flag for my bank.
And my bank will immediately communicate to me that there is a very high amount of transaction which
has been made.
So this amount will actually be waiting for every person based on their normal transaction.
Hence, anomaly detection is the process of finding out some different vibin or something which is different
from the normal behavior of someone stato next is dimensionality reduction.
We have discussed about the feature selection in case of supervised learning.
We have been working on feature selection by different methods like using feature importance by removing
columns based on different ideas.
But now there is the need or dimensionality reduction.
And let's say we are not able to remove columns based on all of the midterms, which we have known better.
Now we can use dimensionality reduction to reduce the size of data to a very large extent.
So how we will do that, we will be launching it in unsupervised learning.
Now, let us see some few important things which we need to begin off by walking on unsupervised learning
problems.
So the first thing which we need to consider is scaling of data scaling or data is also required in
case of.
Gannon and SBM, because there again, the information is being dealt with on terms of distance, so
it is important to scale the data in Gannon and SVM and in all the distance based algorithms which are
majorly present in the unsupervised learning.
So why do we actually need Killingworth data?
Now, let us see if the data is normally or uniformly distributed, then standardization is a suitable
method.
Now let us see what happens here.
So let us say this is an on normalised house data.
So here we have data about the years or is of how much years old is the house.
And here we have details about the numbers of rooms now based on the data which is present here.
How can you see that the data is distributed?
We can simply say that the data has a horizontal distribution and points us in a horizontal.
But if we normalize the data now here, if we want to find out the distance or we want to see how different
number of rooms are impacting or how yours are impacting some particular value, then it will be very
difficult to find us based on the number of rooms because there is no change in the number of rooms.
The changing number of rooms is not prominent enough.
But if we normalize the data, this is the normalized data here.
The range of the data has been changed earlier.
The range for number of rooms, four zero two hundred and the years of old was zero two hundred.
Now, here, the values of the houses have been changed and the number of rules now ranged from zero
to one, and the years old range from zero point zero zero point four.
Now you can see how the data is really leaning.
Now we can actually find out two clusters that this type of data is different and this type of data
is different.
Now, if I want to find out where this point actually lies.
So this point is very far away from these points, but it is.
There is no impact of the number of rooms we cannot see anything which is coming out of the number of
rooms here if you want to find out any distances.
If I want to learn something about this data on the basis of a number of rules, it is very difficult
here.
But if I want to have a look at this particular data on the basis of number of rules, I can easily
see that for this one, the number of rooms is zero point five.
So I can easily understand this based on this normalized data.
And see, in fact, comes in picture when we have we are dealing with different machine learning algorithms.
So what will happen is if data is not normalized, then one particular column will have all the impact
on the decision which would be made by the column, which has a very small range, really get diminished.
The impact of it will be lost.
So that is why it is very important to scale the data now in case the data is normally distributed,
then we will follow the standardization procedure.
And if the data is not normally distributed, then we will go for normalization of the data.
Now you can see the changes which normalization and standardization will bring.
So if this is the actual data, then normalization will try to bring it from between zero to one, bring
the data between range of zero to one in both X and Y axis, while in case of standardization, it will
try to bring the data between one standardization that this.
Between minus one and plus one in the axis.
So this is the difference between normalization and standardization.
So you can simply have a look at the distribution of the data and based on the distribution of data,
you can decide if you want to do standardization of the data or normalization of the data.
So here are the formulas for that.
If you want to do normalization, then normalization can be done by X minus Xman, divided by X, max
minus X Y.
Standardizations will be done by X minus you, divided by Sigma.
Well, new is the meaning of the data, and Sigma is the standard deviation.
Now, here you can see how that changes, so here we have an original data.
Well, we have different columns with different ranges, so when we normalize the data, you can see
all the data points to bring growth to the same range.
So now the impact will not get diminished.
So if we do normalization of this data, then the impact of the first, second and third column, which
was getting diminished, will not be diminished.
So this is how normalization and standardization helps us.
Now, next, we need to know what clustering is, clustering is a process to create groups based on
similarity measure.
OK, so what is similarity measure?
Similarity measure is different criteria on the basis of which these groups are similar.
So let's say we have some fruits and vegetables and we want to find out the similarity measure.
The similarity measure would be the taste of the item which we have or if it is a juicy fruit or vegetable.
So there are different tribes.
So based on that, we can decide.
So let's say we are deciding on the basis of taste, then apples and potatoes will be brought into the
same category because that is just using.
So principle of maximisation of interest, lustrous similarities and minimization of investor similarity
is the only principle here.
So what we want to do is we want do group items together.
So how will we roll them together?
We will try to maximize the distance between two different clusters and minimize the distance between
the points or data points which are belonging to the same cluster.
Now, here we have a grouping of details of how we can grouplove data.
So let's say we have these data points so you can easily see that for these data points.
There are two groups present.
One is this particular group and another is this particular group.
So here we can easily group this data into one and two groups.
But now let us say, if we want to have more number of groups, then what will happen?
These are the two farthest groups so we can create these two groups.
Now, let's say I want to create four clusters out of this data.
So create two to create four clusters out of the data.
The farthest ones will again be subdivided, will belong to two different clusters.
Now, these points belong to a similar area.
So these will be clustered together.
And these three points are a little separate from these points, so it will be considered in a different
class.
Similarly, these three points are a little far away from these two.
So these will be considered in another class.
Now, let us see if we want to include more number of clusters, then these points also seem to be different
from each other.
They still have a little distance between them.
So we can have the number of clusters so we can create six clusters like this.
And if we start to find out more a number of clusters, if we start to find out more number of clusters,
then it will kind of create clusters which don't really exist also, because now it seems like the data
has been clustered properly.
Now, one more thing is here, if you see this particular data.
Here, when we see this particular data legacy of this data actually was in the form of.
This data.
Then think about it, would you be able to find the clusters it will have been.
It would have been very difficult to find out clusters from this kind of data, while it is very simple
to find out clusters using this kind of data, because now the data is present in a similar scale.
Now the state data has been either standardized or normalized.
So the distance is clearly visible here because the distance was not clearly visible, because the scales
were so different, the points were overlapping.
So no clustering could have been possible in this particular situation.
That is why normalization or standardization is so important.
Now, you can see here.
We have these three clusters created, so the distance between this and this line is called in the cluster
distance that is distance between two different clusters and the distance between points in the same
cluster is called an intra cluster distance.
And what is the definition of clustering, the principle of maximization, of the cluster similarity
and minimization of the entire cluster similarity, which is we want to minimize this particular distance
and maximize this distance?
That is exactly what we have done here by creating the flusters.
Now, let us have a look at a different type of distance findings, so one is missing the link.
Single link is to find out distance between the closest points when we find out the distance between
the closest points, it is called single link distance.
If they want to find out the distance between the farthest point, then it is called a completely.
Because in this view of finding out the distance between the farthest points, the points which are
farthest from each other.
Here, average linking is when we want to find out distance between the average of all the bears.
So what we will do, we will basically consider all the bears, all the bears.
But as NPR's so I will bear this with this also this with this also this with this.
So I will create different bears and doing Albrecht's average of all the.
Once President S.A.G. distance is when I will find out the distance from the centroid, like this so-called
Synthroid distance, so I will create the same idea and find their distance from the centroid.
This is about unsupervised learning, I will be discussing different unsupervised learning algorithms
from the next session.