Now that we have discussed about different unsupervised learning methods for clustering, now comes
the dawn of a very important algorithm, which is dimensionality reduction.
Dimensionality reduction helps us in reducing the number of columns and appropriately features, selecting
now feature selection and damage.
Double digit option.
Do the same thing, but in a different way, because in case of feature selection, we already have
several columns and several attributes, but we only have to select some of them and drop other columns
from the dataset while in case of dimensionality reduction.
We completely transform the dataset from, let's say, 80 columns or hundred columns or three hundred
columns into all different data, set off a smaller number of columns, see 11 columns.
Now, the amount of information which will be present in the original dataset with eighty or a hundred
or two hundred columns, will be similar to the amount of information which we will get after the transformation
using dimensionality reduction.
So in feature selection, we use to select a few features from the entire dataset.
While in dimensionality reduction we convert the data into a new fresh dataset where we have the same
amount of information.
The information which we have in the original data is intact, but the number of columns has now reduced.
Now, because the number of columns has reduced, it is less complex to use them, and it also reduced
the time consumed for training, different it.
So let us see for the.
So what does feature engineering feature engineering is the process of creating features so that we
can use them for machine learning algorithm training.
So feature selection is one part of feature engineering where we actually select different features
from all the features which are already present.
Now, once Abraham Lincoln said, give me six hours to chop down a tree and I will spend the forest
for sharpening the axe, the above quote has a great influence in machine learning to when it and when
it comes to modeling different machine learning models most of the time needed to spend on the data
processing and feature engineering stage, where we actually transform the data and create new problems
and select good columns from all the columns which we have.
So what this course of damage to the incomplete data with few features led to the development of the
common myth.
So when we have incomplete data and when we have few features, then we think that our model is not
working fine because we have less amount of detail, we have less number of features.
But that is not true.
Having less number of feature is not a problem.
Having features which does not have good information, good quality of information is the problem.
So having more features and more data will always improve the accuracy of solving the machine learning
problem.
So this is the myth which is there.
But in reality, this is of course, more than a game and a lot of features with very few data points
in this case for doing a model in this scenario often leads to low accuracy model, even with many features.
This is called cause of dimensionality.
That is when they increase the number of features reserves in the decrease in model accuracy.
So increasing number of features.
In Greece, the model complexity that is more precise, the model complexity increases exponentially.
So basically they don't want to have such large features because we want to have a smaller feature space,
some good features which will be able to explain the problem, which will be able to represent the actual
information instead of having irrelevant to hundreds or hundreds of columns which don't really help
us much and only increase the complexity and increase the calculation time.
We don't want that.
We only want some few good quality features.
So what how can we do that?
So there are two ways to stay away from this course of dimensionality one.
Add more data to the problem, second, reduce the number of features in the data, now adding more
data to the problem is not possible because we have a limited amount of data.
And if we have data, then we would have already used the data.
But we have some limited amount of data.
But the feature space is so large that we don't know how we can reduce it.
So I think the data may not be possible in many scenarios.
Hence reducing the number of features is more preferable.
So for that we have a technique which is known as dimensionality reduction.
So what does feature engineering and engineering allows us to identify the influence features over all
the available features, so the identified features used to train the model.
So identifying the influence feature or the influential features does not mean picking the features
in an analytical way.
So we don't it doesn't mean that if we have analyzed and found out some features, then we have done
an analytical way.
Something would be done in a different way also where we can actually do some conversion and get the
meaningful features out of that.
So without really applying the different methods which we have been using till now, we can actually
reduce the size of data by using dimensionality reduction without applying any analytical knowledge,
without applying different concepts like correlation, value, finding out multipolarity or ViiV value
or applying find us providing.
No, none of that is really required.
What we can actually do is we can basically use them as a digital.
One famous approach for dimensionality reduction is principal component analysis, so of what we will
be doing is we will be using PVC for now.
So what is Fessey exactly?
So let's imagine a simple problem where we have all the gold.
We are recording the motion of the spring here.
This is one spring or pendulum.
And this moves enough of one single direction in this direction in which I am pointing it out right
now.
Now, in this particular motion, if someone is not really aware of where they should check this particular
motion, whether someone should check for this one, what they will be doing is they will apply different
cameras at different angles.
And now applying cameras at all these angles will not really help.
So if someone knows where the camera has to be, then they will apply three cameras and actually be
able to capture the emotion.
But if someone is not aware of the exact direction of motion, then they might be applying a number
of cameras to record the moment with at least three cameras.
And we will play some different or cameras in case we don't know what cameras we need to apply.
So we will keep on giving cameras to capture this motion, but we will not be able to capture the motion
entirely or we will need a lot more number of cameras to capture the motion.
If we don't really know that we need to apply these cameras at 90 degree angle with each other.
So what we will have to do is how what we actually get from this is that a camera at 90 degree angle
will be able to capture more information in comparison to cameras placed at different locations.
So the same thing is what we will be doing in Keisel.
So what we do is in B.C. a technique, it transforms the odd features that bizarre cameras into new
dimensional space and represents it as a set of new orthogonality water onto one of the variables or
one of the variables are the variables which are perpendicular to each other.
So we already have some dimensional space.
We already had a feature space which had multiple dimensions, but all of these dimensions were capturing
the data in different, different ways.
But because these dimensions were not really at the 90 degree angle and they were not able to capture
much of the information.
Now what we did.
The applied B.S. I'm using this year, we created another dimension, another dimension of space where
all the dimensions were at 90 degree degrees.
Now, these 90 degree angle were able to capture more information from different dimensions then they
were actually able to capture.
So what happened is these orthogonal features are actually called principle components.
So these orthogonal variables will be able to observe the problem with reduce the cycle features.
Now, in practice, our data is like the motion of the pendulum.
If we had complete knowledge of the system, we will require a smaller number of features.
Now we have to observe the system using a set of features which will convey maximum information if they
are orthogonal in nature.
So the maximum information will be captured using the orthogonality.
So this is done using principle component analysis.
The new set of features which are produced after a transformation are minutely uncorrelated as they
are or to woman.
So the features will be created, will be linearly uncorrelated, that the features will be independent
in nature.
So the thing which we were trying to do earlier with all those columns is actually being given to us
and with a lesser number of columns, basically in terms of these principle components or the orthogonal
features, which principal component analysis will be giving to us.
And on top of that, getting these features is already good enough, but are also getting these features
in sorted order, and what is this sorted order?
This sorted order is we will get principal components.
So let's see.
We get then principal components out of all the hundred columns which we had have.
Now these principal components will be sorted in in order to be the first principal component will be
able to capture the maximum amount of information, then the second component to recapture another amount
of information, which will be at next quantity, quantity ways and then lowering it.
So what we can do, we can select the top three, top five of seven principal combatants, which will
be able to capture most of the information, the amount of information we actually want them to capture.
So it will what will it do?
The first principle component alone will explain a very large component of the data.
The second principle component will explain less than the first component, but more than all the other
components, the last principle component will explain only a small change in the data.
So we run back into the top principle components such that they together explain most of the data.
So in most analytical problems, explaining 90 to 99 percent of the data is considered very high, so
we will select, convert the data into principal components and then select the top few principal components,
which will be able to explain most of the data.
Now what it actually looks like.
So we will have some data.
Let's say we have one hundred columns and they were able to explain hundred percent of the data because
that is some data.
So the data will be hundred percent in those hundred columns.
Now, we have applied principle competent analysis and we have received these then vincible component.
So each of these principal companies will be able to explain some variance, what is this variance?
This variance is the amount of information.
So the first component here explains 40 percent of the information.
Next component explains almost 20 percent.
Next component explains a little less and so on.
So now we will try to get these components.
So I will pick the first component.
It gave me what people, when I consider another component, don't along with it.
That is, I converted a hundred columns and I got these 10 columns out of those hundred columns after
transformation.
And now when I'm considering one column, I get 40 percent information.
When I consider two columns, I get almost 50 percent information.
When I consider almost these three columns, I get almost 70 percent information.
And like this, when I go to the seventh columns, seventh principle component, it is able to explain
more than 90 percent of the information, which is exactly what they want.
So I think these seven components out of the thin components which were created for me, I now I have
captured the information which was provided by one hundred columns earlier after transformation and
selecting these principal components, I have reduced the number of columns from one hundred to seven
here.
So now I will use these seven columns for my model generation.
There is only one drawback which I have here, which is these columns will not have any legal associate,
so these columns will not give me the column name.
Earlier, I had column names like Age Silalahi, the number of children, the number of number of houses
they own and different divisions.
But now these companies have the information.
But I cannot say that this is the age criteria.
This is the salary criteria.
No, because the agent salary and all those details have been somehow distributed between these seven
components.
The information is present here, but I don't know how it will be represented.
It is somehow fixed and created and combined and presented in this way.
So this is a very useful and very nice implementation, so it is something which has done wonders for
us.
So now, instead of walking with all those features, if they are not really concerned with all the
features, we can simply convert them into a principal component and use them.
But the only drawback which we will have is we will not be able to explain the relationships.
If you want a Blalock's model and you want to create that blackbox model with less complexity, just
get the data, do.
P.S. apply in principal component analysis, get the components and apply the regression or classification
algorithm on top of it.
And you're go to.
No need to do different things like data transformation and then feature selection selection, that
all is not required.
So this is something which is really helpful, but it comes with its own good idea that you will not
get to know what is explained inside this.
You just get the features, used them and stay happy.
So here are the domes of a few domes like covariance and what is cool readings.
We have already discussed this.
So cool radiance is of here.
You can see the value is decreasing.
So this has a large negative covariance here.
We have near zero readings and here we have a large positive comedians.
I hope you already know this.
When greater value of one variable mainly corresponds to lesser value of other.
This is covariance has negative.
If the sign of convenience shows the tendency in the relationship between the variable, the magnitude
of the winds is not easy to interpret because it is not normalized.
Hence depends on the magnitude of the variable.
The normalized version of comedians is called correlation coefficient.
However, shown by its magnitude the strength of the relationship.
So evidence provides the direction of the relationship, like the correlation provides the magnitude
of the relationship.
So these are the details about the principal component, that is how do we transform a given set of
features into new features such that they are orthogonal?
So the answer for this is Eigenvectors of the Matrix.
So we know that eigenvectors are to talk to each other.
So transforming our features in the direction of eigenvectors will also make them or the one before
transforming the Matrix.
It is always recommended to normalize, so always normalize the data and then apply.
B.C..
So if the matrix is not normalized, our transformation will always be in favor of each other with the
largest scale of values.
This is why I became sensitive to the relative scaling of the original variable.
So it is very important to normalize the data and then we will apply this year on.
Now, why do we need this?
We need this to be political linearity, to reduce the number of features and to be able to visualize
the data.
Here you can see one diagram.
So here we have converted a huge of dimensional data into just two dimensions where we can easily find
clustering.
So we are able to visualize the clustering because the data has been transformed into two dimensions.
So this is about PC, so I hope you will be able to use PC very nicely.
So we will discuss the coding for dimensionality deduction using PC in the next session.
So I hope it will be a great learning experience.
And with this, we end the sessions for the general topics which we have in machine learning.
Apart from this, I will be adding another video for The Matrix, which we will be using for the clustering
algorithms.
So thank you very much.
The next thing which you have to do is work on the project and I will provide proper guidelines on the
project.
How you need to work on the project and I hope you have been working on the assignments and the assessments
and performing really well on it.
So I have a great learning time and I hope you will learn a lot and grow a lot in machine learning domain
and have a great future in the scene.
Thank you.