In this session, we will implement dimensionality reduction, so the first thing which we will be having
a look at is Deacy.
DNA is an algorithm which allows us to convert a very large dataset into a small dataset and into a
very less number of rules in a very small number of components.
Now, the companies generated by DNA are not as good as the ones which are generated by a PC, but the
is usually used for the.
Visualization purposes.
So in case we want to visualize the clusters in the data, then we may use as otherwise the most frequently
used one is VXI.
So for this, we will implement by importing Findus, no B.S. from Escalon decomposition scaley A.S.A.
from Escalon manyfold.
So for this particular problem, we will be using the digit data set, this is in handwritten digits,
data set.
So this dataset consists of different values, consists of small images, small black and white images
which contain the sixty four fixes that this.
Twelve across 12.
That is eight plus eight, which is so far this this is the dataset.
So here you can see there are different values available.
So when we combine these values, we actually get a digit on any of these values.
So now we will visualize this.
So for this, we can actually get the data and printed using flawed thought, I am sure, giving the
data into a great form.
So here I have rented one of the values.
So here you can see that this looks like five.
Next, we will have a look at.
Inthe.
Here you can see that this looks somewhat like one we will try.
40.
Which looks like seven.
Next, we have.
Two hundred.
Which.
Not looks like one, but it is one.
Next, we have.
See, 40.
So this one is an eight, so you can see this is somewhat highlighted towards the end.
So this is what we have in the images.
So now we will import Bazzani and from the assembly, we can basically run Disney by giving the number
of components.
I need an them stage value, and after this, we will get the.
The Senate object and begin simply to foot in France, one on top of that and get the data frame from
this, this data frame will now contain the transformed values.
And we are adding the digits from the biodata frame which we had created.
This is the date of which we have obtained and on visualizing the ideas.
And we can see here, these are the digits.
So here you can see these other clusters which have been formed internally, so here you can see that
the value six is present here while we have the value nine.
Value zero here.
This is the value one.
This is value for now, you can relate how one in four looks somewhat similar.
That is why these points are closer to each other.
Here we have green, which is to.
Here we have three.
With just seven and it has a few points of nine.
And growing closer to it so you can see how the numbers, which are closer in shape, are presented
near to each other.
Now we will see how we can implement this.
So far, Disney, we only have to do with we will simply both Disney and the Woolverton transform on
top of it.
Now the next is FXR.
So let us see how we can implement this.
So for this year, we will import Escalon decomposition and we will import, in fact, that analysis
from we will get the frame from the same handwritten digits data set and we will see the correlation
matrix.
So here you can see the central values are actually highly correlated.
And we will then have a look and we will scale the data after scaling the data, we are taking out the
principal component.
So the principal components come out to be 30.
So now we will have a look at this, we will fit the beast on top of it, and these are the key components
of a generator.
Now, these components have a shape, 40, 60, for that is 30 values for those sixty four columns,
which we have.
Now we will find out the BCA explained valiance ratio, so this will show how much variance is explained
by this particular PC.
So now here we are applying a cumulative sum and broadening of the PC explained variance ratio.
And this will give us how much variance is explained by a combination of factors.
So here you can see that the first component is able to explain twelve point six percent of variance.
The first two components are able to explain 20 percent plus three components are able to explain.
Thirty three percent plus four components are able to explain 42 percent next 48 and so on.
Now, here, because this is an image data the BCA explained is very less in comparison to the usual
cases which we will be seeing.
So what we will be doing for that is we can we will select the top components and using the top number
of components, whichever we want to have.
So here we can select 11 or five or 20, whatever we want, and then we can apply photonics.
So once we will do photonics, then we will get the details of what how we need to apply.
And after that, after the speaker has been trained, we can simply do a backdoor transform.
And with these transform, we will be able to transform our dataset from sixty four columns of data
to one less number of columns of data here, which I have chosen 11 columns.
Going further, if you want to load a single component, then we can simply see a component and the
index of the component which warns would be one of the first components, we will be looking for the
index.
Now, let us try to implement this for another dataset, which is having numeric values, presenting
them.
So now we are picking another data set, which is existing base, so we have read the CSP file and we
have filtered out the columns from the object database.
So we don't want objectivity.
We are wanting only the numeric columns.
So we have filtered out the columns.
So this is big drop, so we have dropped all of the columns from this, the next thing will be we will
run the algorithm on top of it so we can simply run it like this.
So these are the components which we have received from this.
And now we will find out the explained variance, so this is the explained variance and this is this
some of the variance explained.
So now here you can see that we have 30 percent explained by the first one in forty seven point five
percent explained by the force to Compellent and so on.
So this is how we get to decide how many components we want to have full of it, 90 to 95 percent variance
explained is good enough so we can give zero.
One, two, three, four, five, six, seven, eight, nine, so we can keep all 10 components out
of it or 11 components.
That is completely up to us and just how we did earlier.
We can simply do the first on the principle component by providing the number of components we want
to have finally and do a fit and transform to get the updated data with less number of columns generated.
Now let us have a look at the actual size of the Web, so let us have a look at the actual size of the
data frame.
So the actual size of the frame was.
They need or shape.
So Sudi.
So it was 32 columns and we removed a few.
So let's have a look at the.
Size of the state of frame.
So here we can see that we have 18 columns now instead of 32 columns, so we have reduced the number
of columns.
And similarly, when we have a very huge dataset in that case, we will be able to reduce a lot more
number of columns and help us a lot in reducing the complexity of floodwaters.
So I hope you will be able to implement these algorithms and maybe use it in any of your projects,
which you will be working on.
And I hope this will be really helpful to you.
Thank you.