In this session, we will discuss the vote feature selection.
So before actually discussing the feature selection, let us discuss about the type of problems we will
be dealing with and the type of data which we will be having for these kind of problems.
So the problems would be of two types.
One problem could be where we are trying to find out a continuous value.
Just taking a moment when you are trying to find out about the amount can have value like one to two
thousand three hundred or.
Nine ninety nine or one million or one or any amount would be and value.
Now, this amount value can be an individual, can be a floating point, so that is a continuous value.
But does a numerically.
Another type of problem would be that we are trying to find out if a loan has been approved or not.
So in this case, the two types of values would be either the loan has been approved or the loan has
not been approved.
So if the loan has been approved, we can have a value one.
If the loan has not been approved, the value or.
So we have these two types of problems now when we are solving a problem in supervised learning money,
and that is we have two types of things.
One is the predictor.
Predicted is the value which we are using for finding out a relation between these values so that we
can derive a formula between these values, I'm using that formula.
We can calculate the amount.
It is simply just like they call us an anomaly and let us see in a normal equation what we will have,
you will have some excellent value, some extra value, some X we value for value in X Y value.
And we can form a formula where we can have something like B X one plus two into X two plus B three
in the X three plus B four into X four plus B.
I even do X five is equal to the value.
Similarly, we can create a formula such as the one x one does that with two plus three x three plus
for export plus with the five X is equal to some probability value, which will be giving us if something
should fall into one or zero, if the probability is high, then we can see it to be one.
If the probability is close to zero probability of having something is low, then it will be a zero.
It is something like, let's say, the probability of the loan getting approved.
Is 90 percent, then we can say the loan will be approved, then we can put the label, we can put the
glass as one.
If the loan will not be approved, the probability of loan being approved is six point five percent.
That means the law will not be approved.
So in that case, the value of approval will be close to zero or.
So here we are calculating the probabilities and based on the probability we are deciding if will be
or one.
So these are the types of problem which we will be having.
Now, all of these problems with the aid, certainly gorditas, and we will use those algorithms to
calculate the solutions of these problems.
And to find out the very values based on certain algorithm, based on certain formula which will apply
on these X finds now repeating again and again the films which we use for X and Y, the domes, which
are used for X, are independent values because we want X to be independent from each other.
We don't want X values to be dependent on each other.
We don't want the age to change according to the dependent value or to change according to the gender
values.
These values have to be completely independent from each other.
Another thing is these values are also gold features or attribute.
These values are also input values.
Then the values which are the target values or the liberal values, the values which are output values
to which we want to find out our food labels predicted values that values or output classes also sometimes.
So these are the labels which are used for the life value.
Life value is what we want to find out.
I'm X values are the values which we are using to find out the Vivan.
Let us get further.
So why do we need feature selection, so we need feature selection because of the course of dimensionality.
Now here, when I am seeing these details set in this day, I have one body which I want to predict.
And I have certain values now when I have only five columns for my X values.
The problem is pretty simple.
Even you can solve this problem by simply calculating by hand.
So it is easy to solve when the volume and the number of columns are less in number and it is also less
complex for calculation.
But when we have a lot of columns, when we have a lot of features which we use for calculating these
values, then the complexity of the problem increases.
Then the amount of calculation which is needed also increases.
This is the reason why we don't want to have any irrelevant column which does not actually provide any
useful information, but only increases the complexity of the problem.
So as the dimensionality of the future space increases, the number of configurations can grow exponentially
and thus the number of configurations covered by an observation decreases.
That is the number of rules of data.
The ratio which we are having of the rules of data with the number of columns is also decreasing because
the rules of data will still remain the same.
But the number of columns which we are having been keep increasing and then the number of X values increase.
We need to have a sufficient amount of rules of data so that we can find out an adequate way for that.
So this is why we need less number of features, and that is why we need to select some features which
give a good amount of information and nothing relevant is given by.
So what we are doing here is buying data modeling in relevant or partially relevant features can negatively
impact model performance when we have any relevant or partially relevant features.
What happens is that they then need extra information and extra piece of information would be needed
to calculate.
So a lot more complex, you're going to need to be used to consider those features on how they impact
the Vivan.
That is why we need to reduce these irrelevant or factually relevant jobs.
We don't want these features which don't give enough information, but only increase the complexity
of them.
So the most important aspect of data modeling is speech and creation and feature selection.
So what we do is we keep in consideration of different types of information, which we have different
features or barometers we have, and try to create a lot of features and viognier, creating a lot of
features.
The need of architect feature selection also increases because we do need the feature creation, because
we need to have adequate amount of information.
But then once we have created these features, we also need to select the features which are actually
important.
And there is an upcoming business rule which says that we want that models to be simple and explaining.
We lose the expletive deleted when we have a lot of features, so we want to have the models to be simple
in nature.
We don't want the work towards a complex model.
And the thing is, garbage in, garbage out, which means that most of the things we will have many
non informative features, for example, Naem or DVD booths, and these are for quality input, which
will produce whole quality output.
So in case we have some features which are irrelevant in nature, they just lead to a bad performance.
And if we have a lot of features and they do not have enough information, they do not have the actual
BIPIN, which we are looking for.
Then, no matter how many features we have, we will not have good results.
So this is the reason why we need to select the good features.
Now, what are the benefits of performing feature selection?
The benefits?
First, it reduces over.
Now, they have not really discussed what footing is now.
You can learn or understand overfitting.
By just.
Thinking about a simple example, that is if they are preparing for a simple exam and we have two options,
one option is to learn from the entire syllabus, and another option is to cram from the bus to your
question papers.
So what will happen is when we have learned from the past, you question papers, we will be able to
answer only the questions which we have launched earlyish.
So this causes overvoting, so that that is basically we have learned from us and obviously we have
learned from a certain set of questions only, so we will be able to answer only those specific questions.
While when we learn from the entire force curriculum, what happens is we go to each and every topic,
which enables us to answer the extra questions or some questions which could be asked in different.
So a model which loans from a lot of data and the model which does not just stick to specific topics,
it learns more vastly and performs better.
So we don't want anybody to learn from all these specific things, we want them to learn from a vaster
space so that it is able to answer more accurately, it is able to predict more accurately.
So this is what Overfitting is.
Overfitting is when something launched from a very small space and is not able to answer for a larger
space of data.
So when we reduce the feature, when we do a good feature selection, what happens is the more we will
not learn from something which is irrelevant in nature and actually focus more on learning from the
vaster space and more important things to learn, then improves accuracy because it is less misleading
in nature.
So that data is less misleading.
So it will improve the accuracy also.
And because we will not have extra data points to learn from, so we will have lesser training time
and again.
It reduce the complexity of model, which is the main thing.
We want to have a simpler model.
So that is what it will allow us to achieve.
Now, there are different methods of feature selection.
One is unique variable selection, another one is feature importance than the it is a correlation matrix
with Hedra.
So first of all, univariate selection is using the filter method for another method is basically created
using the selected best from the cyclone law library.
And this is the moppy, so if we have the features as a continuous form and the output is also cantinas
in nature, then we will use some correlation in case the features are contiguous in nature and the
response, the output is a classification problem, then we will use Hédi in case the input is categorical
in nature, but the output is continuous.
We will use ANOVA and in case the input is categorical in nature and the output is also categorical
nature, then we will use Chi Square from.
Now, the silk vest is something we don't use that often because we already have a lot more methods.
So this is something that you can explore more in case you want.
But we will go towards the major and more easier things which we can implement just like that.
So we will talk about those more in depth.