In this session, we will discuss about boosting, so let us have a look at this particular.
So here I have imported the packages.
So the first package which I have imported is Mumfie, then Fondas, then McGlothlin, then Seabourne.
And I've imported oil.
Next, I have ignored the warnings so that I don't get extra warnings to.
Here, I've loaded the data set, so my data safe is house prices and train and house prices underscored
this dataset and here I am checking the shape of the data using the chip function.
So training data has one thousand four hundred and sixty rolls, while testing data has one thousand
fifty nine rows, four hundred fifty nine rows, that retail training data has eighty one columns and
testing data has only eight columns.
This will actually help us to find out which is the feature, which we want to predict here.
So I have created a set and removed the training test column from the print column sick.
I could have used all the difference also.
So that would also have given us the value of the missing column.
So the value comes out to be see price, so we have to predict these sales price values, so because
we are predicting sales price.
So it is clear that here we want to solve a regression problem.
So we will be using extra boost repressor for this particular problem.
So whenever you are used to solving a regression problem, you will use it agressor.
And whenever you're solving a classification problem, you will use to classify it.
Each and every library in Ashkelon provides the details of the classified and agressor, the corresponding
hyper parameters and all the details about them and all the things will be in sync with what we have
discussed in the session.
So there should not be any problems in exploring these photo.
So let us have a look at this, so we are checking the prediction column, this particular column,
Visages Seed's price column.
So we have applied describer on top of it.
So here you can see that the minimum value is thirty four thousand nine hundred.
Maximum value is seven fifty five thousand.
Here we have twelve twenty five percentile, which is twelve thousand one twenty nine thousand.
Then we have 60 percentile that is desired, 16 one like sixty three thousand seventy five percentile
at twenty one thousand two fourteen thousand.
So you can see that the maximum value has a great gap and so does the minimum value.
So it shows that there are certain outlier values in the cities, prices which are also present.
Now we will have we will prepare certain plots, so the fullest plot is of the living area, the ground
living area of in comparison with those cities, places.
So here we can see that the living area has seen house prices are having a positive relationship.
And as the living increases decrease, prices also increase.
And after a point of time, the there are these are a few outlier points.
And out of these boys, all the boys look in sync with the relationship, but there are these two data
points which actually do not follow the relationship between where the living area is very high, but
still the price has been set very low.
So this is something which is unusual in nature and might actually impact the performance.
So we might decide to get rid of these two data points because these will actually impact the performance
of the model and will try to divide this model towards these two data point.
While here, you can see for a given area, the prices here are very high.
And here again for the these prices are very high in comparison to the usual prices which are there.
So these are, again, certain outlier values.
Next, we will have a look at data preprocessing.
So first of all, we will replace all the numeric missing values.
That is not a number with the zero and non numeric with none.
Then we will create dummy variables for the categorical features and transform the Spearwood diplomatic
features by taking the log of feature plus one.
This will make the features more normal in nature because here you can see that the values are Sadikov.
So we want to remove Desco from it so we can take a log and do that.
So first of all, we are filling on not a number in the values.
So for the training ID you can see we are creating or train ID and from the train dataset we are dropping
the column IDA because it is not the important column and it will not combine the prices as we have
already discussed, that application numbers, IDs, roll numbers have no impact on the predicted variable,
which we are trying to find out.
Next, we are removing the column from the first dataset.
So if you remember, we have done this kind of transformation only a also, but then we combined the
cleaning and testing dataset.
So this is a different way of doing it.
You can either choose that particular way or this way here.
You will have to be more careful about making the conversions while in that scenario you would simply
make the combined the data, do the conversion and simply split the data.
So there is no chance of any discrepancy.
So it is completely up to you which method you are using.
So here I have removed I am from the testing data.
Now from the object column, so I'm creating an object list of object columns by getting the training
data from the columns, I'm taking the data type object so we can also we could have also done select
the type.
So this is another matter.
So there are several methods of doing the same thing.
It is completely up to you will which particular method you are more comfortable with so you can use
whatever you like.
And the main target is how we retrieve the the data and how we process the data.
If the processing is going fine and we are getting the desired.
Then it does not matter if you are using select the type or you are filtering these out.
So these are simple or different methods which we are implementing.
So you can choose whatever you like from these.
So then here I am selecting the Nonobjective, so here I am selecting the unknown object, which will
be either the in type or a float type, so all will be selected in this particular category.
So I am getting the twin columns, which are not objects into the numerical column list from the testing
dataset, also on the training data, but now in the object column.
Whenever we have null values, so wherever the values are not a number, I am basically filtering out
the green object or dataset with the object column.
So it is checking all the objects column and wherever we have not a number, it will fill it with none.
Same thing in the best data set.
Whenever we have an object column and we have not a number, we will fill none now in the null value,
we will replace the null values with zero.
So whenever we have null values, we will replace it with zero in the numeric data type.
And they have renamed the date frames to planes zero one zero zero zero zero one.
Now, we have failed the values.
So now we've we've what the ordinal features.
So here are certain ordinal features.
So we have ordinal features as well.
Second, basement quality, basement condition, basement exposure, basement final day.
We've been finishing the basement, finishing day to see garage conditions, so these are different
details of the house and these are have ordinary values.
So for this again, I'm splitting the data and the.
And do X and Y, so from brain zero zero one, I am dropping the prediction column that this is a science
price, I'm creating my X column and similarly from Brain 002, I'm creating by removing, by keeping
only the Sears price column that is F.P. predictable.
Now I'm converting the columns, so I'm generating an object of the one Hawtin Khuda.
So in this we are giving the column name as ordinal feature.
So all the ordinary features will be put into this and these ordinary features will be converted into
the ordinary columns using this one quarter.
So we will simply be the dot vanguard C one not this is the object which we have created and then we
are doing to transform.
So that is we are footing the columns also and then transforming at the same point of time.
So we have transformed 13 zero zero zero two zero zero two.
Similarly, we will be transforming the best zero zero one.
That is the testing data.
And we have created three zero zero three out of it on this 003 11.
Now we are getting the nominal feature, so we will convert the nominal features, so nominal features,
four x four.
We are basically checking for X in object columns.
So we are checking whatever object columns are present and removing all the ordinal features from these
object columns.
So these are the nominal features, which is also known as the categorical features.
So now here have the we have the categorical features.
So one way of converting the categorical features we have seen was using and dot where we have used
as type.
Also, we have also used this condition for converting the dummy variable and we can also use get the
means function.
And here is another method of converting categorical features into.
The one thing that is using the label in the.
So here you can see we have the training and testing data from the original form, so here you can see
different values which are present in the testing and training data, and you can see the numerical
scenes place state.
And here, these place is missing.
Now we have them, the transformation's, so after transformation's, there will be several other changes
in the values.
So I will show you those one by one.
So here we can see the ordinals features, so let us have a look at Extoll call an extension.
Or.
Here you can see the you see value has been changed, emetic and similarly other values have been changed
to new type.
For the let us have a look at the phenomenon of baby boomers that this categorical variables, so what
we are doing here is we are importing the label encoder and using the label and we are looping on the
different nominal features president in this particular dataset.
And we are applying the label in order to transform or transform one of these.
And after that, we are getting the dummy variables for these nominal features by using we don't get
dummies and we are dropping the first one from these.
So let us have a look at the data center, which has been generated so you can see we have all the columns.
Which have been converted into numerical columns.
And here you can see all the columns have been converted into one hot including.
So this is how all the values have been changed now for those, we will apply a fee to engineering.
So here we are generating new columns because we will have to create new columns to find out new relationships.
So here we are creating a column as a total board, which will have a combination of full board hoverboard
and basement full board and basement hub.
But then we are going to work on creating a complete or total Baulch column, which has a sum of all
the open porch, different type of porch and deck.
Then we are checking if there has been a recent remodel by subtracting the years old from the remodel
at.
And similarly, we are checking if the house is new or not or if the house has a pool or not, then
if the house has a fireplace, has second floor, his garage has basements.
So these are different conditions which we have created using the lambda functions and comforters and
operators.
Next, we will be fitting the sort of the model, first of all, let us divide the in to prevent a split.
So we are dividing the data into X and Y volumes.
We are getting the X values in two three zero zero five by dropping the C splice column and then by
keeping only the Sainsbury's column.
After that, we are splitting the data with the testing size as zero point to five.
So we are keeping twenty five percent of data in two of the.
Thing said and.
Seventy five percent did in decorating.
Next, we are importing the packages, so here we are importing the extubate agressor and Igby orif
regressive and the grid search c.v and we're importing the mean, absolute better and mean square error
for the matrix.
So these are the default parameter values.
So the default parameters are maxed out as three learning rate, as one number of estimate, those as
hundred Mobilicity as one would be basically defines if we want certain messages to be printed or not,
which will show you the progress of the learning.
So in case you want to have a view of the progress of the learning by training the model, you can increase
the value of Bilbo's D.
I usually define it as to be the silent level.
None of this actually means that if you want to have the message is silent, then objective is the.
I get that you want to have that is we want to apply a regression and the regression method is squared
error, we want to find out this squared away from this particular model.
Now, similarly, if you are applying classification, then for classification, there will be a different
type of barometer.
So for you that you will have another objective function, which you can take from the classification
documentation which we have in the in the library.
So booster is the geometry, which means we want to have a gradient boosting tree and job is the number
of traits which you want to have.
So it was always suggested to have in jobs as more than one.
If you are working on a multiple computer or if you are working on, let's say, a GBU or Onwubiko lab.
So on those platforms you can have more number of jobs running.
But usually when we are working on our Lupo's systems and using Jupiter more for training, then increasing
in jobs might not be beneficial enough.
So it is completely up to you.
You can try increasing in job if actually it works out on your system.
Otherwise, in other foster systems with processors, it would work faster.
The more value, the better.
It will run faster.
Then then Intrade is the number of trades you are trying to run together.
Guerma is the gamma value.
Minimum wage is one maximum.
Leptis Zeitels subsample one column sample by three one.
So these are different values which we are keeping as the.
Default parameters, so there are certain important parameters that is booster, the type of booster
which we want to use, that is the small model which we want to use for this particular regression.
So there are two options.
That is geometry.
That is, if you want to have a three based model or a linear model of Kloner, then the end is the
default number of the maximum number of trades available.
So if we don't define Intrade, then it will by default to the maximum number available and then choose
that objective defines the loss function to be more minimized.
Then we have subsample going on by three, which is to provide the randomness and estimate the is the
number of threes which we will be generating or the number of sequential Vitullo knows we will be generating
for this particular.
Then we have low grade, maximum depth of the three, then the minimum wage, that is a minimum some
of the above all observations required in the.
So these are different type of barometers.
So for during these high bhavana meters, we will use a grid, such CV.
So Ingrid TV again, we have the meter.
That is the model which we will be using Vadum grid, which will be these parameters listed out, which
we will be selecting from scoring, is the scoring method which we will be using in job is the number
of jobs which we will be running in parallel then CVS's the cross-validation which we will be using.
We have been using this for a long time now.
So these are different barometers, which we have so far from using these bottom feeders, you can find
out the number of models which will be generated.
So if you want, you can use Garito TV also.
And if you want, you can use random fortressed.
Also, it is completely up to you.
So we will simply do grid search, dog food and then find out the best barometers from the best barometers,
we will find the best model again based on the best barometers which we have been.
And finally, we will check the value.
So the mean absolute error comes out of fifteen thousand three hundred and thirty a thirty seven, which
is the price difference, which is present out of the oil prices.
So we are plotting the result of between different values.
So here is the plot.
So the blue ones are the real values and red ones are the predicted values.
So you can see that it has predicted the values pretty nicely, but only has defaulted in those two
points which were coming here.
So one of that has already been actually predicted nicely, but that is one value which has not been
predicted correctly.
Then we have predicted result down submission.
So if you want to submit to the predictions or send out these submissions to any other person in the
team, or if you want to export the predictions so you can simply use or predict to predict the values
and then simply send it to the door to KVI method to convert it into a certified.
Now, there is one more thing which I want to capture here, that is by training your model while running
this great TV, when we are training with these small number of barometers, it is completely fine.
But when we are training with a huge number of barometer's, then it might become difficult for us.
So what we can do with that is no particular level of hyper barometer that this hypovolemic that is
more important or this hyper badme that is more important.
And there is nothing like which hyperbola meter has to be reinforced or lead.
So keeping this in mind, what you can do is let us see.
I will not train these.
Hyper barometer's, so I just command these.
And.
I can simply provide the objective function and I can provide the investigators, let us keep the investigators
focused and keep the learning.
So what I can do is I can provide the investigators and the objective function for us and give several
other values for the investigators so I can give seven hundred here and thousand here and fifteen hundred
here.
And then I can run this particular dorning.
Once I've done this, I will get to know the values for an estimated and objective function, using
my best model, using the best barometer, I will be able to obtain the best values for an estimated
and objective function.
So then what I can do is I can simply comment this.
Or instead of commenting this, I can simply go ahead and remove all of these values and keep only that
one value, which has given me the best result.
So let's say I get rid of this entire thing.
And I keep only led to save my best estimate that came out to be the one with five hundred meters so
I could put five hundred.
Now, once I have fixed to these values, then I can again bring my A model by changing different learning
rate so I can test a different learning rate and then find out the best value of the learning and then
updated here.
So let's say it comes out to be say zero point five.
So I fix the learning.
Now I will once I have fixed the learning rate, then again I will train this great sword and I will
train in four different max depth values.
Now, using this, I will be able to find out.
One makes the max depth, which is working best for me.
Then again, I can figure this out and put only one value here.
And once I fix the value of max depth, I'm learning Grayden and an estimate, an objective function.
Now I can go to the next photometer and then print.
Now why do I do this?
Why have I told you to do this?
Is because it actually helps us in reducing the training thing.
Because let us consider we are training only these many we only these many barometers.
So what will happen is.
It will actually cause a problem for me because I will have too many parameters to do and every time
I the barometer here it.
Actually increases the number of models exponentially.
So let us consider the number of Morbius, let me do a little calculation for you.
So let's see, I had zero point one year and zero point zero one zero point.
And is 0.01 here?
So these were the different values, so what are the combinations I have I have these values.
So for these, I have.
To.
Cross for.
Across three.
Cross to.
Cross St..
Clause six.
Values.
So this comes out to be five seventy six.
And if you will notice, I am creating a hundred models here.
I am creating 200 models here.
So fifteen hundred models being created for each one of this in 215 hundred, if it is generating only
fifteen hundred models are not generating all of these.
Only this much then if I'll just get rid of this so you can see for only this much I am creating.
One forty four thousand Modern's, so this is such a large number of models, which I am reading, but
if you will be using the method I just told you, what you will be doing is.
You are just training for.
In the very first go, it will take some time because now you are training for a more number of values,
so you forced you will be competing between one hundred five hundred and thousand, literally.
You will just compare between one hundred five hundred ten thousand.
Then you imbued values for to wonder if the value comes out to be, let us say five hundred, then you
can change one hundred and thousand two hundred and seven hundred and see what values comes up.
If it comes up as two hundred then you can find out another value between two hundred and five and that
is how you can decide.
Now once I have found out the estimate, then what will happen is I will be tuning for lesson number
of values.
So I will be training only, let's say 10 or 15 models act one thing, which will be a very huge reduction
in the model training thing.
So instead of planning a lot of barometer's together, you can use these sequential approach which will
allow you to be will only fuel barometer's at one point of time and you will be winning them.
Now, one more benefit which you will be getting by using this mechanism is that you can actually.
Get the result out of it, sometimes what happens is that are training with so many barometer's, so
what happens is that somehow the the running, it just gets stuck or it hands the computer or something
like that happened because not everyone is using BU's.
Right.
So in that case, it will take a lot of training time and sometimes it may happen that you will get
an error and all the things which you have done, it will be completely lost, but then you will be
training with only one photometer.
I think it will be running faster.
So after like a seven or 15, 20 minutes or 30 minutes, you will have a desire for at least INDISTINCTIVE.
Then after that you will run for the learning date.
So after fixing the number of an estimated Pacifics fix the estimate or two hundred now I will I will
train for learning.
Great.
And I will decide, OK, these are the learning rates and after training I get a particular value of
learning grade.
So I will fix that.
Learning great value.
And now after 15, 20 or 30 minutes, I will be getting the results for my next learning barometer.
So this is how I will eventually keep on getting the values of my hyperbola meters instead of losing
out on a lot of time and again losing out of my world because of some particular reason, maybe the
laptop was unplugged, maybe something was not going well, maybe the laptop restarted, or somehow
all of this got interrupted or something just happened, or maybe the controller stopped this one doing
so because of all these reasons, the world can actually get lost.
So we don't want that to happen.
So we can use the sequential approach of what you can simply do is.
First of all, government everything keep only fuel barometer.
So these are the two Hyppönen we designed Duni.
I have fuel for these high barometer's using my guess the best badam.
I will get the best value for these.
Let's say the best value comes out to be 500.
So I will change this to five hundred.
Now I know an estimated value has to be five hundred.
Now I will bring this and add certain values to learning by and we do see a zero point zero two or zero
point zero five and zero point zero seven.
So I have these values which I want to buy right now out of all of these values.
One, I all of these values I really get to get.
This is one value which is giving me the best results.
So I will keep only that one particular value and now I will bring for my next hypovolemic.
So now I will check which maximum depth is giving me the best result.
So again, then I will select the best Maxted and then go ahead with tuning the next model so you can
iteratively keep doing this until and unless you get the best hypovolemic.
And this will also show you if this learning, this barometer tuning is actually improving the model
or not, because if that changes, which you are getting in the model improvement are not enough, then
probably you don't really need to do the immediately anymore.
So it will actually tell you that exactly if if the model is improving by this tuning or not.
So you can actually judge those things without running that extreme board for 10 or 20 hours and then
getting the conclusion that it did not really.
So you can use this particular method and it is a very useful method.
I have been using it for a very long time and it really helps a lot in reducing the training thing.
So please try this particular method as it does a very useful and very efficient method of training.
So that's it for extra boost.
So next, we will be learning of the next model that is, I think, Canada's name, but we will be taking
up the nearest neighbor next.
Thank you.