Hello, everyone.
So in this particular session, we will discuss about the implementation of the junk property classification
problem.
So this particular solution is slightly tricky because here we need to decide what we want to achieve
here.
We have to balance out the situation of identifying the junk property properly.
So there would be certain cases with the junk property would be written as a junk property and there
would be certain cases where the good property might be classified as junk property.
My old model.
So you need to make sure based on your understanding and your business requirement, if you want to
classify all the junk properties properly or you still need to keep all the non junk properties.
So let's see if it is important for me to make sure that no junk property is classified as junk, then
I then there wouldn't be a case where some properties which would not be junk would still be classified
as junk properties, because here I would keep my position such that some non junk properties would
be classified as junk, but it would make sure that no junk property is skipped or left.
So we need to make sure what we want to get out of this.
Another scenario could be, let's say it's fine if I have a few junk properties listed, but I don't
want to give up on any good property.
So in that case, I will keep my threshold such that the good properties are all intact and they are
tagged correctly.
I have some bad properties, junk property as well as good properties.
So you need to balance this out.
Now let's see for them.
So in this particular dataset, I have simply imported all the libraries and this is the data which
I have.
I have the flag of junk zero and one.
Then there are different columns which are in your state price index listed material price index agency.
Then the rating, then we have expedited listing price index for price index fund.
These are different price indexes, which would be pertaining to some different criteria because they
are not domain experts.
So we don't know much about it.
So we will have to analyze these columns and then find out what is important, what is what next, what
we have our John knows the zip code, the insurance premium, then the blood type architecture.
And there are a lot many quotes associated.
So in total, we have these many levels of detail that the sixty two thousand and thirty one columns
out there.
So these are all the columns which are present and this is the description of those.
So from these you can easily identify outliers if there are any.
So there are 25 percentile of in price index nine.
The twenty five percent is fifty four fifty four hundred.
Then we have fifty percent less.
Sixty seven hundred seventy five percent less.
Seventy nine hundred.
But the maximum value is very high in comparison to the seventy five percent.
Then this surely shows that there are a few outliers in the price index.
Maybe then you can see there are so many zip codes available after that there are different values.
So again it is of normalized population.
Again, there is a huge gap that is there are certain outliers present here.
So let's look for those.
So in different values, we have these in this signing estate and all of these
here.
We have price index, full price index one, price index six, which should be numeric, but I actually
object.
So we need to make sure that we convert all these objects into numeric shapes so we will decide.
So then cutoff.
So these are different kinds of values.
So we have decided that the value to be five percent.
So anything which has less number of values in a category less then could be one of three thousand one
hundred we will come to.
We will ignore that particular subcategory next, there are certainly medical columns, as we have already
seen price index for fun six, price index three, these are all numeric types, but they are an object.
So we will be changing them into a numeric book.
So we are converting these columns into numeric columns by using numeric edwardson, which would basically
take care of the warnings for the we have made the transformations and then we're converting these categorical
variables into the using the cutoff of which we have decided and same has been done.
So now all our data looks like a numeric data.
So once we have this numeric data, we are going to build up more.
Now, on top of this, what else you can do is you can find correlation values, you can apply logistic
regression and you can find out which are important.
So all these things can be done.
So that is completely up to you.
How you take this up.
If you want to apply Pinder's profiling, you can apply that.
Also, this is just one of the implementations, so I won't be showing everything.
But I have already given suggestions how you can create models that you can use those suggestions.
So these are just a method defined to create the report of the mean validation score and everything.
Then we are running any grid search random such that this is one droplet which I have created here.
This particular option will predict it will bring a grid search and after finding the grid search,
it will give me desired Informatica cumulative metrics so that when I make next week I will be using.
So here you can see there are different metrics which we have given.
We will be giving different metrics, values and using those different metrics values.
You can use any particular metric as a result.
So here what we have done, we have given the start, if I gave it and using this, if I gave I'm giving
in to it next, I'm applying a grid subsidy and I will be.
This method will take any model.
It will apply the parameters.
It will take all these values which we usually give.
And for scoring, it will use this quarter and it will use the money for school, which has been given
here.
So basically it is it will scoring the models, the modules based on these quarters.
And then I want to refer this particular module to get the final module, the best model out of it.
It will use it using the best score.
The score, which I have as the equal scored, has been set here.
So why have I done this?
Because I am assuming that for me, any injunct property that comes in my listing, then I will have
to face certain losses.
So that is why I want to make sure that I identify these properties correctly.
So I how I want to increase my record.
So that is why I'm giving my first score as required here.
So then this will apply the search and the grid search and give the predictions, make the predictions
using, biproduct, and later it will give the confusion, metrics and all of the details back to you.
So what we are doing here is we are deciding our different values.
So here what I am doing is I have created certain scoters.
So here is one Schauder, which is taking place in the school reclose for accuracy score Evita's.
So I'm checking all of these scores on.
The first thing which I am trying to do here is because in every time I'm not aware of what we value,
should I be doing so I'm just trying different with the values.
You can skip this entire thing, go with the original flow.
Also, there is not much difference, but this is just different trials which have been done.
So here what I am doing is I'm taking the Beta Schauder evidence and based on the evidence code, I'm
trying to find out the full value.
So I have just run all of these models using this grid search of output again and again because it's
a trap.
And what it allows me to find out is it allows me to find out the best evidence because it is fitting
the model again and again based on the best performing every.
It just simply fix it and then it gives me that the best barometer for better school is class with this
and we did this and then I have defined the confusion.
So it gives me the confusion that results from this.
You can identify the false negatives and two positives and false positives.
All of those you can identify easily and then decide.
So here these are the predictions.
Well, these are the wrong predictions.
OK, so this you can see this is my data, so this seems fine next.
Again, there are different combinations, different values coming out then four that I have tried from
two hundred to two thousand.
And further, I have just tried different values.
Next, I have simply implemented extra reclassified examples classify.
I'm using that to classify it.
I'm just finding out different values again.
Now this is completely up to you.
I came to a conclusion that I will use two hundred as my value in here.
I'm going with every other school.
So this is how I'm finding out which every school I will be using.
Now you can completely stick with Recall's because recall is the actual metric which will be giving
us the best result.
So we will be using recall at the end.
So again, I'm just completing the scooter again, using the scooter, and after that I'm giving the
class with investigators, then flipping the model after fitting the model.
Here I am trying for precision scores.
So this one gives me the best precision school here I'm trying for, because for all I have to do is
I will just give the equal school here to different.
So it will give me best results based on school year.
I've given precision scored here in the inside.
This school is giving me designs based on precision school.
So how you want to use it is completely up to you.
You just need to put this precision school inside this quarter.
So the next week what we will be similarly, every school has been done here.
So next, how you will be doing it is completely safe.
So here what I am doing is let me show you this is orif the importance.
So this will basically give me the importance of my features.
Let's see what it is doing.
So for this, I have used another method.
So what is this method?
This may explain this to you.
So the basis of this particular method is that I am having some columns, so I have, let's say, around
60 columns already present.
So in this 60 columns I want to find which feature important.
Now, when I'm finding out we feature importance.
I'm not sure which feature which how many top features are actually important and how many important
features are actually unimportant.
OK, there is one line to find that this is good and this is bad.
Obviously something that is a completely, completely random is a sure thing that it is a bad idea.
But so if I have something like an idea, then surely that would be a random variable and that is why
it is unimportant.
So I'm doing the same thing here, so I'm introducing a random variable at random.
Now I will bring my model by including this variable also, and then I will finally find out the feature
importance of my video.
But now now I have some features.
Now there are 60 features of that, including this random variable.
So there will be some variables, say twenty variables, which will be above this random variable in
the future importance.
Josh, just so these are actually important features because they do not have random behavior and whatever
comes below this random variable in the feature importance of Java, which I will be generating, those
are completely random variables because they are having importance less than other items.
So these are short, unimportant and not important for me.
So I will completely get rid of those.
So that is what I will be using here.
So what we do here is I'm.
Fitting the model, finding out the importance.
So when I find out the importance, these are different variables, see price indexes, normalized population,
all of these are there.
And further, I have this column that I know, which was a completely random variable in this particular
column by completely random values.
So then you see this insurance premium property.
It states you these are having importance less than this random.
So this means that these variables are shortly having no importance in my modern building process so
I can get rid of all the variables from this random building and.
So here this is one solution which we have found of and same thing I have just gotten rid of all those
columns, I simply kept only the columns for which the importance value is greater than the important
value of my life.
So I kept need those.
I'm then I have simply drained my model one by one in sequential manner, using the sequential training
by finding out the lower ninth grade class weight.
It gives me the best learning, great best number of any estimate, the best class, which would be
that then for the it gives me the maximum depth, which would be the answer.
So I keep on doing that and for good I get the best out of these.
So that is exactly what I have done.
So this was the implementation.
This is my implementation.
You can try your own gradient before you can apply any other Muggleton.
You can try and my this also for this.
So that might also performed really well.
You can go ahead and try neural networks as well.
So you can try different combinations.
You can try stacking.
You can try a random.
So that is completely up to you how you want to make your model.
This is just one of the samples.
I hope you understood the problem and this particular solution.
Thank you.