In this section, we will be learning mode by generation, so first of all, let's have a look at a
normal piece of covid we would have created a model.
So here we have a textual data.
And this text data is the same as from collection data, which we have already used.
Now, we already know that this data contains target value and the message.
So how we would have used how we would have implemented it earlier is we would have imported the strain
split, we would have imported the multinomial navys divided victimiser Stopford.
What tokenized would need?
Climatized.
And then we would have created a word Natal Amitiza object called Lemmer.
Next, we will get the sake of stop words from the language English, and this becomes de stopwork.
Now we will define a function which is splitting dilemmas which will basically take the message, conveyed
the message and do a lowercase.
Then we will organize the word would the mess of the message.
And after that, we will create a blank list of words.
Stop.
Now, what we will be doing is we will agree on all the words which are present in the message.
And in case the world belongs to the Stopford, then we will not do anything, but if the world does
not belong in the Stopford, then we will upend the world in the blank list, which we already created
here.
Now, after this particular run, what it will do is it will the raid on the LEMMER, which is the word
legitimatize it, it will limites each and every word that is not belonging in the.
Would stop.
So it will memorize all the words which we will be getting after removing all the stock quotes from
here.
So basically what it has done is it can water the entire world entire message into a lowercase and then
collected all the words.
In this particular list, which did not belong to the Stopford, so if the words to belong to the stop,
then it did not do anything.
And if the word did not belong to the Stopford, then it will added the word into words and stop and
we will get the words and stop, which has a list of all the words which do not have stopped.
Now, out of this list it with celebrities, all of these words.
So now we have retrieved the words, removing the stop words, and now we will limites these words and
this will be returned by these fluking dilemma's.
And then we have created a festering split out of the data which we had.
And applied the idea of victimiser.
Now this victimiser will analyze all the data, all the rules of data using split in two levels, and
it will make sure that the minimum document frequency of any word is twenty and the maximum document
frequency of any word is three thousand.
That is a word to be a part of the video vector.
It has to be at least 20 times a be a part of a document or a part of at least 20 documents.
And it must be a part of three thousand document if it is belonging to less than 20 documents.
That means that it is a very rare word and in case it is belonging to more than three thousand documents,
it means that it is a very often acting.
Now we will fit the IDF on the training data and also transform the training data on this data.
And after that, we will apply the classifier or any classifier which we want, and after applying declassified,
we will declassify and find the probability of the same and then we can use it to apply and find out
the classes of the particular data.
Now, this is what we used to do, but now because we are aware of pipelines, so we will be creating
pipelines.
So how would we implement this, using a pipeline to create a pipeline?
First of all, we will import pipeline from Escalon Pipeline.
This pipeline allows us to.
Provide a list of the books.
So basically, this list contains all different types of actions, so this list contains a couple of
actions which we want the pipeline to perform.
What are these actions?
So the first action is the idea.
This is the name of the action.
And after that, we provide the declaration of the action.
So here we declare what action this has to be.
So what is the action?
The action is the idea victimiser.
What it has to do, it has to analyze using splitting the lemons.
This is the minimum document frequency.
This is the maximum document frequency.
Then next action is classify it.
What is it?
It is the multinomial name.
We classify it.
So now we have provided that it has to analyze the data using Vidia Victimiser and then use this particular
classifier.
So this is our pipeline.
This is a complete pipeline which we have created.
Now, how will we use this pipeline?
We will use this pipeline by simply using five dot fit and we give the training data.
When we do fight the good fight, it will basically run this particular thing if I leave the laser on
this data and then learn from it and apply it to declassify it on top of it so it will run the video
victimiser on the data and then declassify it on it.
And then it will fit the this data using this classifier.
And then I would be ready.
Now I Arpatepe actually knows what the model is, what the model parameters are.
So next time when we will have to predict the values, what we will do is we will simply use five predict
proba and we can simply get the probabilities of the predictions that we want to make.
So just we have to declare the fight.
We have to inform the public about the fight.
Then we will have to foot the pipe once and then we can use it a hundred million times using five one
proper.
Now, this is about just one type of feature, it is just having a tax data.
Now, let us say they have a particular problem where we have numerical data and categorical data now
that data will have to be dealt with.
Together.
So in that case, we can vertically apply the idea of victimizer that needs a lot more detailed transformation.
So we need to provide that also.
So what we will do that.
So let us say we will be creating a digital union.
So this one will be we will be using existing base data.
So this data contains reference, no children, age, violence, status, occupation, occupation.
And I know our core values.
So it has 32 columns.
Now, out of this we want to predict that is we want to classify and we want to find out did a new grid.
But now here, what we will have to do is as a simple problem, what we used to do is we would have
four civil.
Converted all the numeric column into numerical.
So this is age one, this is family income.
These all would have to be converted into numeric form.
Next, we have this categorical cholo's.
These categorical columns will again have to be converted into categories.
So these are a few things which we would have done.
But now we have to implement this using right now.
How would we do that?
So for that.
We will get the details.
So we did our unique.
Then we got the data types, now we will get to more libraries that is based, estimated and transformative
mixing.
So what we do is we create one class, this class is for selecting the particular variable.
It allows us to select the particular variable type.
So what we are putting into it, it is taking the input barometer as the best estimate and the transformer
mixing as the input value.
Now it requires a little initialization, so as initialization, it needs the variable types, so which
variable dipen you're looking for.
So in that selected, we you have numeric value, ordinal value, categorical value.
So any time could be present.
So in these variables, we want to have the numeric type or the object.
These are the only two types which we will be having.
So we are giving either numeric or object.
And out of these, we will have some ignored with what is a good variable ignore variable is the variable
that you want to get rid of.
So any variable which you would have wanted to drop initially, you can provide it as I ignore that.
For example, reference number, I don't want to have a reference number in my dataset, so I can simply
put it as I ignore that.
Next, I will provide different details like what I want to do on instead what I want to return and
transform.
So in case of transform, I want to return the X dog to be self.
Not that they don't drop cell signal.
So what we want to do, we want to select the debate which detailed with the one district.
We want to select the data five which we provided here.
So whatever database we will be provided while running the pipe, we want to select those particular
columns.
And when we select all these columns before that one to drop these ignored bad columns from Axis one,
this is what we want to do.
So we have declared the same.
Next is we want to get dummies, what does get them these pipelines and so this is get them is a class
which will help us to create dummy variables.
So what we need in this, we want again, what do we want?
We want the frequency.
That is the frequency of which we use to provide while converting a particular column into the column.
Then we want the variables.
Categorical dictionary.
That is which dictionary of the variable got its.
Now, next, what do we want to do?
We want to fix what is the food?
We want to get the columns and from the columns, we want to get the account of the different columns,
values of the categorical variables.
And based on the value count, we want to find out if the category has a value greater than the frequency
of which we have provided here or not.
If it is greater, then then we will put it in the variable dictionary.
Otherwise we won't.
So what is this?
This is the variable category dictionary.
So this will have the values of the columns which we actually want to convert into dummy variables.
Right.
This is the fitting.
But now what is transforming power in transforming five?
We have this variable guide dictionary keys.
We have the key values.
Now, based on this, we get the column names from this and from each column we get the categories and
from this categories we basically combine the column and category and convert it into a dummy variable
dummy column.
So this is what we are doing in this game.
Dummy Spyplane.
Now, what we have to do, this function will help us to select the new video of this particular will
help in converting a particular column into dummy columns.
Now, what we want to do, we want to create a pipeline and create a union because we didn't know.
What we have is we have all the numeric columns generated here and here.
We have all the object type.
Now, we want to combine both of these because these will be presented to different data frames as of
now.
So now we want to combine this so we will combine this using feature unión.
So once imported this pipeline and feature union, we will now import the logistic regression and strain
split.
After that, we will simply get the data set accordingly that this extreme Vedrine Vytenis values.
And now we will create the biplanes.
So the first five, which we will be creating is the categorical pipeline.
So what is this categorically pipeline?
This pipeline contains?
But what are we doing here?
We are getting the word type selectors.
So what is it they're doing?
It is selecting the object and ignoring post code and posted from.
Next, what we are doing, we are getting the dummies and for dummies we are fixing the biplane value
of the frequency of value was one hundred.
So based on frequency of value, we're creating the dummy variables.
Now you can go back.
To get the means by plane.
So here in the get the spy plane, you can see you have this initialization with self and frequency
of value and it has different values like that can fictionalise and all these values.
So based on this, it will basically carry around this particular guidebook by plane, this guy by plane.
Well, first of all, run this categorical variable, get the categorical variables, then push all
of these categorical variables which it will have in this dummy spy plane into this particular function.
And this function will generate the dummy variables.
Then we will create the next pipeline.
This next pipeline takes the features and creates it is a part of two tuples you can see.
This is the first step and this is the second tuple.
What does the first step do this for?
Stapel is of feature unión, so it applies feature unión function.
On what is it applying the Feagin union function, it is for running the pipelines, so entreated union,
first of all, it will run the pipeline.
While running the gas pipeline, it will again be categorical variables using five types selected,
it will get the object variable and remove the post for the full stadium.
Then it will get the NUM variable.
It will run this particular double.
This double has that pipe selected for in sixty four and float sixty four and it will remove the reference
side as ignored variable.
So now what we have generated, we have two values here.
The false data frame consists of the output from this one and the second data frame contains output
from this one.
This particular contains the output from this particular cat fight.
What does cat bipeds get bypassed?
The object variables with converted into dummies with hundreds of cardboard and ignoring these two columns.
Similarly numbered contains in sixty four unflawed sixty four type columns and ignoring the reference
number column.
Now, once these two have run, then after that featured union would be done, so for the inner part
will be done after that, the outside fight will be done.
So now future union will be done and the result of the future union will be passed into logistic regression
and not just a recession will be present.
Now, we even use this pipe to which is a cumulative pipeline.
This pipeline actually contains all the things which we are doing.
So we will fight this pipeline on the training and testing, the training data, and then we will predict
probabilities using this particular pipeline only.
So this is how we can use these pipelines.
Now, let's say the had instead of having just categorical and numerical variables, we had some textual
detail.
So then what we could have done is we could have created another pipeline, just like the pipeline,
which we have created for the factual data.
That is five one, then we could have included this five one.
As a new pipe here, so we could have created get by, we would have needed another pipe, one like
the one which we created in the dictionary data, and by doing the feature union internally, we could
have added another couple.
I'm supposed to have the five one.
Karma and the details of the fight.
So then what will happen is it will run at five.
It will run this particular by five selecter.
Then it will run by one.
I do the feature union and after that run the logistic regression.
So this is one example which will cover all types of variables, all type of data.
Which would you ever get?
So this is what you can do for creating a pipeline.
Now let us see that you want to run this pipeline.
You want to see this particular pipeline.
Now, they know what we have learned is that we will run this file again and again, again and again,
and then use this file.
But that is not the sort of walking option.
What you will be doing is, let's say I make certain transformation in my data center after doing the
transformation at any point of time.
So you want to save your work then?
What you can do is you can save the data frame in to see if we fight.
I next time when you want to start your work again, you don't need to run the entire piece of food
again.
But you can simply import this ESV five, which you have exported over.
Secondly, when you generate a new model or any pipeline, you can simply use this particular piece
of food, which is job live to dump the fire, that is to save the pipeline or the model and to help
Girlfight.
A big downside is a round table fire.
So what you will be doing is you will simply save it and do this fight.
And whenever you will want to load a particular model, you can simply load the model by state.
My model then opened them modified and then you doing we don't load for the model and once you load
this, you can directly load this model into a pipe or in any variable and this variable will hold your
model isn't.
Now, once you have your own model, you can simply use don't predict Roba or don't predict to run the
model itself.
So this is how you will learn and you can run pipelines and this is the entire of pipeline, so you
can create different pipelines using this particular method and use them in future.