In this session, we will begin with linear regression, linear regression is the very first and very
basic model which we will be using for making predictions.
Why this is used, how it is used and how we will actually create the model is what we will be launching
in this particular session.
Please note that we will be going through all this Hilevich we have done until now regarding the preparation
again so that you will get a better view of how we will work on a real life problem.
So even if you did not understand how the data is done and how it is actually implemented, we will
be doing that now.
So let us begin.
So till now, what I expect you to have learned is that you would know the Biton basics, which we have
discussed, you would know that how you can read the data.
We have discussed the basics, which would help you to create some functions and methods which we will
be using while preparing the model, then reading data will actually help us to import the dataset from
various CSP points.
After that, we will need to look at different properties of data for looking at different properties
of data set.
The summary functions like Full and describe will be really helpful.
We will have to understand about different types of features.
For example, the features could be numeric type, category type or audiotape, and another category
being the text data.
Now, out of all of these data types, we will have to convert these data types into a numeric form.
Hence we will need to know how to convert categorical or ordinal data into the numeric Day-To-Day via
one encoding that is Dommy creation or via converting the ordinal data into a numeric value.
Or we will have to convert a particular data type that is legacy date to the month they and your format.
Another thing which we will have to know is how we can convert the text data into account like that
or the idea vector.
So these are a few things, but we have actually discussed by learning about Python and and so we will
know how we can convert these.
But why is it needed and how we will be doing it on the building data is something that we will be learning
through the projects, which we will be doing by working on each and every method and each and every
every time, which we will be learning.
Next is we will have to create certain subsets of data.
Then we will have to work on the data things now, although the data which we will be working with would
be kind of modified and we will not we have to do a lot of modifications, but it will give you an insight
of how the actual data will look like.
Then another thing is how we will analyze the data.
We can analyze the data numerically and visually, which is, again, we we have learned uncovered in
the data publishing module.
So we will need all of these learnings to actually implement the machine learning algorithm.
So in case these are not clear to you, you can go ahead and look at the videos again and try to get
familiar with these topics so that you will be able to understand what we are doing more to the.
So let us talk about linear regression.
In our introduction to Machine Learning, we have discussed the three types of machine learning problems.
The first one being supervised learning, the second one being unsupervised learning, and the third
one the the reinforcement learning.
The cutting modules, which we will be covering that initially in Galtos, which we will be learning
will be a part of the supervised monitoring.
So let us discuss about the supervised model first.
The supervised model is the model which has the input data and output data both provided.
So what we will be doing in a supervised program.
So the problem will be something like that.
So here we will have the idea age among salary, dependents, sex and children.
These are different X values.
These are different input values which we have been given.
These input values are also known as features, attributes or input values or independent values, because
we expect these values to be independent of each other and not have any relationship amongst each of
the.
We will have these the age amount, these columns, which would be the input, values or input features
or attributes or independent columns.
Now, apart from these columns, we will also given input.
That is interest rate.
Now, this is actually the output value or the target value, which we are expecting.
So as an input in supervised learning model, we provide the input values and the output value both
to the model so that the model can loan the relationship between the independent columns and the dependent
column, and it can formulate the limit equation or any kind of equation or connection between different
columns so that it can create the relationship between this.
So this relationship which will be creating will be to find different patterns from this particular
data.
So based on this, we will be applying the supervised learning algorithm.
Now, when we talk about linear regression, linear regression is again a supervised learning model,
and it is the simplest supervised learning model and it is trying to fit the data to a straight line
and create an equation so we can see how this actually works out in something.
So let us, first of all, see what is unsupervised learning now.
So in case of supervised learning, we used to have the target value also available by giving the training
data.
But in case of unsupervised learning, no target value is provided.
In case of supervised learning, we tried to find out different patterns from this data.
I'm the equation or create to establish a relationship between the data so that we can predict a particular
value.
So here I mean, that's supposed to predict the interest rate, which is a continuous one.
We wanted to predict a particular value here.
But in case of unsupervised learning, the focus is on grouping dissimilar entities by finding structures
in the data.
We are not concerned with finding out something and predicting something for the future.
These is what go the data is with similar entities together.
And in case of unsupervised learning, we have to find anomalies that we can use it to find something
which is out of the bag, something which is not usual, something unusual, and another implementation
of unsupervised learning is dimensionality reduction.
But we will discuss in depth separately.
So now let us look at different types of business problems.
So there could be a business problem, such as the user might be looking for planning, there could
be an organization who is looking for improving their sales or reducing the cost or increasing the quality.
So they might be looking at some particular aspect of the business problem.
Now, based on these, they might be looking for predicting a particular value.
Now, when we are predicting a particular value, we might and I do predict a continuous rally also
and the categorically value also.
A convenience value would be of value, such as age of a person or height of a person or weight of a
person or the temperature of a particular date.
So these would be the prediction of continuous values.
Categorical prediction could be something like if a particular loan should be approved or not, or if
a particular animal is a cat or dog or if someone is happy or unhappy.
So these kind of things are categorical difference.
So these kind of predictions can be made.
One is the continuous value and the one is a categorical value now for continuous values and categorical
values when we are trying to predict these values either continues or categorical.
These are called supervised learning problems.
That is, we are trying to predict a particular value here.
So these are called supervised learning problems and in case of supervised learning problems.
We have to find out our relationship.
We have to find out the relationship between the output value and the input value X.
So we have to create a function of X which would allow us to predict the Y value.
And why is the value which comes out after predicting from the X so Y value is the expected value y
Y is the value which we have actually predicted.
Now, what is y?
What is infix?
So here, when we are trying to predict the interest rate of a particular loan, then the interest rate
is a continuous value and interest rate will be the value, which is the output value, the label or
the target value, which is the output value, which is the interest rate, and what is our input value
or feature or attribute or independent variable.
These are the X values.
What are the X values here?
Idy each amount Sidonie dependent six children.
So whatever values we are using to predict this particular value, these are called independent values
or features or attributes.
And the value which we are trying to predict is called target or output value on the dependent value.
Now, let us discuss about different type of problems, so there is one problem where the bank is facing
loss due to loan defaulters.
So in this case, the predictors that is the features or attributes or the input, values or customer
details, credit history, loan applications, these are different criteria based on which we want to
find out if a particular person will default on the loan or not.
That is, we are trying to find out a target or a label or a dependent variable.
What is this dependent variable?
We want to find out if a customer will default on loan or not.
And the what will be the value of the target?
The value of target will be yes or no.
That is Zettl or one.
So this is called a classification problem where we are trying to classify something in a few classes,
so when we are trying to classify in two classes a this of classification problem.
Now, the next day, the problem is the flight prices keep fluctuating based on the demands on the holidays.
So here, what are the predictors which will actually help us find out, guy, the target value, the
features or attributes.
These are season nearest holidays, the month origin destination and what season is it?
So these things will actually decide if the flight prices will be high or low.
Now, the target value, that is the.
Value, which we are trying to find out the output value, is this revised price of the flight, the
new flight value.
So this is a continuous value because it is a price.
Price is a continuous value.
So this is a regression from.
So these are a few things that you need to keep in mind, although I will keep reminding you all of
these things while we will be working on different problems so that it slowly, gradually feeds into
your mind.
But just keep these things in your mind that what are.
What are classification problems, what are regression problems and how we are deciding on what are
predictors and what are target?
Now, when we are working on these problems, what happens is that when we are currently working on
these problems, the values of the predictors, and it will be actually given to us, we will be given
a proper dataset where we will be given.
The predicted values, I'm the dad, good values, and we will just have to perform the transformation
and after performing the transformation, we will bring on modern.
But in a real life situation, you will actually have to decide upon different predictors, you will
actually have to get the data from different sources and why you think data from these different sources,
you will be understanding which data is actually important and which data is not important.
So for that, actually, the data on summation and the feature selection, which we have done, will
be useful because right now what predictors will be given to you will be a narrower list of all the
features available.
But actually, when the organization will be reaching out to a data scientist, they will be handing
over a lot of data.
There would be hundreds and thousands of columns of data.
Then comes the situation when you will have to analyze each and every column of data and find out if
the column is actually relevant or not.
Most of the columns will be having less detail or they will be highly correlated with another column,
so which will help you on reducing the number of columns initially.
And after deciding on what columns are not correlated by moving columns, by using the practices which
we have discussed by feature selection, like using coordination matrix or using the affect or or by
using a Findus profiling and removing the columns using these additions provided by that, you will
be able to reduce the number of columns initially.
And after that, we will be discussing about different practices, which will allow you to remove more
number of columns.
So we will be learning those practices while we are learning different models so that you will also
get an hands on practice on that.
Now, next is.
These are the words which I have been talking about very frequently.
So what is Viva la, viva la is the output value, which is the label, the target or the dependent
value?
It is called independent value because we are expecting it to be dependent on the independent values
or the X values.
That is.
We are expecting the interest rate to be dependent on the age amount salary, the number of dependents,
the sex of the person and the number of children a person has.
So that is why this is called a dependent variable and these all of value X, I'll call the independent
video and I'll get back to the slide.
Now, what are these X values?
These X values are also called features attribute, input or independent.
Next thing is, while we going to be training our model, we will be formulating a function of X so
that the function of X will be equivalent to a value of vice.
But because this function will not always be an ideal scenario and a perfect function, so there will
always be a slight amount of error present.
Now, this slight amount of error is of two types.
One is the reducible error and another one is the irreducible error.
The reducible error is the error, which can be reduced by improving the data quality, the feature
selection, and by training the model properly.
While the irreducible error can not be reduced, so we will be targeting on reducing the deducible of.
Now.
When we are talking about data science and machine learning, the target is all about business, the
problem solving which we will be doing, so we will be identifying the problem initially.
We will be identifying that.
What is the problem if we need to find out the investigate or if we need to find out if someone will
be defaulting on the loan or not?
So what is the actual problem which we need to find a solution for?
And once we have the problem in hand, then we will I think that is the problem.
We will categorize the problem in the sense of if the problem is a regression problem or it is a classification
problem or is it an unsupervised problem where we don't where we don't want to.
Find out the actual value.
Well, we don't want to predict something, but we actually want to only classify something or group
something to be precise or cluster something based on the items of a present.
So let us say I want to identify a good number of customers, the customers which will actually be buying
my product.
So in that case, I don't want to classify the customers, I want to classify, I just want to cluster
the customers into different groups.
I want the group, the customers, in such a way that I can know that this group of customers will be
more interested in, let's say, coffee.
And that is another group of people who will be more interested in be so including my marketing.
I will be targeting the people who are interested in coffee for my coffee than I am for the B.
I will be targeting them for my fee product.
So here we will be applying the supervised unsupervised learning problem, like when I'm trying to find
out the weather for tomorrow or the temperature for tomorrow, then I will be using the supervised learning
from.
So what if we made the mistake in predictions so.
Here we have this value and we have those X values and we are formulating a function here.
So there will always be some amount of enterprising.
And the some of the evidence is actually the cost.
The sum of all the errors is for the cost of the wanton.
So if we make any mistakes, then we will have certain cost value, which we all will want to reduce
and how we reduce the cost value, we reduce the cost value by using it in the same.
So our main target is to formulate a function and while formulating the function, we will have certain
added value which we will be trying to reduce at maximum as possible.
So let's go ahead and learn about the aggression, so we will be discussing that in the next session.