So in this lecture, we will be looking at the intuition behind these bays now in order to understand
now Hubei's, we must first understand Bayes rule.
So this lecture will assume that you have at least a passing knowledge of probability, which you should
as per the instructions for this course.
So let's begin by reviewing Bayes rule.
Suppose that we have two random variables X and Y.
Now, if you're concerned that these are just abstract variables, please do not worry as we will be
doing examples very shortly.
So let's also suppose that we want to know P of why given x.
Note that this is a conditional distribution because Y is conditioned on X.
Now suppose that we are given X, given Y and also the marginal distribution p of Y.
If this is the case, then we can express what we want to know in terms of what we do know using Bayes
rule.
So on the numerator we have X given Y times, b y and on the denominator we have the same thing except
sum over all possible values of Y.
And just as a reminder, recall that the numerator can be simplified to be of X and Y, while the denominator
can be simplified to P of X.
So that should give you some intuition about why Bayes rule is the way it is.
Now, let's discuss Bay's role in terms of classification for machine learning.
In this case, the input to the model is X, while the target is why.
So these variables are no longer abstract but represents something we want to measure.
For example, why could be whether or not an email is spam?
Well, X could be the email itself.
Note that in this context, because we are not doing a regression, why is a discrete or categorical
random variable?
As such, although we write P of Y given X?
Note that this is not one value, but a whole distribution.
So, for example, if y can take on the values, spam and not spam, then we might find that P of Y
equals spam, given X is equal to zero point three, while P of Y equals not spam, given X is equal
to zero point seven.
So the number of probabilities in the distribution is the number of classes we have.
Furthermore, note that this makes it easy to formulate a decision rule.
We simply pick the class, which has the highest probability.
Mathematically, we choose the Class K star, which is the ARG Max appeal y given x overall values of
Y.
So in our example above, we would choose not spam, since zero point seven is bigger than zero point
three.
Now, it's important to remember not to be overwhelmed by math symbols.
If you happen to have any math phobias, intuitively, this rule makes perfect sense.
If the probability of an email being not spam is bigger than the probability of that email being spam,
of course, I'm going to classify it as not spam.
In fact, we can think of an even more intuitive example.
Suppose that you're about to play a game, say, betting on a horse race.
Luckily, I have insider knowledge, and I can tell you the probability that each horse will win.
So suppose that there are six horses.
Horse number one has a 50 percent chance of winning.
Well, all the other horses have a 10 percent chance.
Note that you only get to play once and you must bet all your money on a single horse.
In this case, you should pick horse number one instead of any of the other horses because it has the
best chance to win.
So hopefully that's pretty intuitive.
OK, so now that you understand the basics of the decision rule, let's go through a full example where
both X and Y represent concrete things.
Suppose that Y represents whether or not a patient will have a severe immune reaction to COVID.
We can think of that as whether or not they need to go to the hospital.
So I hope that you'll find this example very intuitive as it is pretty contemporary.
And hopefully you've read about these factors in the news.
If you have it, please let me know in the Q&A, and I'll be happy to share some news articles with
you.
In any case, note that this is categorical.
We can denote the classes as severe and mild for this initial example.
Suppose that X is just one measurement, which is the patient's BMI, as you recall.
BMI stands for body mass index, which is more or less your weight divided by your height.
It is a common but flawed measurement of one's body fat.
However, for this example, I think it's the easiest to understand.
So given this information, we can now write down how to compute the probabilities we previously discussed.
Remember that we know all the values on the right side, and we want to find the value on the left side
after which we can use those values to make a prediction.
Note that we call the distribution on the left side the posterior.
So it may be helpful to think about how each of the values on the right side will be measured for peace
of year and mild note that these are called priors.
They represent the rate of severe or mild reactions, given no other information.
That is, they are conditioned on nothing.
In practice, they would simply be computed by counting.
So, for example, if you have 1000 COVID patients in 100 of them had severe reactions, then that means
peace of year is 10 percent and thus p mild would be 90 percent.
Now, let's consider a PE of BMI given severe and PE of BMI given mild.
Note that these are called likelihoods.
So one plausible solution is to model these as Gaussians.
As you recall, the Gaussian or a normal distribution, is the famous bell curve.
Recall that it is fully specified by its mean and variance.
Thus, what you would want to do is collect all the severe patients and compute the mean and variance
of their BMI.
This would fully specify the Gaussian distribution p of BMI given severe.
Of course, we could do a similar thing for the mild patients as well, and that would give us P of
BMI given mile.
Now, because this is just an intuition lecture, we're not going to go through any calculations, but
feel free to do that on your own if you feel the need to try it out.
Now, typically in machine learning, we don't just have one input in the previous example, our only
input was BMI.
However, it may be the case that we could make more accurate predictions by using more information
about the patient, as you recall.
Age is a major factor as well.
So suppose X is now a vector with multiple components, one for BMI and one for age.
Of course, the computations we would do are basically the same as what I've shown you so far.
The only difference is that everywhere you previously saw only BMI.
Now you see BMI and age.
So what is naive Bayes and how is that different from simply using Bayes rule?
This all has to do with the form of the likelihood continuing on with our example that would be a p
of BMI and age, given why note that we're just using Y is a generic variable, which in this example
could represent severe or mild.
So as we've discussed it so far, this is a very general way of looking at things.
We haven't said anything about the structure of this distribution.
It could be Gaussian exponential or any other kind of multivariate distribution, even one that we do
not know how to compute.
The most important factor pay attention to is that BMI and age do not have to be independent, which
makes sense.
It could very well be that BMI is affected by age.
That is, as one ages.
Perhaps it becomes more difficult to control BMI.
Well, what makes this naive is is that we make the naive base assumption.
The Navy's assumption is that all the inputs are independent.
Essentially, this makes everything easier to compute.
And as engineers, we have no problem making potentially unrealistic assumptions if they make things
easier to compute.
Now, one common way to visualize the naive based model is with a graphical model.
Personally, I don't find this that useful, but perhaps you might.
So in this case, each circle represents a random variable and each arrow represents a dependency.
In this case, we can see that all of the X's are dependent on why this makes sense.
For example, if y a severe, then you could imagine that the input for age would be larger on average,
thus ages affected by what taking on a certain value.
Importantly, note that all the X's are independent of each other.
That is, there are no arrows going from any X to any other X.
In this diagram, the X represent each individual component of our input.
So in our covered example, we would have two axes, one representing BMI and one representing age.
In general, we will just have x one x two and so forth up to XD, where D is the number of input features
of our data set.
OK, so as mentioned, what makes naive Bayes naive is that it assumes that all the inputs are independent.
However, this still doesn't say anything about what kind of distribution we should use for PLX, given
what, in fact, this is totally up to you.
However, if you choose something exotic or unconventional, note that you would have to implement it
yourself.
In fact, it's not even required that all the excess come from the same kind of distribution.
Perhaps it's the case that X1 is Gaussian, but X2 is exponential and X3 is Bernoulli and so forth.
But in second, learn there are predefined at naive based models which use specific likelihood distributions.
These are the Gaussian, the multi gnomeo and the Bernoulli.
Note that for these pre-built models, all the exits come from the same kind of distribution.
For example, if you choose Gaussian naive Bayes, that will mean all your exes will be Gaussian.
Of course, your next question will be, well, which one do I choose for my specific problem?
And of course, that depends on the distribution of your data.
If your data is continuous and looks like a bell curve, then the Gaussian would be a good choice.
If you have count data that comes from a categorical, then the multi gnomeo would be a good choice.
Note that this is typically the correct option for NLP and specifically count vectors or to fight a
yes.
And if your data is binary, then Bernoulli would be a good choice.
This might be applicable if you choose the binary version of count vectors for NLP.