Welcome back, everybody.
In this section, I want to discuss what's arguably the most powerful feature that Kubernetes provides.
And that is horizontal pod auto scaling.
Okay.
So this is the ability to automatically scale your pods based on the amount of traffic, based on the
amount of load that they experience.
And in order to truly weigh how important this feature is, let's imagine an e-commerce application
that's been regularly serving traffic to its customers all throughout the year, and then suddenly they
get promoted by a YouTuber or something, and they start getting a very sudden spike in traffic that
their application simply isn't ready for.
So the app becomes really slow and unresponsive, which hinders the user experience.
Uh, less customers are purchasing through the website, losing a lot of money.
So the operations team and all the IT guys, they get woken up in the middle of the night to scale the
deployments up in a in a very hurried panic But by the time they did scale the deployment up.
Um, many customers already abandoned their purchases.
And then after this big sale is over, they need to scale the deployment back down.
Otherwise, they'd be wasting very valuable computing resources.
And that takes a lot of time to do that as well.
But also by by scaling your manual.
By manually scaling your deployments back down, you risk potential future disruptions in the event
that there is another, uh, unexpected future promotion.
So all that is to say that manually scaling your deployments up and down is a very tedious, stressful,
and inefficient process.
And the fact that Kubernetes can automatically scale application pods based on the amount of load that
they experience is really, really powerful and really, really important.
So we're going to discuss what it is in this lesson.
And I'm going to show you a demo of horizontal pod auto scaling in action.
But first and foremost, why is it called horizontal auto scaling?
Well, let's imagine that you're this magical construction worker that can create buildings at the snap
of a finger.
Okay, there is hundreds of people outside.
It starts raining and they need to get into the building for cover.
But the building can only accommodate 20 people.
And so you've got one of two options you can, at the snap of a finger, make the building a lot bigger.
So grow it vertically in order to accommodate more people or at the snap of your finger, you can create
many more buildings alongside the original one in order to accommodate all of that traffic.
Okay, so this would be considered horizontal scaling.
This is vertical scaling.
If we relate this back to pods, uh, vertically scaling a pod would just mean giving it more resources,
more CPU in order to handle all of that traffic.
Whereas horizontal pod auto scaling Autoscaling means creating more pods and distributing that traffic
across all of these pods.
This is the preferred scaling approach, and it's what we're going to be discussing in this video.
So how does it work?
You've got this horizontal pod autoscaler abstraction that's provided by Kubernetes.
This horizontal pod autoscaler object where you can define your minimum number of replicas, your maximum
number of pod replicas, and the target by which the HPA should scale your application pods.
Okay, so here we're saying if the average CPU utilization of all the pods ever exceeds 50%, scale
up.
If the average CPU utilization is way below 50%, then scale down.
So here we've got a pod that's been allocated 200 millicores of CPU, and it's only using up five,
which means the average utilization is 3%.
All right.
But then suddenly, our pod starts getting so much traffic, um, that it's using up 55% of its allocated
CPU.
So the horizontal pod autoscaler decides to create another pod and distribute the load.
Now the average CPU utilization is 19%.
And now at this point, traffic is starting to decrease.
So the autoscaler decides to scale back down to one pod.
Now the average utilization is only 16%.
This one pod is able to handle things perfectly and then there is no traffic at all.
We're down to 1%, only one pod.
But then suddenly we get this sudden burst in traffic.
110 out of 200 CPU is being used 55%.
So the horizontal pod autoscaler scales back up, distributes the load, but the traffic is so high
that even two pods are using up.
Most of their CPU at an average CPU utilization of 60%.
So the autoscaler creates another pod.
Now the load has been distributed enough such that the average utilization is 40%, and all three pods
were able to handle the traffic with ease until there was no more traffic.
They all came back down to an average CPU utilization of 1%, which is way below the target.
So the autoscaler brings, uh, scales back down to a single pod.
All right.
So it's simply an object that scales your pods up and down with the amount of traffic, with the amount
of workload that's being experienced.
Now, CPU is the preferred metric for auto scaling, not to memory when there is a lot of traffic,
we want the autoscaler to scale our pods up.
CPU and memory.
They both usually increase as traffic goes up, but then when traffic reduces, we want the auto scaler
to scale our pods back down because resources, they cost money.
Now CPU because it's compressible, it will usually quickly decrease as the workload decreases.
Memory, on the other hand, is incompressible, so it will not reliably decrease as workload decreases.
So if our auto scaler were based on memory, then we would end up with one that only ever goes up,
but rarely ever scales our pods back down.
CPU usage, on the other hand, is more volatile.
It goes up and down very quickly with increasing and decreasing traffic, which allows the auto scaler
to move in both directions to scale our pods up and down.
So obviously in this course, we're going to be using CPU usage as the metric for auto scaling.
All right that's enough theory.
Let's go ahead and implement it.
So inside of the grade submission portal actually you know what.
Let me clear the output first.
Um, I'm going to copy this.
Call it section nine.
And there's one thing I forgot to mention is that in Kubernetes, there's something called a metrics
server that collects CPU and memory usage metrics about all pods and nodes in your cluster without metrics
being collected.
Things like the horizontal pod autoscaler wouldn't work.
So we need to install the metrics server in our Kubernetes cluster before we can perform horizontal
pod auto scaling.
All right, let me go and look for the command that I'm going to give you right now.
So if you go to your resources folder of this lecture, I've left you a file that you can download
And that file has the following Kubernetes command.
Okay.
Enter this command and it should install everything needed to set up the metrics server deployment.
I'm going to clear the output CD out of section eight and CD into section nine.
All right.
Now if you say kubectl top pods dash n great submission.
So this, uh, displays the memory and CPU usage metrics of all the pods in the great submission namespace.
You should get something like metrics API not available because it takes time for the metric server
to install.
I already had it installed like a couple of weeks ago, so it automatically displays the metrics because
they were always being collected.
So I've got my grade submission API consuming five millicores of CPU.
54MB of memory MongoDB using up nine Millicores of CPU.
222MB of memory.
All right.
So I want you to pause this lesson and only resume when your kubectl top pods command actually displays
CPU and memory data for all of your pods.
Okay, once you've got that set up, we can clear the output.
And then inside of the grade submission portal I'm going to create a grade submission portal autoscaler.
I'm only going to demo auto scaling for the grade submission portal.
All right.
Just to keep things simple, because the auto scaler is not something that we're going to carry over
in the remainder of this course.
We don't actually need it because we're prototyping.
All right.
But in an enterprise environment, an auto scaler would be very, very necessary for reasons that were
mentioned earlier.
So if I want to set up an auto scaler using the Kubernetes extension, I can just say HPA and auto generate
a skeleton upon which we can create an auto scaler.
I'm going to call the auto scaler.
Great submission portal.
Uh, HPA.
All right.
And this auto scaler is going to scale all of the pods being managed by the Great Submission Portal
deployment.
The HPA is going to be scaling these pod replicas up and down based on CPU usage based on average CPU
utilization.
We can actually leave this as it is scaling based on 50% average CPU utilization is very common.
It's a lower end for sure.
50 to 60% because that kind of provides a buffer for sudden spikes in traffic.
It allows time for new pods to start up before the system becomes overloaded.
So.
So the minimum number of replicas will be one.
The maximum number of replicas is ten, and we're going to scale from 1 to 10 based on the average CPU
utilization of 50%.
Let's go ahead and say kubectl apply dash f everything inside a grade submission portal.
And it should simply create a horizontal pod autoscaler.
Everything else should remain unchanged.
I'm going to say kubectl get HPA dash n grade submission.
No resources found I always do this.
Namespace grade submission kubectl.
Let's delete the horizontal pod.
Autoscaler.
Um, that's inside of the default namespace.
Okay, let's redeploy it in the grade submission namespace.
All right.
Kubectl get pods.
Dash, not pods.
Get HPA dash n grade submission.
All right.
So we've got one autoscaler that's going to be scaling our pods based on an average utilization of 50%.
Okay.
Here we see unknown out of 50% because it's still monitoring the metrics for the grade submission portal
that are being collected by the metrics server.
We just deployed the autoscaler.
So it's going to take time for this value to show.
Let me get a drink of water real quick.
So hopefully it's done it by now Let's say kubectl top pods dash n.
Great submission.
So that displays the CPU and memory for the great submission portal.
So the great submission portal is using up 16 millicores of CPU, um, from it's assigned to 100 millicores.
So 16 divided by 200.
Let me calculate that real quick.
That is 8%.
So we expect to see an average CPU utilization of 8% out of 50.
All right.
Please show me something beautiful.
And so now what I'm going to do is I'm going to flood this pod with requests and have its CPU go above
the 50% target.
Once that happens, the horizontal pod autoscaler is going to increase the number of pods in order to
Distribute that load and try to bring the average CPU utilization back down.
And it's ultimately just going to scale our pods up with increasing traffic and scale our pods back
down with decreasing traffic.
So I'll go ahead and demo this right now.
I should have postman somewhere.
Oh let me bring it up so you can see it right over here.
So I've already set up my postman runner.
This is not a postman tutorial, so just enjoy the demo.
I'm just going to demo to you how horizontal pod auto scaling works.
Ultimately, you should just observe how the Horizontal Pod autoscaler reacts to a lot of traffic.
So I'm going to simulate real world traffic from my local machine, and I'm going to have 20 users sending
traffic to my app at a time.
I'm going to click run.
So this is going to make consistent requests to localhost 32,000.
I can already see that responses are starting to become really slow.
I'm assuming the CPU has already gone up a lot.
So let me say kubectl get pods dash n great submission.
Wow.
And the horizontal pod autoscaler has already created three great submission portal pods.
We started with one pod replica and the great Submission portal.
Autoscaler has already created two more in order to handle all of the load, all of the burden that's
being placed on it.
All right.
And that's pretty much it.
If you have an application that's bound to receive a lot of traffic, you should consider giving it
a horizontal pod autoscaler.
Okay.
That's all we're going to cover for this section.
I will see you in the next one.