Welcome back everybody.
This section is the most exciting because we're going to talk about a very crucial benefit that Kubernetes
provides.
And that is storage orchestration.
If you are deploying your apps in a traditional environment where you've got all of these machines running
all sorts of apps, while your operations team would be the one responsible for manually provisioning
storage inside of each machine, ensuring that that piece of storage is just the right size, and then
configuring each application to bind to that exact piece of storage.
And if you can imagine running that same app on another machine, then we have to repeat that same process.
So scaling up and down is just a really tedious process when you've got to manually provision every
single piece of storage, depending on the machine that your app is running in.
Well, if you take all of these machines, all right, and you have them collaborate under the supervision
of something like Kubernetes.
Then you can fully rely on the Kubernetes control plane to manage the storage for you.
And that's the benefit that we get by deploying our apps in a Kubernetes cluster.
We get automatic storage orchestration.
How does that look?
Give me just a second to get a drink of water.
It looks something like this.
So here we've got two pods.
Each one is running a containerized MongoDB database.
Whenever we save data to one of these databases, it's going to write that data to a local directory.
And you'll remember that pods are ephemeral.
They're very often destroyed and rescheduled.
So if we don't bind this pod to a durable peace of storage, then if that pod ever gets destroyed.
For whatever reason, the data inside of its underlying database would get destroyed as well.
So what can we do in Kubernetes to make sure that stateful applications can actually, um, back up
their data somewhere durable?
Well, what we do is we bind every single container to a persistent volume claim.
A persistent volume claim, which often is referred to as a PVC, is a specialized Kubernetes object
that allows you to declare how much storage your database needs and the storage characteristics.
All you're doing is you're making a claim for storage, and that claim is bound to your container.
When you make a claim for storage behind the scenes, the Kubernetes control plane is going to look
at all of your worker nodes.
Okay, I know we're just developing locally and we've got the one worker node, our laptop.
In this case, let's imagine we've got this Kubernetes cluster on AWS with all of these worker nodes.
When the Kubernetes control plane sees your claim, it's going to look through all of these nodes.
And within one of them, try to find a piece of storage, a piece of physical storage that matches the
storage requirements defined in your persistent volume claim.
When it does, it's going to bind that physical piece of storage.
It's going to bind that persistent volume to your claim, thereby allowing any data that gets saved
to your MongoDB database to be redirected to the persistent volume that lives inside of some node.
We refer to the pod as being stateful.
All right?
It stores data inside of a physical piece of storage within a machine, and we use stateful sets to
manage stateful pods, in contrast to when we used deployments to manage stateless pods.
Stateless pods you'll remember whenever we scheduled them for deployment, they were given some pod
name, the one we specified in our pod template, followed by a randomly generated name.
It doesn't really matter what that name was, mainly because all of these pod replicas, they are completely
interchangeable.
If one pod replica goes down, the other one can easily take its place.
One great submission portal Pod replica is going to do something very identical to another great submission
Portal Pod Replica.
It follows that scaling of pods within a deployment is unordered.
If we had four pods and we decide to scale down, which pod gets removed doesn't really matter.
Now, in a stateful set, every single pod has a unique identity Entity.
And you'll remember that because each pod replica is bound to a PVC when it gets destroyed, we want
it to be rescheduled using the same name as before, so it can reattach to its original PVC and therefore
gain access to all of its original data within the persistent volume that is taking up actual physical
space in the worker node.
Okay, so we need our pods to have unique identities, and it follows that pods within a stateful set.
Each one has its own dedicated storage.
Let's have a closer look.
So this is one MongoDB pod named MongoDB, one that runs an instance of a MongoDB database.
Whenever a data is saved to that MongoDB database, it's going to be saved inside of the local directory
slash data slash db within the containers is file system depending on which database software you're
using, whether that's PostgreSQL or MySQL.
The directory where data is saved will be different.
So if data is being saved in a MySQL database from memory, that gets saved to the directory var lib
slash MySQL.
In this case we're using MongoDB.
So the data gets saved to the directory slash data slash db.
You just got to read the documentation to know this okay.
Now what we want to do is bind this containerized database to a persistent volume claim.
The persistent volume claim is a specialized Kubernetes object that declares how much storage the database
needs, as well as the characteristics of that piece of storage.
Then the Kubernetes control plane is going to try to find a persistent volume that matches the storage
requirements we defined in our PVC.
Once it finds that actual physical piece of storage, what it's going to do is mount that piece of storage
on the local directory where we are saving all of our data, which means that if the pod were to ever
be destroyed, for whatever reason, the data will still live inside of the persistent volume.
And when the pod gets rescheduled using the same name as before, it's going to reattach to the same
PVC and therefore gain access to all of its original data.
That's the gist of it.
Enough theory.
Let's get to work.
You'll notice over here I've already deleted all the deployments inside of my grade submission namespace.
Please feel free to do the same as well.
Okay, once you do, let's go ahead and copy over.
Section six.
Call it section seven.
And inside of section seven I'm going to create a new file.
Call it MongoDB statefulset dot YAML.
All right I'm going to write Statefulset.
And it auto generates a skeleton that we can use to manage all of our stateful pods.
Okay.
Here we define the template upon which each stateful pod will be created.
And here we specify the behavior of our stateful set.
But let's not jump the gun and we'll start with the metadata.
I'm just going to call the stateful set MongoDB with um we're going to deploy it in the namespace.
Great submission.
All right.
Here we specify the behavior of our stateful set.
We're going to have it so that it manages any stateful pod that has the label.
You know what.
Let me start with the pod template.
I'm going to give every single pod, every single stateful pod the following labels.
The cloud native app is still a great submission.
Okay.
And now while the great submission API is the back end and the Great submission portal is the front
end within the great submission experience, our MongoDB database is a database.
Okay, that is the role that it will serve.
It will store all of the great submission data, and every MongoDB pod is going to be given the instance
label MongoDB.
It follows that here where we specify the behavior of the great submission of the MongoDB stateful set.
We can tell it to manage any pod that has the following label.
The service name is the prefix that will be used to name every single pod in the stateful set.
We're just going to have that be MongoDB.
It follows that any pod replica is going to be named using the MongoDB prefix, followed by some number
like 3 or 4.
The numbering goes in ascending order.
Okay, I'm sorry guys, my computer just shut down randomly.
I just had to reopen everything now to pick up where we left off.
Anyways, we've defined a stateful set that's going to be managing two stateful pods.
One of them is going to be called MongoDB zero and the other is going to be called MongoDB one.
Each one having a unique identity.
Now we can go to the pod template.
The pod template provides a skeleton upon which each pod is going to be generated.
So each stateful pod is going to have the following labels.
The instance label is what binds each pod to its corresponding stateful set, and within each pod there's
going to be one MongoDB container.
So here we specify the runtime requirements of each pod.
Each pod is going to be running a container called MongoDB.
And the MongoDB container is going to be created from.
The Mongo image.
Version 4.4.
So this image captures the MongoDB software.
Version 4.4.
What we want to do is pull this image, because from it, we're going to create a running instance of
MongoDB inside of a container.
Now the MongoDB software, by default it serves traffic.
It responds to requests.
Let me just verify on port 27017 follows that we'll need a container port 27017, where our containerized
MongoDB app can respond to traffic.
Okay, we don't need to give our container port a name.
We'll come back to volume mounts later.
And now let's talk about the volume claim templates.
So just as the pod template provides the skeleton upon which every pod is going to be created, the
volume claim templates provides a skeleton upon which each persistent volume claim will be generated.
So we're going to have one persistent volume claim per stateful pod.
Each persistent volume claim is going to start with the name MongoDB Persistent Storage.
So based on this template we're going to end up having two PVCs.
One of them is going to be called MongoDB Persistent Storage, and this PVC is going to bind to the
stateful pod.
MongoDB zero.
And another PVC will be created that binds to the pod MongoDB one.
Okay.
So this template we can use to generate many PVCs, each one binding to a unique pod.
All right.
And each PVC you'll remember it declares what type of storage we want Kubernetes to find.
For us.
We want to declare one gigabyte of storage with an access mode of read, write once, read.
Write once means that only one node can read and write to that piece of storage at a time.
Okay, so each where is my keynote.
So each MongoDB pod is now bound to a PVC, and then Kubernetes is going to do the hard work of finding
a persistent volume that matches the requirements we defined in each PVC.
So here what I can do is say, hey, Kubernetes, the persistent volumes that you find that correspond
to each PVC, I want you to mount them at the corresponding pods.
Local directory slash data slash DB.
All right.
And that's pretty much it.
And so ultimately we end up with the following scenario.
We've got two pods one MongoDB zero one MongoDB one okay.
Each one is bound to its own PVC, which somehow provisions a persistent volume which gets mounted on
that container as its local directory slash data slash debe, such that any data that gets saved here
will automatically be redirected to a durable peace of storage that exists on our machine.
All right.
Let's go ahead and apply the stateful set.
So what I did before is I deleted all the deployments kubectl delete deployments dash all and great
submission.
And I just want to start off on a clean slate.
All right dash dash all no resources found.
You should have no resources found as well.
Let's start clean kubectl apply dash f.
Um I'm just going to start off by applying the stateful set dot YAML.
I'm going to say kubectl get stateful set inside of the namespace.
Create submission.
A stateful set.
Sorry guys, it's so late right now it's 2 a.m..
Anyways, we've got one stateful set that's managing two stateful pods that were created based on the
pod template.
Now each pod is bound to its own persistent volume claim.
Um, I'm going to prove it to you.
Let's do it.
Kubectl get pvc dash n.
Great submission to persistent volume claims were generated on the fly.
Based on our persistent volume claim template.
We have one persistent volume claim that's bound to MongoDB zero and one persistent volume claim that's
bound to the pod MongoDB one.
Each persistent volume claim is going to make a claim for a persistent volume.
So I'm going to go ahead and create a MongoDB service dot YAML.
Service.
I name the service MongoDB.
And this service is going to select any pod that has the following label.
It's going to forward traffic to any pod that has the following label.
The service port is 27017, such that the Great Selection API can make requests to the host MongoDB
port 27017 and after traffic reaches the pod, the target port is used to direct that request to Mongodb's
container port.
That's going to be 27017 as well.
All right.
Let's go ahead and say kubectl apply dash f MongoDB service dot YAML.
Got our service.
We've got a MongoDB pod.
Now we need to instrument the great submission API to make requests to this service.
How do we do that?
Um, well, first and foremost, this version of the Great Submission API was not programmed to make
requests to a stateful MongoDB database.
So what I've got this is the one we've been using all along, but it's not been programmed to interact
with a database like MongoDB.
Uh, stateless V2 is a great submission.
API contains a great submission API app that was programmed to interact with MongoDB.
So we're going to pull this image.
Up.
Ultimately it just adds a hyphen v2.
From this image we're going to create a great submission API, a containerized great submission API
that was designed to interact with a database.
Let me actually show you the code.
So here our application expects two environment variables one MongoDB host one MongoDB port.
These environment variables are necessary in order for our great submission API to establish a connection
with the MongoDB database through the cluster IP service.
Okay, so our container needs to provide its underlying application with these environment variables.
Okay.
And you'll notice very interestingly that I gave the great submission API.
A very simple readiness endpoint that sends back to its readiness probe a status of 200 if it's connected
to the MongoDB database, and a status of 503 if it's not ready to serve requests just yet.
Um, let's go ahead and give our stateless V2 app the environment variables that it needs so we can
do that somewhere here.
Environment variables.
The first environment variable is named.
The first environment variable that it expects is called MongoDB host.
And we need to give it a value that equals the service name.
The second environment variable it expects is the MongoDB port.
And it needs to be given a value that equals the service port, because ultimately the service will
act as a proxy between the Great Submission API and the MongoDB database.
All right.
And that's pretty much it.
Pretty sure we can just deploy everything kubectl apply dash f.
Kubectl get pods dash and great submission.
Let's try this again.
All right.
Notice that the great submission API its containers were created but it still shows zero out of one.
Why is that?
Because the readiness endpoint is sending back 503.
Because it's not ready.
It hasn't connected to MongoDB just yet.
All right.
But once it does connect, it should send back a status of 200.
So if I were to check back again.
Both great submission API pod replicas are ready.
Beautiful.
I'm going to say kubectl logs dash f dash n.
Great submission.
Connected to MongoDB.
All right.
Nice.
Now, one thing I want to note is that notice how the stateful pods have unique identities, unique
names where as the stateless pods have randomly generated names at the end.
That's because the stateless pods are completely interchangeable and the stateful pods are not.
Each one is bound to its own PVC and has access to its own piece of storage.
These ones are stateless.
Whether one gets deleted or replaced by the other doesn't really matter.
All right.
But now, ultimately, to make sure everything works, we're going to start by making a request to the
great submission portal.
All right.
So now from our great submission portal, we're going to be making requests to the Great Submission
API upon submitting great data.
And that is going to persist our data to one of the MongoDB databases.
Okay.
One of these databases will have one piece of storage.
The other database will have another piece of storage.
The cluster IP service is going to round robin our requests between MongoDB zero and MongoDB one.
Each one has their own dedicated storage.
Why will that be a problem?
Let me show you in a bit.
Harry Potter will give him a score of 54 in potions if I submit it, it may or may not show up.
Let me try it again.
um, I don't know.
Ron, 32 potions.
I'm going to submit it.
Try this again.
You might have experienced it already, but just wait for me to experience it.
Neville, 87 and potions.
All right.
Notice how the data is different now.
We've got two databases, all right?
Not just MongoDB zero.
We've got MongoDB zero and MongoDB one.
Whenever the Great Submission API makes a request through the cluster IP service, the cluster IP service
is going to round robin the requests between the two databases, which means they're going to hold different
data.
So whenever that page gets refreshed it might make a request to MongoDB zero or MongoDB one, which
intrinsically have different data sets.
Okay, here I made a request to one of the databases that had Neville.
Now it made a request to the other.
Database that has Harry Potter and run.
And so I only deployed to.
Databases just to show you how the volume claim templates work.
But it only makes sense to have one.
MongoDB stateful pod that's bound to a single persistent volume claim, with one piece of storage mounted
at its container slash data slash DB.
We only want a single MongoDB zero.
That is all.
Okay, so why don't we say kubectl apply dash f MongoDB statefulset reconfigure what we've got.
Kubectl get pods dash n great submission.
So by scaling down, it took off MongoDB one.
Now we're just left with MongoDB zero.
Both of these, they're both ready.
So the readiness endpoint send back a positive response meaning they're both connected to the database.
Beautiful.
Now we should see more consistent results.
All right.
So it left the database that had Harry and Ron.
And it follows that um whatever I submit going forward.
Should remain consistent.
Okay.
I accidentally pressed submit twice.
Sorry about that.
If that confused you.
Let me do it again.
We'll say Snape 98 potions.
I won't press the submit button twice now.
Okay, beautiful.
Uniform.
So now the great submission API, no matter what, is making a request to.
One stateful pod that is bound to one piece of storage.
Okay, everything should work.
Beautifully.
Now, I want to demonstrate to you that all of this data exists in a durable piece of storage within
my node.
I can do that by saying kubectl delete all the pods inside of the great submission namespace.
They're going to get rescheduled because they're being managed by the stateless pods are being managed
by a deployment.
The stateful pod is being managed by a stateful set.
And once MongoDB zero is recreated using the exact same name, it's going to get reattached to that
PVC and gain access to the original data inside of the PV.
So even if this pod gets deleted.
Upon it being rescheduled, we're going to see our data.
Again, the Great Submission API will be able to access that original data from MongoDB zero.
Okay, finally it's done.
I can stop talking and just demonstrate dash and great submission.
All right.
Notice the great submission API is not ready to receive traffic yet.
It hasn't connected to the MongoDB app.
Aunt.
Readiness endpoints.
Just beautiful.
No matter what you do when you containerize your apps, give them liveness and readiness endpoints.
Trust me, you will not regret it.
All right.
If I refresh, all the data is still there.
That's all for stateful pods.
PVCs.
PVS.
I hope you enjoyed this lesson.
We did bring everything together by also incorporating, um, internal pod to pod communication with
cluster IP services.
That's something we covered in the service discovery section as well.
I hope you can just appreciate just how amazing Kubernetes is in provisioning storage.
All we got to do is define our storage requirements without having any idea what the underlying infrastructure
is.
Kubernetes will do all the hard work of provisioning that storage and in essence, binding it to our
application no matter what node it's running on.
I know we're just prototyping in our node is simply our laptop, but this becomes all the more handy
when we're actually running things in an enterprise, in a cluster that's probably running on an actual
cloud provider.
Okay, I hope you enjoyed this lesson.
I'll see you in the next one.