In order to create the CI CD process, we need to first understand a bit more with a GitHub workflow
is in its simplest form.
A GitHub workflow is an automated process made up of one or multiple YAML scripts, which contain a
set of steps to execute.
You define these YAML files in the GitHub workflows directory, so let's create it right now.
Let's create a folder dot GitHub.
And then here we will create a subfolder called workflows.
And let's also go ahead and in our case create one YAML file that will contain all the ci CD workflow.
Let's name it something like ci CD for YouTube.
So new file ci CD, YouTube, elt dot YAML.
Don't worry if it's showing in red.
Once we create and save, this will no longer be red.
Also, this icon is different from what you might have.
Since I have the GitHub actions extension, each GitHub workflow YAML file will vary, but there are
certain elements which are common in almost all of them.
We will now go through the YAML file and customize it for our needs.
The first line of YAML file will contain the name of the workflow.
This name will also be displayed in the GitHub actions interface.
Let's give it a name.
For CI CD pipeline.
Next we need to define what events trigger the workflow.
The key word en is used to trigger a workflow, and in this case we want to trigger on three different
scenarios.
It will either be a push on the main branch.
It could also be ideal because of a pull request on again the main branch.
And also, for those who don't know what a pull request is, it is basically a proposal to merge a set
of changes from a feature branch into a target branch, which in this case is main.
This would be a good idea to include workflow dispatch, which is useful when you want to allow manual
triggering of the workflow for debugging or special cases.
This manual triggering can be done on the GitHub actions UI, and we will see it when we come to the
manual triggering part of this section.
Now in our case we only worked with the main branch, but while developing you will use some feature
branches to test out code before pushing to the main and before pushing to main, we would want that
our CI CD pipeline is tested.
So we can do one small change here and include the feature branches.
And this will be written as follows.
So this feature here, what it signifies is that the pipeline will be triggered on any push to a branch
starting with feature slash.
Now that we define the triggers, we can move on to the jobs which are effectively the tasks to run.
And each job contains a sequence of steps.
That's right jobs.
And let's start by defining the continuous integration jobs with the first CI job being building and
pushing the Docker image.
If either the docker file or requirements.txt change.
So we can define the name of the job and which runner will be used to run the job.
In our case, this will be using the latest ubuntu image.
This will look as follows.
Here I am naming the job And I am also saying the runner, which will be the ubuntu latest image.
When we talk about runner, we are referring to an environment where the job is executed and GitHub
will provide a hosted virtual machine.
So coming back to our code, as I said, a job consists of steps.
And here we declare the steps that will be performed in the specific job.
So we use the step scheme.
The first thing that we need to do is to get the code from the repository we have on GitHub.
GitHub actions provides a built in action called the Checkout Actions.
It allows you to have access to your code base and the recording of this video.
We are at v4 of this action, so we will use that.
I will leave a link in the appendix of this section to the checkout action which you see over here.
So let's place the checkout action in our code.
And now that we have access to our repo based, we need a way to check if the Dockerfile file or the
requirements.txt have changed in the current commit compared to the base branch.
Also, we are checking only these two files, since the docker image is based on solely these two files,
and to do this we can use another inbuilt action which is called changed files.
This would also be in the appendix of this section, and is the action that you see in the screen here.
So writing this out we have the name which is the description of the step.
We specify the ID of this step so its output can be referenced in subsequent steps.
This is a very important step.
And now we can specify the actual action which we are invoking which is currently at version 45.
What is left then is to specify the files we want to monitor, which we can do as follows.
Now that we have a way of checking if the docker file or the requirements.txt has changed, we can start
the process of rebuilding Docker image.
We will do this by first configuring Docker to use the build tool for building Docker images.
For this we will use the setup build action, which is another action of GitHub actions currently at
version three, which is what you see on the screen right now.
Again, all of this will happen if and only if the docker file or requirements.txt have changed since
the last commit.
So we can combine all of this together and we can therefore write the following.
So what I'm doing right now is simply formatting, as I'm seeing that on some lines I'm getting errors.
And this is because for example name and uses should be indented to the same indentation.
Same goes for name id AD and also with they all have to be in the same indentation.
So with the build next step, we have only written the setup of the configuration.
But to actually build and push the Docker image, we need to still write the code for it.
And even before we can do this, we need to ensure that we are first logged in to Docker Hub where our
image resides.
Since we said we need to log in, that implies that we need to pass sensitive information like Docker
Hub password in order to avoid hard coding this or any other variable for that matter.
We can use GitHub actions, secrets and variables which you can access by doing the following.
So first Google in your GitHub repo.
In my case this is this one.
Over here press on settings on the left hand side.
Down here under security you will see secrets and variables.
Click on it and press actions.
Here you will see two options either the secrets or the variables which, as you might have already
guessed.
Secrets is to store sensitive data and variables is for the non-sensitive data.
Inside both secrets and variables, you have the option to define both the environment and repository
secrets and variables.
This is a great thing to have to distinguish between your development and production environments,
where your secrets and variables will be different if you are not working with environments, which
is our current use case, you can use the Repository Secrets, which will define your secrets variables
across environments.
So let's use that one.
We can now define the Docker Hub namespace and Docker Hub username in the variables section.
So let's go here.
Click New Repository variable and define the key under the name and the value for the specific value
it will have.
As a side note, regarding the namespace and the username.
By default these will be the same, but it is good that we make a distinction between the two as the
namespace is essentially a container for your repositories, and the user name is the unique identifier
for your Docker Hub account.
So let's define the Docker Hub namespace and username.
And in my case these will have the value as follows.
I will also define the username.
And while we are here we can also specify the Docker Hub repository variable, which will be the repository
name we pushed to Docker Hub.
In my case, the name is as follows.
So the key will be Docker Hub repository and the actual value is the YouTube underscore API underscore
ELT.
What is left to define is the Docker Hub password.
Obviously we will specify this in the secrets of GitHub actions.
I will define this password here.
Make sure that's from your end.
You define your Docker Hub passwords under the secrets.
And as you can see, the actual contents of the password will not be shown, unlike the variables.
Here the variables you will have the value present, but for the secrets this will be hidden.
Having done these steps, we can go back to the YAML file.
So first we need to add this step where we are logging in to Docker Hub.
This will look as follows.
And to login we will use the login action which as of latest is at version three.
Here you will see the documentation for the login action.
The login action will look as follows.
Notice how here we are calling the Docker Hub username and password, which we just defined in GitHub.
Actions, secrets and variables.
Since password was defined under the secrets, we use this secrets dot and the variable key.
And since the username was defined as a variable, we use the dot Syntax.
Now that we are logged in, all that is left is to build and push the Docker image to our own personal
Docker Hub account.
Again, this is only needed when there is a change in the requirements of the XD or the docker file,
so we can write the following.
From here we will touch upon the topic of Docker image tags and we will give the image two tags.
We will have the latest tag to indicate the image being pushed is the most recent stable image.
This will look as follows.
Again, we note that here we are making use of the GitHub actions variables that we just used a few
minutes ago to define the Docker Hub namespace and the repository.
Coming back to the tags, we also have a second tag based on the commit hash, which is GitHub actions
variable that uniquely identifies the image for the specific commit.
This will look as follows.
The commit hash being this path over here.
At this point, you might ask, why do we have the second tag with the commit hash?
This is important as it acts as a unique identifier, ensuring that we have traceability to the exact
code that produced the image.
This is equivalent to what we were doing with the EMV.
Specifying the image tag variable and setting it to, for example, at the start 1.0.0, and then when
a change was done to the image, we set it to 1.0.1.
The very last thing which we can forget is that we need to include the build context for Docker, which
in our case is the current directory.
So we specify this dot over here before we can conclude the first job of our CI pipeline.
We need to perform some changes in Docker compose.
Currently in Docker Compose we are using the Docker image with this image tag.
Environment variable.
In the docker compose we are referencing the environment variables which we get from the EMV in this
ubuntu environment offered by GitHub actions.
We will not have access to the env as it was being versioned by git, so we will need to specify all
the environment variables you see here as GitHub actions, variables or secrets.
For now, the only change we need to do is to change the image tag variable to latest, which will be
the same as we specified in the tag command in the workflow YAML file.
We will do further changes to the docker compose, but these will come later on.
So with that we finished the first CI job.
Now we can move on to the second and final job.
This will join both the CI where we run the unit and integration tests, and also the CD where we run
the end to end tests.