Now that we have the docker compose ready with the required changes, it's just a question of spinning
up the containers.
We have already went over some Docker commands briefly in the previous lecture, but here we'll cover
them in more depth.
Before you can do this, make sure that Docker is running.
You can check this by going on Docker desktop and at the bottom left make sure that the engine is running.
Once you know that it is running, run the docker compose up minus d command where the minus d flag
means that the containers will run and detached mode, meaning they will run in the background.
Let's press enter and the first time that you run it, since you probably won't have the airflow, Postgres
and Redis images, it might take some time to run the docker command until the images are all installed.
Ultimately, what we want is to get to the point where the containers we need are running and healthy.
We will continue when the containers are running.
So it looks like the containers are now running.
We can confirm this by running the docker PS command.
By default, this command will only show the running containers.
And here you will see things like the image that is being used, the status ports and even the container
names.
A powerful command that you can use to access the containers is Docker exec commands.
So if, for example, you want to access the scheduler container, we would use the command docker exec
with the minus ID flag, which I will shortly explain.
We will be using the scheduler container and then we use the bash command.
So before we move further, the minus eight flags essentially give you an interactive shell session
inside the container.
And the bash command at the end invokes the bash shell.
Now we are inside the airflow scheduler container.
We can do a simple check while we are inside this container to see whether the JSON data file that is
inside the data directory on our local machine is also inside the Docker container we have here.
Let's check this right now by running CD command into the data directory.
So CD into data.
And then we'll run a simple LS command to see what we have.
As you can see, the JSON file is also present inside the Docker container.
Now in terms of airflow commands, if you write the following command airflow dash dash help, you will
get a list of commands that you can use inside airflow.
You can go through them in your own time.
Another command that you can use is airflow info to get information about the airflow environment.
So here, for example, we know that we're using version 2.9.2 with this particular executor and so
on.
As you can see there is a lot to explore and I invite you to do so in your own time.
One final command that I would like to share with you with regards to airflow is the airflow cheat sheet
and variables mode.
And here you will get a list of airflow commands that you can use in your own day to day.
I will leave you with two final docker compose commands, the first one being docker compose up minus
d dash dash build.
This is a useful command when you want to rebuild your services with, for example, updates in the
docker file.
It's a quick way to apply new builds without stopping all services, keeping your container orchestration
smooth and running.
If then you want to completely stop your Docker containers, you would use the command docker compose
down.
So this is a command that you should run if you want to apply bigger changes in your docker compose,
like adding or removing services or changes to named volumes.
So you would run this docker compose down command and then to spin up the containers again you would
run docker compose up with the minus d flag.
So let me run docker compose down.
And here you are seeing Docker containers are stopped.
Something that I want to mention for the sake of completeness is.
In a previous lecture, I had said that the volume would be the volume that you defined here in the
Docker compose.
As you can see, this is roughly similar, but you also have the YouTube underscore at the beginning.
This is there because the repository that we have is named YouTube underscore ELT.
So the actual named volume by default will show as the name of the repository and then the volume defined
here.
One last piece of advice that I can give you is to always run the docker compose down command when you
want to shut down your laptop, to prevent potential issues such as conflicts.
The next slide will address the situation of this potential issue, but is not something that you should
worry about if you properly shut down your Docker services after you have finished using them.
I will see you in the next section of this course, where we will go over the Python scripts we built
in the first section and apply the changes to create the airflow DAGs.