The Docker file.
At its core is a script containing a series of instructions on how to build your custom Docker image.
The first step is to actually create the Docker file.
In our case, Docker image that we will build on is the airflow image, which you can find on Docker
Hub.
This is the official airflow image you see here.
If we were to press on tags.
We would see that there are many different tags depending on the airflow version and the type of Docker
image we want to use.
For example, this Docker image is using the Docker slim version with the Python 3.9 version.
In our case, we will use airflow version 2.9.2, which is a more recent airflow version.
And with that, we also want to specify the Python version we want to use since airflow is built using
Python.
In our case, we'll use a more recent version of Python as of recording of this lecture, which is Python
3.10.
If we were to write what we've said so far, we would have the following.
So let's go back to VS code, and let's define the airflow in Python version as arguments.
So first we define the airflow version argument.
As we said we will use airflow 2.9.0.
We also define the Python version, which we said will be Python 3.10.
Now we can reference these arguments in the from part of the docker file.
So from Apache Airflow this is the official airflow image we just saw on Docker Hub.
And here we specify the airflow version.
With also the Python.
Next we will define the airflow home environment variable, which is the directory which will contain
the main important airflow folders and files like the DAGs and logs folders and obviously the airflow
config file.
The default is as follows.
So specify the environment variable airflow home and we set it to slash opt slash airflow.
So notice how so far we have not defined anything that is custom in terms of the actual image we are
building.
We are still using the base image with airflow 2.9.2 and Python 3.10.
Now at this stage we can define the requirements.txt, which will contain all the extra packages that
we want the image to have.
That doesn't come with the base airflow image.
So let's define this text file as well.
Requirements dot txt.
In this text file we will list the libraries that we want to install.
One such library is the.
So the library that we will use for data quality tests.
In later lectures, we will get more familiar with the requirements.txt at a later stage of the course.
For now it will be empty.
Just note that the command we will use in the docker file to follow the requirements is copying requirements
dot txt.
Here we are essentially saying to copy the requirements.txt file from our local directory to the root
directory of the Docker images file system.
Finally, we use the run command which installs the specified version of airflow and the packages in
requirements.txt using pip install.
So let's actually write the run commands.
I'll write the full command and we can go over the specific parts of the run command.
So no cash.
So this is the full run command.
And the no cash option ensures that Pip does not catch the packages, which helps keep the image size
smaller.
And with that we have the Dockerfile script.
That's it for this lecture.
In the next lecture I will show you how to actually build the image.
I will see you then.