In this lecture, we will get introduced to a remote state in Terraform.
Previously, we saw how Terraform uses the state file
to map resources in the configuration file to real world infrastructure.
So far, we have been working with the Terraform state file
that is stored locally.
When we run Terraform apply for the first time,
a file called terraform.tfstate is created
inside the configuration directory by default.
Some of the benefits of the Terraform state are,
mapping Terraform configuration to the real world infrastructure,
tracking metadata such as dependencies,
which allow Terraform to create and delete resources in the correct order,
improving the performance of Terraform operations
while working with large configuration files
and especially those that make use of a number of different cloud providers.
Finally, the Terraform state allows members of a team to collaborate
and provision resources as a team.
However, in the examples that we have seen so far,
Terraform state file is created locally on the client machine
and more specifically within the configuration directory
that is being used as a default.
This is an example of a local state file,
and it does not provide a good opportunity to collaborate.
This is because the file is only available on the client machine,
such as a developer's laptop.
We have also learned that unlike the configuration files,
it's not a good idea to store the state file in a version control system.
This is because it would mostly contain sensitive information
pertaining to our infrastructure.
Let us see why it's a bad idea to store the state file in a version control system.
For this, let's see the Terraform workflow when used individually and as a team
with the state and configuration saved within a Git Repo.
Let us consider that Abdul, who is a developer,
writes the Terraform configuration for creating an S3 bucket.
He then proceeds to run the Terraform plan.
After reviewing the execution plan, proceeds to apply the configuration.
At this stage, Terraform creates a local state file called terraform.tfstate.
Once the bucket has been created,
he checks in all the files in the configuration directory into a Git Repo,
including the Terraform state file
and if any changes are to be made, the same process is repeated.
The files are fetched from the Git Repo,
changes are then made to the configuration file.
This is followed by a Terraform plan and then apply.
Finally, the updated configuration file and the state are pushed to the Git Repo.
Now, let us assume that the team is growing
and a new developer called Lee also wants to manage the infrastructure using Terraform.
The logical way would be for Lee to pull down the Git Repo into his own client machine.
He would then follow the same process as Abdul to make changes to the infrastructure,
make changes to the configuration as needed,
review the execution plan and then apply the configuration.
Once applied, he would make sure to push the updated configuration
and state file into the remote Git Repo.
While this can technically work, there are some major flaws when using this approach.
As we saw earlier, state file stores all the information about the infrastructure,
including IP addresses, initial passwords for some databases, key names, et cetera.
Keeping all this information in a Git repository is not recommended.
Secondly, when we work as a team,
it is important that only one person runs operation
against the same configuration and state at any given time.
If Abdul and Lee both try to update the same S3 bucket at the same time
using the exact same state file but within their individual laptops,
it can lead to unintended consequences such as the corruption of the state file.
When using local state files as an individual,
Terraform protects itself from getting into the situation
where concurrent operations are run against the same configuration,
and this can be tested by running Terraform operations one after the other,
such as running data from apply command simultaneously from two terminals.
While the first operation is in progress, Terraform locks the state file.
As a consequence, we will not be able to run any operation from the second terminal
until the first operation finishes.
This is a very important feature of Terraform called state locking.
It ensures that the state file does not get corrupted
by multiple operations trying to update it at the same time.
Version control systems such as GitHub do not support state locking.
As a result, if it is used to store the state file,
multiple users can use the same state file simultaneously,
which can result in issues like conflicts and data loss within the state file.
Finally, if a user forgets to pull the latest version of the state file
from the version control system and work with an older obsolete state file,
it can result in disastrous ethics such as rollback
or even destroying of the resources.
Because of all these reasons,
it's a much better option to store Terraform state in a secure, shared storage
by making use of remote backends.
With this option, the state file no longer resides in the configuration directory
or version control systems
and are moved to a shared storage solution such as AWS S3,
HashiCorp Consul, Google cloud storage and Terraform cloud.
When a remote backend is configured,
Terraform will automatically load the state file from the shared storage
every time it is required by a Terraform operation.
It will also upload the updated Terraform state file after every apply.
Most importantly, many of these remote backend options
such as the AWS S3 allows for state locking.
This ensures that the integrity of the state file is maintained.
Finally, the remote backends provide different ways to secure the storage
such as encryption at rest and in transit
to make sure that all the sensitive information stored inside is secured.
In the next lecture, we will see how to configure remote backends
using AWS S3.