How To Migrate Terraform State to GitLab CI/CD

As a software professional handling Infrastructure as Code (IaC), chances are you work a lot with Terraform. When helping new clients use IaC, it is common to simplify things, but managing a Terraform state file is the first challenge you face. Essentially, Terraform state contains sensitive information, which shouldn’t be stored by source control but, at the same time, won’t scale if you have multiple users working on the same Terraform state. The answer to that? Backends.

It is important to note that you could store that state file on an S3 bucket and use DynamoDB to manage the locking state. However, this approach will force you to create additional resources, which makes it a complicated option, especially if the client is using GitLab. GitLab recently lowered the entry barrier to integrating Terraform by providing a way to store and manage Terraform state, as well as an easy way to set up a CI around it.

In this article, we will explain what a Terraform state file is, how to migrate it to GitLab, and setting up a CI pipeline for it. You can visit our repository here.

Table of Contents

• What Is Terraform State?
• How To Get GitLab to Manage Terraform State
• How To Get GitLab to Run Your IaC Through a CI Pipeline
  • Bonus Tip: Infracost
• Conclusion

What Is Terraform State?

Terraform records any information about the infrastructure defined in your code via a state file. Written in JSON, it essentially records a mapping from the Terraform code to the real resources created. Below is an example of what a terraform.tfstate would look like. Primarily, every time you run Terraform it will fetch the latest status for its EC2 instance and compare it with your Terraform configuration to determine what changes need to be applied:

{   
     "version": 4,    
     "terraform_version": "0.12.0",    
     "serial": 1,    
     "lineage": "1f2087f9-4b3c-1b66-65db-8b78faafc6fb",    
     "outputs": {},    
     "resources": [     
      {        
         "mode": "managed",        
         "type": "aws_instance",       
         "name": "example",       
         "provider": "provider.aws",       
         "instances": [         
           {           
             "schema_version": 1,           
             "attributes": {              
               "ami": "ami-0c55b159cbfafe1f0",             
               "availability_zone": "us-west-2a",             
               "id": "i-00a123a0accffffff",            
               "instance_state": "running",             
               "instance_type": "t2.micro",            
               "(...)": "(truncated)"           
            }         
          }       
        ]      
      }   
    ]  
 }

By default, this terraform.tfstate is stored locally where you have your Terraform files, plan, and apply your changes. For a personal project where you are just running some tests; it’s fine, but not the recommended way, here’s why:

• Stored in a shared location: if you were hosting this state file on your local workstation and had to work with another engineer, things would get complicated. Both of you will have to make sure you are using the latest version of the state and you could run into race conditions if you run a Terraform plan or apply at the same time.
• Protect sensitive information: A generated state file can contain encryption keys and infrastructure passwords. However, state files aren’t encrypted by default, and storing sensitive information in plain text is a bad idea.
• Locking: Most version control systems do not support any form of locking, which prevents two team members from running Terraform apply simultaneously on the same state file. This is another reason why we will not see a state file managed by source control.

How To Get GitLab to Manage Terraform State

With Terraform being considered the standard in cloud infrastructure provisioning, it has been a year or so since GitLab began to offer a way to store and manage your Terraform state. For this reason, we wanted to share the migration process with you as we recently started using GitLab to manage our IaC.

For this article, we presume that you are using a local state, and have your state managed with an AWS S3 Bucket or another backend solution. 

Firstly, you will need to change your backend.tf to use HTTP: 

terraform {  
    backend "http" {} 
  }

Next, you will need to set up four variables in your terminal:

1. PROJECT_ID: You can find this easily by navigating to your repo on the “Project Overview” page.

2. TF_USERNAME: The Gitlab username that has access to the repo you’re working on.

3. TF_PASSWORD: Access token generated from your GitLab user.

4. TF_ADDRESS: URL of the remote state backend.

PROJECT_ID="28450092"
TF_USERNAME="florianpialoux"
TF_PASSWORD="123456789"
TF_ADDRESS="https://gitlab.com/api/v4/projects/${PROJECT_ID}/terraform/state/aws-buckets"

You may now run the migration command that will move your Terraform state from its previous location to GitLab with the following command:

  terraform init \  
    -migrate-state \  
    -backend-config=address=${TF_ADDRESS} \  
    -backend-config=lock_address=${TF_ADDRESS}/lock \ 
    -backend-config=unlock_address=${TF_ADDRESS}/lock \  
    -backend-config=username=${TF_USERNAME} \ 
    -backend-config=password=${TF_PASSWORD} \  
    -backend-config=lock_method=POST \ 
    -backend-config=unlock_method=DELETE \ 
    -backend-config=retry_wait_min=5

You will need to provide a confirmation by a “yes” so that GitLab can start managing your state file. Here’s an example from a local state to GitLab:

Example S3 to GitLab:

I noticed for some of the state files I had from S3 will be blank even after using the migrate-state command ran previously. In this case, you can run this:  

terraform state pull > aws-buckets.json

Copy and paste the content from the S3 state and run a push:

terraform state push -lock=true aws-buckets.json

GitLab supports versioning for your Terraform state file but viewing/restoring older versions through the WebUI will require you to be using a GitLab Premium plan. If not, you will need to make a GraphQL API request.

How To Get GitLab to Run your IaC Through a CI Pipeline

GitLab provides a Docker image that contains GitLab-Terraform, which is a thin wrapper script around the official Terraform binary. Alternatively, you could use the official Docker image by Hashicorp. You can find more information about the GitLab Terraform Image here.

Once the Terraform apply job runs, you will be able to see when the state was used and with which pipeline.

Bonus Tip: Infracost

As you might have noticed, looking at our gitlab-ci.yaml we added Infracost, which allows us to have more control over our cloud billing as it gives you a cost estimate whenever you define a new resource to your IaC.

Conclusion

Having your Terraform state and CI running on Gitlab is a great way to follow GitOps best practices. They both make a great combination to develop and deploy IaC. Since most of you might be already using GitLab for your repositories, it becomes much simpler to have your IaC under one roof and let GitLab manage your Terraform state by supporting encryption in transit and at rest, as well as versioning, locking, and unlocking the state.