Maven Gives Terraform a Big Boost

Terraform is my company's corporate standard for Infrastructure as Code (IaC) development. And there are a lot of things to love about Terraform. It's syntax is relatively simple, it supports modules as a mechanism for code reuse and it's easily extendable. But there are also some not so great things about Terraform. Integrated testing options for Terraform are limited, module sources and their versions must be hardcoded (aka no dependency management) and private module registries are sparse. There are some other issues too, but let's not nitpick.

Prior Attempts at Terraform Testing and Dependency Management

The issues listed above are not new for Terraform developers. So, it's no small wonder that various attempts have been made to resolve those issues. For example, Terrafile is used by some Terraform developers to remove the module source locations and versions from the Terraform source code. The trade-offs are that transitive dependencies are handled manually and it is a unique one-off solution. Still, it did take a huge step towards separating Terraform dependencies from Terraform source code.

In terms of Terraform testing options, the best-integrated options were Terratest and Kitchen-Terraform. Kitchen-Terraform was a lot of overhead for setting up in an integrated development environment. Don't get me wrong, it can work, but it's not the most seamless thing to manage for developers wanting to test their infrastructure code. TerraTest was pretty easy to setup. But it was a little cumbersome to use and I found it to be a little buggy.

Maven Brings It All Together for Terraform

What I really wanted was the same experience that Java developers in my organization had with Maven. Maven handles Java dependency management at the project level, including transitive dependencies. There are also a ton of great Java testing frameworks out there that integrate seamlessly with Maven: JUnit, Spock, etc. And all of that is wrapped up nicely into a managed build lifecycle under an opinionated project structure.

Since Maven was already widely-used within my company, a Maven repo was already available. Piggybacking off of that using Terraform essentially gave me a Terraform module repository, but with all the features and maturity of Maven. The trick here was going to be piggybacking off of Maven by creating a near seamless experience for Terraform configurations. After all, who wants to use a tool that makes development more difficult?

The Terraform Maven Plugin

Maven is extendable through plugins. So, a Maven plugin would be the key to bringing integrated testing and Maven's dependency and build lifecycle management to Terraform. In addition to a Maven plugin, a Terraform API for Java was needed to support Terraform commands through Maven and to allow integrated tests to provision and destroy infrastructure under test.

The net result was that Terraform functioned just like any other JVM language in Maven. This image shows a Terraform Maven project that has two Terraform configurations (s3 and s3_replicated_src) along with unit tests. By default, when a Terraform configuration is initialized, its external dependencies are brought into the .tfmodules directory from a Maven repository.

Terraform Configuration Initialization Example:

 mvn tf:init -DtfRootModule=s3 

Consuming an external dependency in Terraform code then looks like the following:

module "s3" {
  source = "../../.tfmodules/s3"
  bucket_name = "${var.bucket_name}"
  environment = "${var.environment}"
  region = "${var.region}"
}

Note: The above example is HCL, not JSON. But there was no HCL or Terraform language option.

You might be thinking, "That looks a lot like what Terrafile does." And you would be correct. The difference, of course, is that Maven can also resolve transitive dependencies automatically and it can use Maven version qualifiers along with the rest of what Maven offers, too.

So, what else does Maven offer? Well, as alluded to above, Maven provides support for integrated tests. So, you can have Terraform tests that look like this.

class S3Spec extends Specification {

    def "S3 module provisions a bucket in AWS"() {
        given:
            Properties tfProperties = new Properties()

            String region = 'us-east-1'
            String environment = 'dev'
            String stackName = 's3'

            TerraformInit init = new TerraformInit(stackName)
            TerraformApply apply = new TerraformApply(stackName)

            AmazonS3 s3 = AmazonS3ClientBuilder.standard().withRegion("us-east-1").build()

            tfProperties.put('tfRootDir', stackName)
            tfProperties.put('tfVars', "region=${region},environment=${environment}".toString())

        when:
            init.execute(tfProperties)
            apply.execute(tfProperties)
            def jsonOutput = getTerraformOutput(stackName)
            String bucketName = jsonOutput.bucket_arn.value[(jsonOutput.bucket_arn.value.lastIndexOf(":") + 1)..-1]

        then:
            s3.doesBucketExistV2 bucketName
            s3.getBucketTaggingConfiguration(bucketName).getTagSet().getTag('environment') == environment

        cleanup:
            TerraformDestroy destroy = new TerraformDestroy(stackName)
            destroy.execute(tfProperties)
    }
}

Now, let's say you wanted to run those tests and deploy the Terraform configurations in your project as a versioned Maven artifact in your local Maven repository. You just run the following.

 mvn clean install 

If you wanted to deploy it to a remote Maven repo then you could just run something like this.

mvn clean deploy -Durl="<remote repo url>"

Summary

It's not hard to see that Maven dependency management, Maven repository support, integrated tests and build lifecycle management for Terraform adds a ton of value to the Terraform development experience. If you want to learn more about the Terraform Maven plugin, including how to start using it in your own projects, check out https://github.com/deliveredtechnologies/terraform-maven.

Finally, I think it's also worth stating that although I did contribute extensively to the development effort, I wasn't the only contributor to the Terraform Maven plugin. Sai Chaitanya and Robert Hutto were also contributors. The functionality of the Terraform Maven plugin as it exists today would not be where it is without them.