Go CDK! What This Means for the World of IaC

CDK has landed, and the AWS community is hyped - it was all the rage at the last AWS re: Invent, and that’s why I’d like to talk about what CDK introduces to an IaC world.  CDK is a game changer for all things infrastructure as code for AWS resources, and in this post, I’m going to dive into why that is, what it introduces, and how you can get started and even migrate existing resources.

So we’re on the same page; what exactly is AWS CDK?  CDK, AKA AWS Cloud Development Kit, is the successor of AWS’ CloudFormation that has been used to date for automating and codifying AWS cloud resources.  Because of CloudFormation’s inherent limitations, this opened the door for the popular infrastructure as code tools we have all grown to love, such as Terraform and Pulumi that, while they are not AWS native, still often delivered better IaC results than CloudFormation (particularly with diverse stacks).

CDK is set to introduce a completely new and universal way to configure cloud resources that don’t require a proprietary DSL or config language - like HCL or YAML - but rather is built to be configured in your programming language of choice, namely Golang, Python, and NodeJS. The move to CDK is a result of two main factors.  

First, AWS is looking to provide a much more native experience for their own cloud, which wasn’t very native with CloudFormation and led people to prefer non-AWS tooling to configure their cloud resources.  The second reason is that CloudFormation wasn’t built for the complexity and scale of cloud operations today and simply couldn’t keep up.  When you need to write really complex IaC with many dependencies chaining one resource to another, and build a dependency graph and tree in CloudFormation, { but really any config language }, it’s going to be very long and many times super buggy. 

Another limitation in CloudFormation is when working with multi-region operations (which is certainly a common and correct practice for high availability purposes).  In order to be able to use CloudFormation with multi-region, you’ll need to employ another AWS duct tape service called “stack set,” which makes your IaC even more complex.

What’s All the Hype?

CDK not only enables developers–– who today, many times have gained the added fun of having to configure their own environments–– to write their cloud config in their favorite programming language, but on top of this, it’s super Amazon-oriented.  It has been built to work seamlessly with AWS services and be much more AWS native than its predecessor CloudFormation and is now slated to be better than third-party tools for configuring AWS services and resources.

The first improvement this introduces over existing IaC tools is that when you have the option to configure your resources in the same programming language as your application’s code, you can use the same language libraries, and you don’t need to create a new YAML repo, for creating specific IaC code. So now, you’ll gain the benefit of having your configurations be in the same repo and code library when doing any other code implementations.  The second improvement is that you also gain all the benefits of using your programming language for your configurations in the form of code reuse, shared libraries and classes, environment variables, and much more.

How Does AWS CDK Stack Up Against Other IaCs?

We’ll start with Terraform, as Pulumi is actually built upon the underlying Terraform engine, and some of this will hold true for both.  When it comes to third-party IaC tools, CDK isn’t a hands-down 100% winner as it clearly is vs. CloudFormation, which has been in need of an upgrade for many years.  Like with all things in development, when it comes to third-party IaC, it highly depends on the use case.

So the most obvious reason CDK would trump third-party tools for AWS config is that, unlike CloudFormation, CDK has been built to be highly native to AWS.  And while both Terraform and Pulumi work very well for the AWS config use case, they are not specifically built solely for AWS, and there are times well you’ll run into edge cases where it simply is not a perfect fit for AWS.

An example of such a scenario.  I’ve had times when I’ve written security groups (and even log groups) that have been dependent on an instance, and I’ve wanted to delete the security group but not the instance, which I still need.  Terraform has failed in such a scenario when you don’t delete the instance and don’t always quite know how to deal with such scenarios.  This would also hold true for Pulumi in this case, which leverages the same Terraform engine.  This is, of course, only one example, but there are many other similar examples that exist in the DevOps world.

Just to make sure you’re really convinced, an even more frustrating scenario that I’ve likely encountered even more often than the security group is with scaling groups.  When modifying a launch configuration’s user data that is attached to a scaling group - you’ll encounter similar behavior that is equally difficult or impossible to solve.

Third-party tools, however, would be a much better fit for your environment when you are working with a diverse toolset - for example, multiple clouds, or platforms such as Datadog and Auth0, that have better native integrations with third-party IaC tools.  Also, note that all of the IaC providers have understood that the move is toward CDK, and they are building their own native CDK implementations - such as CDK for Terraform and CDK8S.   

How Does AWS CDK Actually Work?

This is probably the part you have been waiting to get to.  So how does AWS CDK actually work in practice, and how do you migrate existing resources?  

If you are already working in Golang - and have a repo written in Golang - migrating your IaC into Go is pretty easy, and the outcome will be that it will be fully attached to your existing project. This, of course, always depends on the number of resources being migrated and their complexity, but overall the rewrite itself isn’t too hard; it might just be time-consuming.

Since you will need to rewrite all of the resources, as there currently isn’t a robust enough tool out there to enable backward compatibility or the reverse engineering of YAML to Golang, for example––however, since you do have the benefits of code reuse, variables, and classes, this often times not a very heavy lift.

Also, it is possible to migrate in a hybrid approach until you manage to convert all of your resources.

Another benefit is that you can see the diffs between the config files, so you can see how much progress you have made and what remains to be converted and get a pretty accurate schedule for the migration.

Show Me the Code

Below we will dive into a real example of migrating your CloudFormation to CDK in Golang.

This is what the code will look for migrating a CloudFormation YAML file:

type CdkBasicStackProps struct {
	awscdk.StackProps
}

func CdkBasicStack(scope constructs.Construct, id string, props *CdkBasicStackProps) awscdk.Stack {

	/********************************
	** set up CDK stack properties **
	*********************************/

	var stackProps awscdk.StackProps
	if props != nil {
		stackProps = props.StackProps
	}
	stack := awscdk.NewStack(scope, &id, &stackProps)

	/********************************************************
	** create new resources straight through the CDK stack **
	*********************************************************/

	queue := awssqs.NewQueue(stack, jsii.String("ExampleQueue"), &awssqs.QueueProps{
		VisibilityTimeout: awscdk.Duration_Seconds(jsii.Number(300)),
	})

	bucket := awss3.NewBucket(stack, jsii.String("ExampleBucket"), &awss3.BucketProps{
		Versioned: jsii.Bool(true),
	})

	/****************************************************
	** migrate Cloudformation stack into the CDK stack **
	****************************************************/

	cfn := cloudformationinclude.NewCfnInclude(stack, jsii.String("ExampleMigratedStack"), &cloudformationinclude.CfnIncludeProps{
		TemplateFile: jsii.String("cfnTemplates/cfnInstance.yaml"),
	})

	fmt.Println(queue.QueueName(), bucket.BucketArn(), cfn.Stack())
	return stack
}

func main() {
	app := awscdk.NewApp(nil)

	CdkBasicStack(app, "CdkBasicStack", &CdkBasicStackProps{
		awscdk.StackProps{
			Env: &awscdk.Environment{
				Account: jsii.String(os.Getenv("CDK_DEFAULT_ACCOUNT")),
				Region:  jsii.String(os.Getenv("CDK_DEFAULT_REGION")),
			},
		},
	})
	app.Synth(nil)
}

And this would be the end result of your migrated file to Golang (as you can see, it’s significantly shorter and easier to read):

AWSTemplateFormatVersion: "2010-09-09"
Description: A sample template
Resources:
  CFNMigrationExampleInstance:
    Type: "AWS::EC2::Instance"
    Properties:
      ImageId: "ami-0bf84c42e04519c85"
      InstanceType: t2.micro
      KeyName: sample-key-pair
      BlockDeviceMappings:
        -
          DeviceName: /dev/sdm
          Ebs:
            VolumeType: io1
            Iops: 200
            DeleteOnTermination: false
            VolumeSize: 20

Once the CDK template is deployed, you will be able to find the CDK artifacts in the AWS console (where you will be able to find the resources that the sample code above created in AWS ).

The Case for AWS CDK

So I hope this was a useful overview of what CDK is and how and when to use it in your stacks and environments with a practical example of a CloudFormation migration.  We expect there to be much more traction around CDK and many more integrations and examples to be rolled out in the coming weeks and months. 

We may also start to see useful tooling to help convert existing files at scale––albeit there are already some, they aren’t quite ready for showtime. So this is definitely a great resource and an AWS area to watch, and start planning your move to CDK when the time is right for you and the fit right for your stack.