AWS Cloudformation recently launched a feature with a promise to generate templates and CDK applications from existing resources. This sounds almost too good to be true, or atleast there might be some small print to be aware of. Lets get some hands-on experience and find out what it is good for …

Previously on Cloudformation

IaC generator isn’t a stand-a-lone feature but it’s value builds on earlier features, most importantly

I did struggle to find use-cases for those features but now with IaC generator I think all clicks together.

Problem with resource import was that it was extremely difficult to create a set of template and parameters that would allow importing non-trivial resources into Cloudformation stack. Idea of adopting manually created resources did sound great but implementation just wasn’t there.

Git sync announced a couple of months ago felt like another feature looking for a use-case. While the idea of keeping the templates in Git and automatically updating stacks when commits were made is good, typical use-cases I could think of, almost always had a code deployment step that you should sync with infrastructure changes. Now with Iac generator and resource import, there might be a good case for Git sync as well.

Hands on

There is a nice blog post describing how generate the template and import resources into stack, so I’m not going to repeat that, but point out some things I wasn’t expecting.

Retain or delete

The very first step is to decide how you want to set UpdateUpdateReplacePolicy and DeletionPolicy for resources that are going to be in the template and imported into stack. Usually when you deploy a template, the default policy is Delete, but here it is Retain. If you are not familiar with these, I wrote a post about the topic last year.

While having default of Retain for deletion and updates is safe choice, it also means you need to take care of ‘carbage collection’ for resources left from stack updates.

Selecting resources

Next step is to pick resources you want include into the template. Here the user experience could be better as it can be difficult to identify from multiple resources of the same type, which one you want to include. Showing some tags or providing links to resources would help.

You can search by tag key and value but there is no way to say I’d want to filter certain Key=Value combination. Below example filters for resources with Name -tag AND any tag with value of Webserver, not only ones with Name=Webserver.

Another detail I didn’t catch from announcement was that you can only select resources that are NOT part of any existing stack. If your plan was to use IaC generator to build reusable code to deploy application stacks, this can be a disappointment.

After you have selected resources you want to include, Cloudformation will add other resources that are part of selected configuration. In my example I did pick an EC2 instance and an application load balancer, and got resources for ALB listener and target group, plus EC2 security group, ENI, EBS volume and attachment to instance. Even though my instance was part of ALB target group, it didn’t get included automatically but I had to add it manually.

NOTE: As the goal is to import resources into stack, you can pick only resources supporting import.

Every parameter is secret

I was surprised to see that in some cases generated templates had parameters too. These aren’t clearly enough to make the code reusable but it is a start. Another thing to notice is, every parameter has NoEcho: "true" -attribute making them secret. If you would make one edit before deploying, setting everything, except obvious secrets, to false would be it.

Parameters:
  RDSDBSubnetGroup00defaultvpc0be245a3ede8d2c6100yupRxSubnetIdsz8axp:
    NoEcho: "true"
    Type: "CommaDelimitedList"
    Description: "Value of write only property SubnetIds of resource RDSDBSubnetGroup00defaultvpc0be245a3ede8d2c6100yupRx."

With the default setting it is very difficult to enter e.g. a comma separated list of subnet IDs into UI where you can see only *’s. And if you change it later, you must reverse engineer the value as you can not find it from stack details :-(

Code quality

Cloudformation YAML generated is … very verbose. You get all possible LoadBalancerAttributes listed explicitly recardless if those are left to default values or not. This makes the template a lot longer than it would need to be, but also exposes all settings, which can be a good thing.

There are also many references to resource IDs, wheather those are included in the stack or not. Good example is EBS attachment with EC2 instance (both defined in the template) where InstanceId and VolumeId refer to specific physical resource IDs, instead of using logical resource names as you would do in hand written templates.

  EC2VolumeAttachment00vol06b7d183e4bb867a700jaYoF:
    UpdateReplacePolicy: "Retain"
    Type: "AWS::EC2::VolumeAttachment"
    DeletionPolicy: "Retain"
    Properties:
      InstanceId: "i-060079affca8f6dba"
      VolumeId: "vol-06b7d183e4bb867a7"
      Device: "/dev/xvda"

It would be fair to say template wouldn’t make through any peer review, but maybe that isn’t the point. Don’t give up yet, but keep reading …

Importing the resources

The Best feature of generated templates is resource import that works 100%. If you got the resource included, import will work. And if you didn’t make any edits, import of existing resources into stack is all that will happen. If you’d stop here, all this wouldn’t make any sense.

From ClickOps to GitOps

Here is the beef, I think. While you didn’t make any changes at this point, you did create

  • Cloudformation template
  • Cloudformation stack with imported resources

Next you should create a Git repository and store the template there. Then enable Git sync from the repository for the stack you created. Add some pre-commit validation and you are ready to switch from ClickOps to GitOps for infrastructure management! I’d recommend reading Automate safe AWS CloudFormation deployments from GitHub as a walk through of Git sync setup and workflow.

Summary

AWS Cloudformation IaC generator can speed up tranformation from clickops to gitops for manually deployed infrastructure. And this shouldn’t be undervalued, even if it doesn’t transform your application into re-usable code.