Building a Custom Cloudformation Resource Type
Getting Information Outside of Cloudformation
Infrastructure as code is great ideal, but in real world it is often not possible reach. Ideally all your cloud infrastructure would be sharing the single state, and allow references to any resource needed. In more realistic scenario your stack is one of many and some of the resources aren’t managed in code at all (or are created with another IaaC tool). Referencing to resources, or data, outside of the stack, hasn’t never been the strongest point of Cloudformation.
You can pass information between stacks with export/import, nested stacks, SSM parameter store, stack sets or sometimes even with copy-pasting. But what has been missing is the ability to reference external resources the same way as Terraform data sources do.
Extending Cloudformation
It has been possible to extend Cloudformation with custom resources for a long time. And with SAM templates, it became possible to combine the infrastructure and logic of custom resource into single template, assuming your code was compact enough to fit into template and didn’t depend on libraries outside of standard lambda runtimes.
While resource types are solving the same problem of extending Cloudformation, they are real 1st class citizens and comparable to any AWS provided resources. Major difference is you are responsible of running custom resources (e.g. Lambda-function), while resources types are run by AWS as part of Cloudformation service. This combined with Cloudformation Registry makes it much easier to share and consume private resource types, than custom resources, across multiple projects.
See Managing resources using AWS CloudFormation Resource Types for an overview of both models and their differences.
Nordcloud::Dataprovider::Variable
While thinking what would be the most simple data provider to test resource type development,
I came up with the idea of a pseudo resource that wouldn’t do anything but allow setting
the state at resource creation/update and then return it’s value with GetAtt
-call.
Demo & Implementation
AWSTemplateFormatVersion: 2010-09-09
Description: Nordcloud-Dataprovider-Variable
Parameters:
MyValue:
Description: MyVar Content
Type: String
Default: HelloWorld
Resources:
MyVar:
Type: Nordcloud::Dataprovider::Variable
Metadata:
Content: !Sub "This could be non-trivial combination of multiple inputs but now it is just a simple Ref to ${MyValue}"
Outputs:
Output:
Description: Content of MyVar
Value: !GetAtt MyVar.Content
Code for Nordcloud::Dataprovider::Variable
is available at GitHub.
In summary the workflow goes like this
- Install CFN CLI and dependencies
- Initialize a new project with
cfn init
- Write the resource schema and generate the handler skeletons with
cfn generate
- Implement the logic in handler functions
- Validate the resouce type with
cfn validate
- Deploy the new version of resource type and set it as default with
cfn submit --set-default
- Deploy a template using the resource type
- (cleanup old versions of resource type)
Nordcloud::Dataprovider::Variable
doesn’t have much code in handlers. All other handlers are really
just a dummy functions returning success, except the read_handler
that will return Content
value of resource Metadata. This way, I was able to avoid implementing any resource to store variable state
but could get Cloudformation take care of it for me.
What Next?
There is a long list of ideas for more serious and useful data providers e.g. finding an AMI, VPC or subnet ID based on given attributes, or mapping between HostedZoneName and HostedZoneId. Or maybe, instead of creating separate types for each use-case, have a generic data provider resource to get attributes of any AWS resource needed.
Resources
- Cloudformation Provider Development Toolkit and repos for Java/Python/Go -plugins. While cfn cli is usable at it’s current state, GitHub repos can provide valuable support. Java seems to be the most mature language for resource type development and popular for AWS resources too. It is more difficult to find good examples written in Python or Go.
- Build your first AWS Cloudformation resource provider re:Invent 2020 session describes the details how resource types works. I found this helpful in understanding the callback mechanism that will be necessary for any non-trivial (de)provisioning processes.