AWS Cloudformation got recently a new feature that brought loops for templates.

I wouldn’t have thought in 2023 adding a loop to programming language would have been exciting announcement but it was ;-)

Loops have obvious benefits in simplifying the code, especially the maintenance of it, as you don’t have to repeate yourself so much. So let’s take loops for a spin (pun intented) …

Declarative loop ?

Declarative programming is described as programming paradigm that expresses the logic of a computation without describing its control flow. Instead of describing how the program works you define the what it should accomplish. This is also how Cloudformation works. You define resources but e.g. the order in which those are created is left for the service to decide.

Loops being a prime example of control flow it might look like this would break the declarative paradigm. But it doesn’t. For -loops are implemented as Cloudformation Transform, i.e. a macro that is hosted by Cloudformation, and simply pre-process the template before actual execution. This means resources inside the loop are not (necessarily) created in the order you defined them, but Cloudformation will decide build order based on implicit or explicit dependency as for any other resources.

This is similar to using Count -macro or tools like Jinja and CDK to create Cloudformation templates. So while template creation might be imperative (have control flow), template execution remains declarative.

Thing to keep in mind is Cloudformation template limits apply to execution phase. You can not have a loop (or count -macro) to create 1000 resources because that would still exceed the maximum template length no matter if the original unprocessed template would be only a few 10s of lines.

Converting a template

First thing that come to my mind when thinking what template I should try the loops was VPC networks. Those have basicly the same code repeated for availability zone with some simple logic to enable and disable resources based on how many zones VPC should cover. For simplicity I simplified the template to create only 2 identical VPCs with no subnets or other necessary resources.

---
AWSTemplateFormatVersion: '2010-09-09'
Description: 2 VPCs using ForEach
Transform: 'AWS::LanguageExtensions'

Resources:

  'Fn::ForEach::Network':
  - X
  - [ A, B ]
  - VPC${X}:
      Type: AWS::EC2::VPC
      Properties:
        CidrBlock: 10.0.0.0/16
        Tags:
          - Key: Name
            Value: !Sub '${AWS::StackName} VPC'

Structure of ForEach -loop is

  • Transform: 'AWS::LanguageExtensions' in the beginning of template telling Cloudformation to run template through preprocessing before executing it.

  • 'Fn::ForEach::Network': defining the loop with unique name as any other resource.

  • First parameter X is the loop identifier. This is the variable that gets a new value assigned on each iteration.

  • Second parameter [A, B] is the list of values the loop will iterate over.

  • Third parameter is the resource definition. Here you must use loop identifier in resource name to make it unique for each iteration. As resource names must be unique, you can not use the same value twice in the values the loop iterates over.

Above works fine and creates resources VPCA and VPCB as expected. There are also more usuful examples to test and copy from. So far, so good.

Speed bump ahead

Next I wantedd to expand above snippet and have an unique Name -tag for each VPC. One would think this would be as simple as

        Tags:
          - Key: Name
            Value: !Sub '${AWS::StackName} VPC ${X}'

Unfortunately it wasn’t :-( Trying to deploy above resulted an error message Template format error: Unresolved resource dependencies [X] in the Resources block of the template. This was clearly because of the reference to loop variable from tag value.

After numerous debugging iterations, I got this isolated down to a case where there is a combination of AWS pseudo parameter and loop identifier inside of the same substitution. I could reference to regular template parameter without any problems. It feels like there is some conflict during template transformation when both pseudo parameters and foreach loop are present.

EDIT: Actually it seems not to be just any pseudo parameter but AWS::StackName that gets this issue triggered.

Fast-forward more trial-and-error and I found Join would allow me to use loop and pseudo variables together. To create Name -tag I had to use the old-school concatenation of strings instead of Sub. Below works but feels like a big step backwards compared to using Sub.

        Tags:
          - Key: Name
            Value: !Join
              - ' '
              - - !Ref AWS::StackName
                - 'VPC'
                - !Ref X

Other things to consider

Fn::ForEach works only within Conditions, Resources and Outputs sections of template. You can not use it to create repeating Parameters or simplify your Mappings. Especially not being able to create one loop for parameters and another for resources matching parameters feels like a missing feature. E.g. in VPC template you still must write parameters for your subnet CIDRs while you can create subnets using ForEach.

You can create multiple resources (of different types) inside one loop. You do this simply adding more resources into the last attribute of ForEach and take care of indenting YAML correctly (or matching JSON angle brackets). There is a good example of creating NAT gateways with matching EIPs in single loop.

You can also have multiple properties for resources created from a loop. For example if you would want to have bunch of EC2 instances but with different instance types and AMIs, it can be done using a loop variable as reference to a mapping table. If most of your resources will have the same properties, you can also use FindInMap default value and only specify mappings for the resources that differ from defaults. FindInMap Default value is part the same LanguageExtensions transform as ForEach so you already have it enabled ;-)

Logical ID’s of resources can only contain letters and numbers (A-Za-z0-9) so below would not work.

  'Fn::ForEach::Subnet':
  - X
  - [ A, B ]
  - PrivateSubnet-${X}:

Also AZ names are case sensitive and must be in small-caps so below would not work either.

  'Fn::ForEach::Subnet':
  - X
  - [ A, B ]
  - PrivateSubnet${X}:
      Type: AWS::EC2::Subnet
      Properties:
        AvailabilityZone : !Sub "${AWS::Region}${X}"

To be continued

My original plan was to refactor some VPC templates using ForEach and to see how much it would help in simplifying the code. Due to time spend on debugging template transformations this was left for another post but I hope this would be helpful for others running into one of these speed bumps.

NOTE: You can find the continuation in Lessons in Cloudformation Fn::ForEach