AWS CloudFormation – An Architect’s Best Friend

Automating the AWS deployments has been a key driver to ensure consistent and reliable deployments. Whether it is zero-touch deployments, immutable architecture or continuous delivery (CD), automated deployments are critical to successful delivery. And, this is not just about deploying the underlying infrastructure. It is also equally applicable to the application stacks as well. AWS CloudFormation is a key service when it comes to automating AWS deployments. Be it a simple stack with a couple of resources or complex stacks that are deployed to multiple AWS regions and accounts, CloudFormation provides several useful capabilities like reusable deployment templates, powerful CLI, automatic change detection, rollback, resource dependency management, parallel deployment, and many more. In this post, we will learn some basics along with an actual example and see why is CloudFormation an Architect’s best friend!

What is CloudFormation (CFN)?

First things first. CloudFormation (CFN) is a service from the AWS portfolio that delivers Infrastructure as a Code. It offers the following key capabilities.

• Automate complete infrastructure setup: Manage the complete deployment recipe of your deployment by specifying resources and their configuration. A CFN deployment is also referred to as a stack.
• Focus on “what” and not “how”: CFN is developed in the form of text-based templates (a.k.a. CFN template) in which the majority of times you specify “what to do”. For example, when you create an EC2 instance, you simply specify it’s configuration (like the instance type, storage size, etc) and not the actual commands to create the resource. This makes the templates relatively small in size and helps you focus on what you want to accomplish. Having said that it does offer (limited) scripting capabilities for common needs, such as string manipulation.
• Consistency with speed: CFN offers features like parallel deployment for faster turnaround. At the same time, the templates can be designed to ensure all deployments are consistent and avoid manual configurations altogether.
• Manage the complete lifecycle of the stack: You can create/update/delete stacks.
• Version Control Infrastructure Releases: Yes, that’s possible! Since CFN templates are text-based documents (JSON or YAML), you can simply check these into your version control and use the same standard best practices that you are used to for a typical application release. In fact, I highly recommend to only deploy from the “master” branch to production to ensure only validated changes are deployed.
• Apart from these, it offers several other useful capabilities like dependency management between resources (dependency resources are deployed before and deleted after the dependent resources), delta detection when making updates, an automated rollback of failed deployment, and so on.
• CFN offers a powerful CLI and SDK support for programmatic integrations. This is extremely useful for DevOps and automation purpose.

CloudFormation Core Concepts

Let’s talk about some basic CFN constructs now. Keep in mind that the key idea with CFN is to have reusable templates that can be used to deploy multiple stacks typically.

• Parameters: These are used to take user inputs and capture variables that can change between deployments from the same template. A parameter has a type (such as String) and can optionally have validation associated. AWS offers pre-defined parameters, known as Pseudo Parameters, for some useful values, such as the deployment account ID, the target region, and so on.
• Mappings: These are used to capture derived information. For example, you can specify resource attributes for different deployment type (development vs production).
• Conditions: A condition can be used to specify the conditional creation of a resource or use of a resource property.
• Resources: The resources and their configuration specified using properties.
• Outputs: A CFN template can produce outputs to provide relevant information. For example, an RDS template can provide the database instance connection information.
• Metadata: Useful metadata can be specified in the CFN template that can be consumed by other tools, such as CloudFormation Designer and automation tools. For example, CFN offers helper scripts that can use the metadata to install the software.
• Intrinsic Functions: CFN provides several useful intrinsic functions for common computing needs, such as string manipulation, resource lookup, etc. These are evaluated at deployment time.

CFN Template Example

Let’s walk through an example to understand these concepts better.

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "An EC2 instance.",
  "Parameters": {
    "InstanceName": {
      "Description": "The instance name.",
      "Type": "String"
    },
    "DeploymentType": {
      "Description": "The deployment type.",
      "Type": "String",
      "AllowedValues": ["Dev", "QA", "Prod"],
      "Default": "Dev"
    },
    "Subnet": {
      "Description": "The subnet for the EC2 instance.",
      "Type": "AWS::EC2::Subnet::Id"
    },
    "SecurityGroups": {
      "Description": "The Security Groups for the EC2 instance.",
      "Type": "List"
    },
    "KeyPair": {
      "Description": "The key pair name to use to connect to the EC2 instance.",
      "Type": "String"
    }
  },
  "Mappings": {
    "Globals": {
      "Constants": {
        "ImageId": "ami-0b898040803850657",
        "AssignPublicIP": "true"
      }
    },
    "DeploymentTypes": {
      "Dev": {
        "InstanceType": "t2.small",
        "StorageSize": "20"
      },
      "QA": {
        "InstanceType": "t2.small",
        "StorageSize": "30"
      },
      "Prod": {
        "InstanceType": "t2.medium",
        "StorageSize": "50"
      }
    }
  },
  "Resources": {
    "EC2Instance": {
      "Type": "AWS::EC2::Instance",
      "Properties": {
        "ImageId": {"Fn::FindInMap": ["Globals", "Constants", "ImageId"]},
        "InstanceType": {"Fn::FindInMap": ["DeploymentTypes", {"Ref": "DeploymentType"}, "InstanceType"]},
        "NetworkInterfaces": [{
          "DeviceIndex": "0",
          "SubnetId": {"Ref": "Subnet"},
          "AssociatePublicIpAddress": {"Fn::FindInMap": ["Globals", "Constants", "AssignPublicIP"]},
          "GroupSet": {"Ref": "SecurityGroups"}
        }],
        "BlockDeviceMappings": [{
          "DeviceName": "/dev/sdm",
          "Ebs": {
            "VolumeType": "gp2",
            "VolumeSize": {"Fn::FindInMap": ["DeploymentTypes", {"Ref": "DeploymentType"}, "StorageSize"]},
            "DeleteOnTermination": "true"
          }
        }],
        "KeyName": {"Ref": "KeyPair"},
        "Tags": [{"Key": "Name", "Value": {"Ref": "InstanceName"}}]
      },
      "Metadata": {
        "AWS::CloudFormation::Designer": {
          "id": "d0aacb0c-1b2c-452c-baf0-b283b0ba4a1a"
        }
      }
    }
  },
  "Outputs": {
    "PublicDNSName": {
      "Description": "The public DNS name.",
      "Value": {"Fn::GetAtt": ["EC2Instance", "PublicDnsName"]}
    },
    "PublicIP": {
      "Description": "The instance public IP address.",
      "Value": {"Fn::GetAtt": ["EC2Instance", "PublicIp"]}
    }
  },
  "Metadata": {
    "AWS::CloudFormation::Designer": {
      "d0aacb0c-1b2c-452c-baf0-b283b0ba4a1a": {
        "size": {
          "width": 60,
          "height": 60
        },
        "position": {
          "x": 546,
          "y": 153
        },
        "z": 0
      }
    }
  }
}

This template deploys an EC2 instance. Following are the details.

• The template format version and description are specified in the beginning.
• The Parameters specifies deployment inputs that can change across stacks deployed from this template.
  • The InstanceName parameter captures the EC2 instance name.
  • The DeploymentType parameter is interesting. I call this a logical parameter. It simplifies the user experience by avoiding to ask unnecessary details from the user. At the same time, it allows customizing different EC2 instance deployments from the template by choosing from {Dev, QA or Prod} values.
  • The Subnet parameter is used to specify the EC2 instance subnet.
  • The SecurityGroups parameter specifies the security groups to be associated with the instance.
  • The KeyPair parameter takes the SSH keypair name that will be used to connect to the EC2 instance.
• The Mappings section specifies two top-level maps – 1) Globals and DeploymentTypes maps. And, each of these maps contains one or more maps that store data.
  • The Globals map is used here to capture useful constants – the AMI ID to use and whether to assign public IP to the EC2 instance. Why would you have such a map? It’s for ease of maintenance. Tomorrow if you have to make a change, simply update the constants.
  • The DeploymentTypes map specifies EC2 instance properties based on the deployment type. For example, for the Dev deployment, we would like to use InstanceType as t2.small.
• It creates a single resource – an EC2 instance. This also shows why CFN is more about “what” than “how”. See we are only specifying resource configuration and not the commands to create the instance. Let CFN do it’s magic!
  • The ImageId property is set by looking up the Globals->Constants->ImageId value and using the Fn::FindInMap intrinsic function.
  • The InstanceType property is set by looking up the map for the DeploymentType. This uses the Ref intrinsic function to get the DeploymentType parameter value and then passes it to the Fn::FindInMap intrinsic function to retrieve the property value.
  • The NetworkInterfaces property specifies a single network interface with SubnetId set to the Subnet parameter value, AssociatePublicIpAddress set to the Globals->Constants->AssignPublicIP value, and GroupSet set to the SecurityGroups parameter value.
  • The BlockDeviceMappings property specifies a single EBS volume of type gp2 and size by looking up the DeploymentTypes-><DeploymentType>->StorageSize value. For example, for a development stack, the DeploymentTypes->Dev->StorageSize value will be used.
  • The KeyName property is set using the KeyPair parameter value.
  • The Name tag is set to the InstanceName parameter value.
• The Outputs section outputs the public DNS name and IP information using the Fn::GetAtt intrinsic function to retrieve the PublicDnsName and PublicIp attributes of the EC2 instance, respectively.
• The Metadata sections of this template contain the user interface data for the CFN Designer tool.

Conclusion

We covered a lot of ground here. But, this is just a highlight of how powerful CFN is. It offers many more capabilities like resource dependency management, nested stacks that make it easy to dploy a hierarchy of stacks, StackSets that allow cross-region and even cross-account deployment, and more. In fact, if you have a resource that is not directly managed by AWS (or not supported by CFN), you can still manage it using CFN via a Custom Resource. So, you see the possibilities are endless and we have only scratched the surface. But, you know by now that if you are an Architect or someone who is involved in AWS deployments, CloudFormation is a tool that can help you tremendously in automating your deployments.

Happy deploying!
– Nitin

 

 

---

Also published on Medium.