AWS From The Cloud Down

A Functional Overview of CloudFormation

Created by Laurence J MacGuire a.k.a. 刘建明 a.k.a Liu Jian Ming

ThoughtWorks Xi’An, 2017/06/01

CloudFormation
- Concepts
  - CRUD for infrastructure
  - DAG for infrastructure
  - Lifecycle management
  - Versionable (CI/CD on infrastructure?!)
- Practical
  - Parameters
    - Like a function / Rendering a template
    - Referable in the other sections
  - Resources
    - Most AWS things available
    - Each have different lifecycles / update procedures
      - Update
      - Update w/ interruptions
      - Replacement (beware!)
  - Outputs
    - Result of your function (that has massive side effects)
  - Mappings (optional)
    - Saves you some typing / querying (beware!)
  - Conditions (optional)
    - Saves you some logic in wrapping scripts
What’s a good template that:
- shows different lifecycle types
  - update
  - inline replacement
  - full replacement
- doesn’t show too many types of resources
- can easily go into all states
  - Can we get URF?

Running an application

Network(s)
- Routers
- Firewalls
Servers
- App servers
- Load Balancers
- Databases
Code & Configuration

Back in the DC

Phone conversation
Pallet of servers
Installation
Configuration
Going live
Complicated

In the cloud

Add a credit card
Click through a UI
Or write scripts (better)
Provision everything
Create a lot of stuff because it’s easy
Becomes a massive mess.

Debatably better

Need for encapsulation

Cloudformation wraps all of your infrastructure elements into one logical unit.

Stack

Stacks

asdsd

Anatomy of a CFN Template

---
AWSTemplateFormatVersion: '2010-09-09'
Description: Some Application Stack Template

Parameters:
  myParameterName: {}

Resources:
  myLogicalResourceName: {}

Outputs:
  myOutputName: {}

Conditions:
  myConditionName: {}

Mappings:
  myMapping: {}

Parameters

---
Parameters:
  myParameterName:
    Type: (String,ListOfStrings,Number,Special types)
    Default: "XYZ"
    Description: "What this parameter means"
    AllowedValues: ["option-one", "option-two"]
    AllowedPattern: "/^[a-z]+$/"
    ConstraintDescription: "Explain the acceptable restrictions"

  etc ...

Special Parameters

---
Parameters:
  ELBSubnets:
    Type: List<AWS::EC2::Subnet::Id>
    Description: "Which subnets to deploy the ELB in"

Subnet Ids, VPC Ids, SecurityGroupIds, etc. CFN Pre-validates the values, so your stack fails faster. Read more: CFN doco

---
Parameters:
  BucketName:
    Type: "String"
  SNSTopicName:
    Type: String
  InstanceType:
   Type: String
   Default: "t2.nano"
   AcceptableValues: ["t2.nano", "t2.micro", "t2.small" ...]
  MaxInstances:
    Type: Number
    MinValue: 2
    MaxValue: 16

Resources

---
Resources:
  MyAWSResource:
    Type: AWS::Service::Entity
    Properties:
      FidgetBlobbyThing: "Blah"
    DependsOn: []
    CreationPolicy: {}
    UpdatePolicy: {}
    DeletionPolicy: {}
    Metadata: {}

Resources

Resources:
  InstanceGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      MinSize: "2"
      MaxSize: "16"
      InstanceType: "t2.nano"

  LaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Properties:
      UserData:
        "Fn::Base64":
          "Fn::Join": [
            "\n", [
              "#!/usr/bin/bash"
              "docker run -e SNS_TOPIC=my-topic -e S3_BUCKET=my-s3-bucket my-image rails s"
            ]
          ]

Logical & Physical Ids

InstanceGroup => InstanceGroup-o98123kjhsdf

Templates can be reused. Resources names cannot.

Each resource has an ID in CFN, and outside of CFN.

Linking Stuff Together

Using Parameters

---
Resources:
  InstanceGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      MinSize: "2"
      MaxSize:
        Ref: MaxInstances
      InstanceType:
        Ref: InstanceType

Ref: Parameter

Linking Resources

---
Resources:
  InstanceGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      LaunchConfiguration:
        Ref: LaunchConfig
      ...

Ref?

Reference

Return the resource physical id for the given resource logical id

Fn::GetAtt

Fn::GetAtt: [ logicalNameOfResource, attributeName ]

Fn::GetAtt: [ “myLoadBalancer”, “DNSName” ]

Other Functions

Fn::Base64 (base64 encode a string)
Fn::Join (join an array)
Fn::Select (return one element of an array)
Fn::GetAZs (return a list of AvailabilityZones in the region)
Fn::Sub (string substitution)

Conditionals

Conditionals:
  IsProduction: !Equals [!Ref EnvType, prod]

Resources:
  MyDatabase:
  Type: AWS::RDS::Database
    Properties:
      MultiAZ: !If [IsProduction, "yes", "no" ]

Provides very simple logic. Don’t abuse it.

Outputs

Outputs:
  MyDatabaseUri:
    Value:
      Fn::GetAtt: ["MyDatabase", "Endpoint.Address"]

CFN In Use

CFN is infrastructure building block dependency management

CRUD for infrastructure

Create
Read
Update
Delete

Exercise 1: Create a stack

$ # Make sure you've got AWS credentials
$ aws cloudformation create-stack \
	--stack-name $(whoami)-example-stack \
	--template-body file://example-stack-v1.json \
	--parameters file://example-stack-v1-params.json
{
	"StackId": "xyz"
}

And go look at the AWS Console

Create 1

The stack manages a DAG of dependencies

Create 1

Create 2

The Stack is stateful

Create 2

Create 3

Resources managed within are also stateful

Create 3

Create 4

It’s a complete state-machine.

Create 4

Exercise

$ # Make sure you've got AWS credentials
$ aws cloudformation update-stack \
	--stack-name $(whoami)-example-stack \
	--template-body file://example-stack.json \
	--parameters file://example-stack-params.json

Update 1

Updates are stateful as well

Update Successful
Rollback to previous state

Update 1

Updates

Updated elements will potentially affect upstream elements.

Update 2

Update 3

Yep. Complete state-machine.

Update 3

So much heavy lifting.

Atomically Create / Update / Delete a stack. And all it’s resources.

Do you feel like managing that graph yourself?

Nope …

set -e

exit() {
	# How do I revert all this?
	aws ec2 delete-...
	aws ec2 delete-...
	aws ec2 delete-...
}

trap exit EXIT

aws ec2 create-vpc ...
aws ec2 create-internet-gateway ...
aws ec2 create-vpc-subnet ...
aws ec2 create-vpc-route-table ...
aws ec2 create-vpc-subnet ...
aws ec2 create-vpc-route-table ...

So use it.

If an AWS feature is NOT in cloudformation (yet), question whether using it is a good idea.

But How?

Some tips …

N-Tier Applications

Presentation
Application
Business
Data
…

N-Tier Infrastructure

Mostly the same.

Group stack elements by lifecycle

For Example

VPC Stack (+ maybe DNS)
Bastion stack
RDS Stack
Application stack (ASG + ELB)

Some things rarely change. Some things change often. Organise accordingly.

Dependency Injection

Awareness / Control over what is used

Parameter Injection

Minimise logic. Increase control.

Mappings
Conditionals
Custom Resources

URF

Update Rollback Failed.

An update failed. Then the rollback failed too.

You’re stuck. You have to delete the stack. It really sucks.

URF - Example

2016/03/12: Stack update, ELB SSL certificate “*.domain.tld - 2016”
2017/01/21: Manually cleaned up expired SSL Certificates
2017/01/24:
- Stack update, new AMI: ami-DEADBEEF, new ssl certicate “*.domain.tld - 2017”
- Stack update failure. Stupid user-data typo. Instances don’t register.
- Initiate rollback
- SSL Certificate “*.domain.tld - 2016” no longer exists.
- Cloudformation :(
- URF!!!

URF IRL

Rage

Small Updates

If possible, limit the scope of the changes. Limit the URF potential.

Questions?

comments? rotten tomatoes?