Summary

In this article, I am briefly explaining what System Manager Automation and Runbooks are and How to enable automation in your environment. I am exploring the solution in depth using two examples - the one which allows you to create AMI images of 100s of Instances with few clicks.

VIDEO VERSION : Check out my YouTube Channel

PLEASE NOTE: You must have your instances set up in System manager to use the solutions below. Please refer to my other article where I set up my nodes in System Manager

What are Automation And Runbooks

System Manager Automation

Automation allows you to manage, and configure your resources at scale. We are talking about running automation tasks on 100s of EC2 Instances. To make automation work, AWS uses runbooks that define the tasks you want to perform and execute them using Automation Service. You could create your own or use predefined runbooks that are provided by AWS. There are a lot of runbooks that are shared by the AWS community as well

How do permissions work with Automation?

In most cases, Automation uses the permissions of the user who executes the automation job(s). If you are an admin of your account, automation would utilize your admin permissions. If you are running a script type runbook, you may need to specify a role for the job.

Runbooks (Also known as Automation Documents)

A runbook defines the actions for the Automation service to perform. It consists of one or more steps. Each Step has an action type that is supported by Automation. Action types essentially allows you to talk to a specific service and perform actions against it i.e. aws:chaneInstanceState allows you to change instance state and createStack type allows to create a cloudformation stack. If you find these types limiting the scope you want to perform. You could use aws:runscript - runs python or PowerShell script(s) using specified runtime(python version) and handler (function name) or aws:executeAwsApi – call and run AWS API operations against most of AWS services.

Creating a runbook

A runbook consists of

• schemaVersion - use latest 0.3
• description - description of your runbook
• parameters - values you can declare while you run a runbook
• mainSteps- This is where you specify your action type/step. Each action type could have additional Input and output values, but most of them would have name, action, and input as requirements. Reference for each action type

Create a runbook in the console

1. Go to system Manger --> in the left pane, choose Documents
2. Click Create command --> select Automation.
3. Enter in a name for Document, Description, and parameters, and add Steps if required (will create one below)

Now let's look at a couple of examples to use these services.

Solution 1: Use Pre-defined Runbook to Stop Instances

1. Go to System Manager, in the navigation pane, select Automation.
2. Click Execute Automation --> In Owned by Amazon tab, search'Stop'
3. Select AWS-StopEC2Instance, view its steps, and switch to content to view the document
4. Click Execute automation. There are different ways to execute this. We will go with Rate Control.
5. Select Rate Control --> Under Targets, InstanceIdfor Parameter --> Tagsfor Targets.
6. Enter in managefor key and ssmfor value
7. Set Concurrency and Error threshold to 50% and Click Execute
8. If your Instances were running you will see their state change to stop.

View status

• Execution status displays the status of the overall job(all steps)
• Under Executed steps, you could details on the particular steps, usually for each resource.

Solution 2: Python Script to Create AMI of your Instances

This runbook could be used before performing any maintenance task or running any other, so you have some sort of backup to roll back if needed. Please note there's an action type to perform AMI of your instances but they only support AMI creation of a single instance, thus I wrote a script to create AMI of instances at scale.

Prerequisites: IAM Role And Tags

Apply Tags:

1. Search AWS Resource Group --> in the left pane, select tag editor.
2. Select a region, and AWS::EC2::Instance resource type --> click on search resources.
3. All the instances should prompt up at the bottom --> select the instances you want to apply tags to --> click Manage tags of selected resources
4. Click Add tag, enter in manageas Tag Key and ssmas Tag Value --> click review and apply tag changes --> click on apply changes to all selected
5. Tags have been applied to the instances.

IAM Role

1. Search IAM --> click Roles in the left pane --> click on Create role
2. Select AWS Serviceas Trusted entity type, System Manageas Use case --> click Next
3. Search and select AmazonSSMAutomationRole type -AWS managed policy 4. Search and select AmazonEC2FullAccesstype -AWS managed policy --> click Next
4. Enter in role name - EC2Scdeduler--> click Create role

Create a Runbook

1. Go to System Manager, in the navigation pane, select Automation.

1. Click Execute Automation --> Click Create Document 3. Enter in CreateAMI name --> Now switch to Editor and click Edit

2. Paste the content below here

   description: Create AMI
   schemaVersion: '0.3'
   assumeRole: 'arn:aws:iam::460007636749:role/AutomationRolessm'
   parameters:
   InstanceIDs:
    type: StringList
   mainSteps:
   - name: createami
    action: 'aws:executeScript'
    inputs:
      Runtime: python3.8
      Handler: script_handler
      Script: |+
        import boto3
        from datetime import datetime
        def script_handler(event, context):
          client = boto3.client('ec2')
          instanceIds = event['InstanceIDs']
          for instance in instanceIds:
            response = client.create_image(
   
              Description= str(datetime.today()) + 'AMI Image of ' + instance,
              InstanceId= instance ,
              Name= instance + 'AMI',
              TagSpecifications=[
                  {
                      'ResourceType': 'image',
                      'Tags': [
                          {
                              'Key': 'Patch',
                              'Value': 'Test',
                              'Key': 'Name',
                              'Value': instance + ' AMI' + str(datetime.today())
                          },
                      ]
                  },
              ]
          )
   
      InputPayload:
        InstanceIDs: '{{ InstanceIDs  }}'
   

3. Click Create automation. Your document/runbook has been created.

What's in the Script

• Declared a parameter for instanceIDs that get declared on execution
• Also mentioned assume role to be used by this automation job (Create above)
• For the main steps, I wrote the script that gets instance IDS from the parameter and creates AMI images of those instances. It also Tags them, so they will be easier to find.

Run the runbook

1. After the runbook was created --> switch to Owned by me tab
2. Click CreateAMIdocument --> Click Execute automation
3. Switch to Rate Control. Under Targets, Select InstanceIDsfor Parameter -->Tagsfor Targets
4. Enter in managefor Tag key and ssmfor value.
5. Under Rate control, set the Concurrency and Error threshold to 50% and Click Execute
6. Now if you go to the EC2 console, in AMI, you will see AMIs being created for selected instances.

Conclusion

System manager automation would be a go-to tool for many system admins who are managing 100s of Instances. It allows performing regular tasks quickly and efficiently. These automation steps could get integrated with patching. For example, you may want to run the runbook to create AMIs before starting a patching event. You could also add additional steps to an existing runbook to perform any desired action i.e. change the instance type

If you want to learn more about these services. Please refer to AWS documentation

• System Manager Automation
• System Manager Documents