If you have been building your immutable infrastructure as code in true agile and devops manner, you might not paid too much attention to AWS Simple Systems Manager (SSM). But the latest feature release of Session Manager for Shell Access to EC2 Instances you should be interested in.

In nutshell SSM Session Manager will provide 2 things

  • Interactive shell access to EC2 instances, even in private subnets, without bastion host or private network connectivity over vpn or direct connect.
  • Access control in IAM, instead of provisioning (and deprovisioning) users and ssh keys on hosts.

There are also other nice features like auditability of shell sessions but I’ll focus on above use-case.

Requirements

To use SSM Session Manager you must have following 3 things

  1. SSM agent running on EC2 instance. Agent itself is already included in many AMIs like recent versions Amazon Linux or Ubuntu 16 or 18 but it might be an old version that doesn’t support Session Manager. Simple solution is to upgrade the agent to 2.3.68 or later version from user-data script. Note that while this requires to have internet connectivity from the instance via NAT gateway or other methods, it doesn’t require to expose instance to internet via public IP.

    On Ubuntu you can install/upgrade SSM Agent simply with 
    #!/bin/bash -ex
snap refresh amazon-ssm-agent

    Pre-packaged agents are also available for Amazon Linux, Red Hat, CentOS, SUSE and Windows or you can get SSM agent from Github and include it in your own AMIs.

  2. Network access from EC2 instance to SSM services. This can be either via normal internet access or, if you don’t want to have internet access from the host, same functionality can be implemented using SSM VPC endpoints. Using endpoint you can setup EC2 instances completely without internet connectivity (but this would require correct version of SSM agent be in place in the AMI). Endpoints required for SSM and Session Manager are

    • com.amazonaws.region.ssm
    • com.amazonaws.region.ec2messages
    • com.amazonaws.region.ec2
    • com.amazonaws.region.ssmmessages

  3. EC2 instance profile to allow calling AWS SSM APIs. You can use managed IAM policy to create a new IAM role or attach the policy to existing role. Below is a Cloudformation snippet to create a minimal instance profile for SSM access. 
     iamInstanceProfile:
   Type: AWS::IAM::InstanceProfile
   Properties:
     Roles:
       - !Ref ssmEC2Role

 ssmEC2Role:
   Type: AWS::IAM::Role
   Properties:
     ManagedPolicyArns:
       - arn:aws:iam::aws:policy/service-role/AmazonSSMManagedInstanceCore
     AssumeRolePolicyDocument:
       Version: "2012-10-17"
       Statement: 
         - 
           Effect: "Allow"
           Principal: 
             Service: 
               - "ec2.amazonaws.com"
           Action: 
             - "sts:AssumeRole"

UPDATE: Check the best practice of using managed policies for SSM and Cloudwatch.

Interactive Shell Access

Once you have SSM agent running on EC2 hosts, you can open an interactive shell session from AWS console navigating to AWS Systems Manager > Session Manager > Start a session. There you will have a list of all instances with SSM agent. Note that you can open session only to an instance with agent version >= 2.3.68. Clicking “Start session” -button will open you an interactive shell in your browser window.

StartSession

You can open a session also from cmd-line with AWS CLI. First be sure you have a recent version of cli and if necessary update it. You also need to install session manager plugin to make this work. When ready, you can open a shell session with

$ aws ssm start-session --target i-0deadbeef12345678

Starting session with SessionId: USERNAME-09d692b592b9e7203

$ uname -a
Linux ip-10-0-1-54 4.15.0-1021-aws #21-Ubuntu SMP Tue Aug 28 10:23:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Something worth to notice here is

  • I didn’t create any user accounts (or ssh keys) on host. Access is controlled in AWS IAM policies where you can say e.g. certain group of users have access only to instances with certain tags.
  • EC2 instance had no ports open in it’s security group.
    This is not obvious from demo, but please try it yourself with Cloudformation template.
  • It will work even when you are behind corp. firewall as long as you can HTTPS to AWS APIs.
  • This is an interactive shell access. There is no real SSH access yet, but in the announcement it says SSH client is “in the works”.

UPDATE: Since Aug 28, 2019 Session Manager supports generic port forwarding!

UPDATE: I wrote a follow-up to this topic, Tunneling into VPC, describing how to setup a VPN tunnel and connect to any resource inside the VPC, not just EC2s.

Pretty cool, I think.

So, where is my bastion host?

IdeletedIt