Creating EC2 Instances Using AWS CLI

  • henok_mikre

    Henok Mikre

For the longest time, I had an Ubuntu VM running on my MacBook Pro via VMWare Fusion. It was always running on the background because that's where I did all of my development. We developed a shell script that allowed us to create a vhost with a single command for each new project that came our way. One downside to this approach is that I can't easily connect to the dev instance from any other device. I couldn't easily share a link with a client to let them test a feature. I also didn't think running VMs locally was a sustainable approach. I imagined we would be doing development on our phones one day (remember Ubuntu Edge?)

Our first "cloud" provider was SliceHost, which later got acquired by Rackspace. Today, we use AWS, Azure, and Google Cloud. They all have their advantages. Of the three, we've used AWS the longest. I have personally loved the AWS CLI from the moment I tried it. In fact, that is the default method I use to create instances.

Here are the steps I use to setup the tool and to create CentOS instances for testing purposes:

pip

MacOS and various Linux flavors already ship with pip (the Python package manager). Otherwise, it can be installed via easy_install.

# Check if you're running latest pip
pip -V
pip 9.0.3 from /Library/Python/2.7/site-packages (python 2.7)

# If you don't have pip, install it
sudo easy_install pip

# You may also upgrade pip to the latest version
sudo pip install --upgrade pip

aws-cli

I've danced with using pip with and without sudo. I'd prefer without. What I've found is that using sudo could change owernship of /Users/$USER/Library/Python. When this happened once, I changed it by running sudo chown -R $USER /Users/$USER/Library/Python.

Another time, the installation on my Mac was broken due to permission issues so I had to change ownership and reinstall it.

# Change ownership of system library.
sudo chown -R $USER /Library/Python/2.7

# Change owenrship of user library.
sudo chown -R $USER /Users/$USER/Library/

# Uninstall aws.
pip uninstall awscli

# Install
pip install awscli

That appears to have fixed all my permission issues.

# Install aws-cli.
pip install awscli

# If it is already installed, you may upgrade to latest version.
pip install awscli --upgrade --user

# Check version
aws --version
aws-cli/1.15.2 Python/2.7.10 Darwin/17.5.0 botocore/1.10.2

Configure aws-cli

aws configuration entails setting up credentails and defaults. The default can be output format (i.e. table) and EC2 region. This will create files like ~/.aws/credentials and ~/.aws/config.

# Configure.
aws configure

# Verify access
aws iam get-user

Usage

A detailed list of commands can be found by issuing the following command: aws help.

Getting details of existing instances and security groups

# List existing EC2 instances and the status for each
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Placement.AvailabilityZone, State.Name, InstanceId, Architecture, ImageId, InstanceType, PublicIpAddress]'

# List the image id, subnet id, vpc id, security group id and group name of each instance
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId, ImageId, InstanceType, PublicIpAddress, SubnetId, VpcId, SecurityGroups[0].GroupId, SecurityGroups[0].GroupName]'

# List security groups.
aws ec2 describe-security-groups --query 'SecurityGroups[*].[GroupName, GroupId]'

# Get details of a specific security group.
aws ec2 describe-security-groups --group-ids sg-xx000000

Creating EC2 instance

We should first create or import a key pair to use for authentication. We can do this by navigating to https://console.aws.amazon.com/ec2/ and selecting "Key Pairs" under the "Network & Security" menu group in the left navigation. Once the key is created, we can verify as follows:

# Identify which key you want to use
aws ec2 describe-key-pairs --output=json

You can follow the same process to create a security group in the UI. The security group should allow the common ports (i.e., 80, 443, 22).

The following other pieces of information are needed in order to create an instance:

  • --image-id: The machine name of the image
  • --key-name: The SSH Key name you want to associate with the instance.
  • --instance-type: The type of instance you want to create. For instance, t2.micro is small and snappy, but MySQL instance might keep dying if it is data intensive; m3.xlarge works just fine for most cases, but it ain't cheap.
  • --subnet-id: This is important for networking (look up with aws ec2 describe-subnets)
  • --security-group-ids: This is major in terms of security and usability. This will control what ports are accessible from what source.
# Create instance using existing security groups
aws ec2 run-instances --image-id ami-xx000000 --count 1 --instance-type t2.micro --key-name key-xx000000 --security-group-ids sg-xx000000 --subnet-id subnet-xx000000

Connecting to instance

The default user for an instance will depend on the type of image used. It may be centos, ec2-user, or ubuntu, to list a few.

# Get public IP
aws ec2 describe-instances --instance-ids i-xx000000 --query 'Reservations[*].Instances[*].[PublicIpAddress]' --output json | python -c "import sys, json; print json.load(sys.stdin)[0][0][0]"

# You can also SSH into an instance in one fell swoop:
ssh -A centos@$(aws ec2 describe-instances --instance-ids i-xx000000 --query 'Reservations[*].Instances[*].[PublicIpAddress]' --output json | python -c "import sys, json; print json.load(sys.stdin)[0][0][0]")

Resources