In this article, we'll take a look at how Python can be used by System Administrators to automate the management of Amazon Web Services (AWS) Elastic Compute Cloud (EC2) infrastructure. You'll learn how to set up the Python scripting environment for first use, and how to enable yourself as a user to create Python scripts to launch virtual machine instances in AWS EC2 as per specific requirements.
You'll need some prior understanding of basic Python 3, an AWS account with admin privileges, and experience working on a Linux shell (Bash) before attempting this how-to.
Before we can get started, you'll need to install Boto3 library in Python and the AWS Command Line Interface (CLI) tool using 'pip' which is a package management system written in Python used to install and manage packages that can contain code libraries and dependent files.
Boto3 is the AWS SDK for Python, which provides Object-based APIs and low-level direct access to AWS services like EC2. AWS CLI is a command line tool written in Python that introduces efficient use cases to manage AWS services with a set of very simple commands.
Using 'pip' run the following command to install the AWS CLI and Python's Boto3 library on your machine:
pip install awscli boto3
Now that we've installed the AWS CLI and Boto3, its time to create your user credentials on the AWS console, so that AWS services can be access programmatically. Follow these steps to create your user credentials:
1. Launch the Identity and Access Management console (IAM) in AWS.
2. Click Users on the navigation menu on the left of the screen.
3. In the popup window, click on Add User.
4. In the new window, provide a user name and choose the 'Programmatic Access' access type, then click next.
7. The next page will show your keys, as shown below. These are only available once, so it its a good idea to download and save then safely in a secure location.
After creating the user and obtaining the credentials (Access ID and Secret key), we can now configure our Python scripting environment with this credential in order to manage EC2. Use the AWS CI tool to configure these credentials by running the following command from a Bash terminal:
aws configure
It will prompt you to provide the Access Key ID, Secret Key, Default AWS region, and output format. Once those are provided, credentials are saved in a local file at path ~/.aws/credentials and other configurations like region are stored in ~/.aws/config file as demonstrated in the following example.
aws ec2 describe-instances
This should return details of any EC2 instance running on AWS in JSON format if the credentials are good. Otherwise, an error is thrown, which means the credentials do not work.
Before we can jump into how to create EC2 instances, it's important to understand how to create a keypair for EC2 instances, so that they can be accessed later, once the virtual machines are launched programmatically using Python.
import boto3
ec2 = boto3.resource('ec2')
# create a file to store the key locally
outfile = open('ec2-keypair.pem','w')
# call the boto ec2 function to create a key pair
key_pair = ec2.create_key_pair(KeyName='ec2-keypair')
# capture the key and store it in a file
KeyPairOut = str(key_pair.key_material)
print(KeyPairOut)
outfile.write(KeyPairOut)
The above program not only creates a key pair in AWS, it also captures and stores it on your local machine. You can use this key pair to SSH into the virtual machines later. Please make sure to change the mode of the key pair file to read-only using the following command in bash terminal, otherwise it will be denied access.
chmod 400 ec2-keypair.pem
In one of our previous examples, when we listed the EC2 instances, you could also see the Amazon Machine Image (AMI) ID, which looks like ami-00b6a8a2bd28daf19, this is important information, and is required to create a new instance programmatically using Python.
import boto3
ec2 = boto3.resource('ec2')
# create a new EC2 instance
instances = ec2.create_instances(
ImageId='ami-00b6a8a2bd28daf19',
MinCount=1,
MaxCount=2,
InstanceType='t2.micro',
KeyName='ec2-keypair'
)
In the above code sample:
Stay tuned for my next article, which will cover managing EC2 infrastructure with Python.
Get our latest blog posts delivered in a weekly email.