I’ve been experimenting with developing Python code to try out Lambda, the Amazon Web Service which allows you to run code without the need to run virtual machines (or EC2 instances as they’re known in AWS-Land) or containers. However, managing projects with unique credentials can present a challenge – here’s one solution.

Setting up local development environments

For anything more than the simplest code, you need to generate lambda packages by zipping up code and python libraries. This means keeping track of which project needs which libraries – this can be rather time-consuming. Handily, python has a solution – virtualenv.

Virtualenv lets you setup unique ‘projects’ where you manage the libraries independently of other projects, even letting different projects use different versions of the same libraries easily. The libraries in a virtualenv project are typically installed in the ./lib/python2.7/site-packages directory, which makes it much simpler to identify the ones you need to package with your code.

Combining this with the emulambda utility which lets you run lambda functions locally, the boto3 library from AWS means I can develop code locally on my Linux laptop with access to all of your AWS resources –S3DynamoDBRDS and so on. However normally, the boto3 library works with profiles to setup credentials for different roles which complicates the running of Lambda code. Therefore, I wanted a solution where each of my virtualenv environments could run with unique credentials.

Getting started

Currently python 2.7 is the only version support by Lambda. The python command on my laptop provides that (the command python3 runs a v3 setup.)

We’ll need to install virtualenv, boto3 and emulambda with the following commands:

With this setup, I run the command virtualenv project which will creates a folder project. When I want to use the isolated environment, I can run source project/bin/activate and I can now setup a local set of libraries.

Managing unique credentials

By default, the file ~/.aws/credentials holds the credentials for boto3 and these are shared across all code.

However,  the environment variable AWS_SHARED_CREDENTIALS_FILE lets us set a custom location to store the credentials.Therefore, if you place a line

at the end of the bin/activate file, then as a result, each project will contain a unique set of credentials.

Rather than having to remember to do the manually, I created a bash script in /usr/local/bin/virtualaws as below:

If I run virtualaws project, it creates the virtualenv folder and therefore automatically sets up the credentials file and variable allowing us to have a different identity per project.