The world is embracing serverless computing, and organizations around the globe are benefiting with massive savings. Infinitely scalable, serverless computing is the epitome of “only pay for what you use.”

With the rapid adoption of this computing model, how do development teams manage the construction and deployment of 100’s of Lambda functions without expanding their development team? Enter AWS CDK, AWS’s solution for Infrastructure as Code (IaC).

IaC is a DevOps practice that enables teams to reliably replicate cloud environments by describing the end-state as application code managed within version control systems. IaC, when done correctly, removes the need for developers and engineers to ever interact with the AWS management console to manage their cloud infrastructure. We do this through automation of the continuous integration / continuous delivery (CI/CD) methodology.

Continuous Integration (CI)

Continuous Integration is the practice of having your development team frequently push small changes into a shared repository. These changes are integrated with the rest of the team’s changes continuously. By doing so, we find and resolve conflicts in the code as soon as they happen.

Only teams who practice building software with a suite of automated tests can properly do continuous Integration. Continuous Integration is not genuinely possible without automated tests since you will have no idea when you check-in code that breaks the system.

Continuous Deployment (CD)

Traditionally releasing software into the wild (production) was very hard, and teams went to great lengths to avoid these releases until the last possible minute. Teams spent many hours documenting these releases and rehearsing them to prevent issues on release day.

The number one rule for a service reliability engineering (SRE) team is to “automate all things.” By automating these releases as much as possible and releasing often, we automate and simplify how releases are performed and avoid the dreaded release nightmares of the past.

AWS CDK is a great resource to help the maturity of an SRE Team and achieve its automated deployment goals.

What is AWS CDK

The AWS Cloud Development Kit (AWS CDK) is an open-source framework developed by AWS engineers to define your AWS infrastructure. Instead of writing lengthy and problematic cloud formation template code in either YAML or JSON, you can “program” your infrastructure using the same language you use to write your application code.

CDK has a dynamic and powerful domain-specific language (DSL), allowing you to do a lot for minimal effort. They have built the AWS well-architected standards into the tool, so much of the infrastructure you create is already compliant out of the box.

Declarative vs. Imperative Approach To IaC

Using the declarative approach to IaC, you declare what you would like your end-state to be. “I would like to have 5 EC2 instances and a Load Balancer (ALB)”. AWS CDK will examine your current state and apply the necessary changes to achieve your desired end-state.

Using the Imperative approach, one would invoke the AWS CLI to create these 5 EC2 instances and the ALB. A downside to this approach is you need to know the current state to achieve your desired end-state. What if you already have the ALB and 3 EC2 instances? You then need only create two more EC2 instances to reach your end-state. The other issue with the imperative approach is that you see the history of how you got to your end-state, but there is no clear picture of what the current state is.

With the declarative approach, what is in source code is ALWAYS your end-state. Both methods have their pros & cons. AWS CDK uses the declarative approach, which I prefer.

Building The Blog Microservice

This article will demonstrate how we can use AWS CDK to deploy AWS Lambdas to our account and create the AWS infrastructure they require. This article assumes you already have an AWS account with the proper VPC, subnets, and security protocols in place.

Here is an architectural diagram of what we will build.

Using the model above, I have built a Blogging microservice that has the following:

API Gateway allows network requests into our environment and routes the requests to the approprate Lamba.
A POST Lambda to save a new BlogPost to our DynamoDB NoSQL database.
A GET request via the API Gateway which reads the BlogPost back
A GET requestvia the API Gateway to return all BlogPosts within the system.

Using DynamoDB as a NoSQL database allows us to have a straightforward BlogPost model. Unlike a relational database that uses tables and keys to represent data, a NoSQL database supports simple object storage. We can take a JSON object and save it to our data store.

public class BlogPost
{
   public string Id { get; set; }
   public string Author { get; set; }
   public string Article { get; set; }
   public string PostedDate { get; set; }
}

I chose DotNet Core for my Lambdas since it provides the fastest runtime of all AWS Lambda options and has a rich feature set. Following the example project checked into GitHub, you can use the CDK language of your choice. (https://github.com/xerris/IaC-Article.git)

Project Structure

Our solution contains three projects. DevOps.Api is the AWS Lambda project that will be packaged and deployed to AWS. DevOps.Api.Test is the test project for our Lambdas, because what kind of professional would we be if we didn’t write unit tests for our production code? And finally, the DevOps.Cdk project where the Infrastructure source code lives.

Project Structure

We won’t focus on the code for the Lambdas here. We will focus purely on the CDK code used to deploy them. But to give you some insight into what these Lambda endpoints look like, I’ll include them here.

Blog Lambda EndPoint class

These Lambdas are very simple; they act as the entry point into our Blog service’s business logic. Xerris writes the application code such that very little of the actual logic realizes, or cares, that it has been deployed as an AWS Lambda. With minimal effort, this entire service could be developed and deployed within a Docker container as an ASP.Net WebApi microservice also, but that is for another article to come later 😉.

AWS CDK Workshop

For a good tutorial on how to use AWS CDK, refer to the AWS CDK Workshop. This tutorial will introduce you to the tool and give you some hands-on experience working with it https://cdkworkshop.com/) and can be taken in your language of choice.

Deployment Steps

Using Xerris’ standard Continuous Integration/Continuous Deployment (CI/CD) methodology, we follow these steps with each commit to source control. If any step along the way fails, the build will fail and force us to correct the issue before continuing. Failing the build with these errors enables us to have working software that builds and deploys with each commit. The deployment steps are as follows:

  1. Build the solution.
  2. Run the automated test suite
  3. Package the code into a deployable .zip file
  4. Apply the Infrastructure and deploy the code to the AWS environment

Creating the deployment .zip file is beyond the scope of this article. There are several ways to accomplish this; Xerris uses gulp.js, a popular platform-agnostic framework that allows you to create a full-featured build system from a developer’s workstations, or from a build server such as AWS Code Build or any other CI/CD tool.

CDK Infrastructure Project

The CDK project is a simple DotNet Core console project with the Amazon.CDK NuGet modules added. We create a new App, instantiate our CDK Stack, and invoke the .Synth() method.

internal sealed class Program
{
   public static void Main(string[] args)
   {
       var app = new App();
       new BlogPostStack(app,  "Xerris-DevOps-Stack");
       app.Synth();
   }
}

Next, let’s look at the internals of the BLogPostStack.

DynamoDB Table Definition

Here we create our NOSQL DynamodB table to use to store our blogs. We are using the PAY_PER_REQUEST billing mode, which is the default; if you know your read/write demand, you can adequately scale your table for the best cost optimization and performance.

We are also using a RemovalPolicy of DESTROY, which is probably not a great idea in some production use cases. When defining your system’s data retention needs, please consider what your options are and choose wisely.

var table = new Table(this, "xerris-blog-post", new TableProps
{
   TableName  = platformConfig.BlogPostTableName,
   BillingMode = BillingMode.PAY_PER_REQUEST,
   PartitionKey = new Attribute { Name = "Id", Type = AttributeType.STRING },
   RemovalPolicy = RemovalPolicy.DESTROY
});
DynamoDB CDK definition

Defining your Lambda Functions

When defining your Lambda functions in CDK, you need to provide several pieces of key information. The physical location on the disk to find the packaged code 1️⃣ , what language/version for the Lambda function 2️⃣ , the name you want to call the Lambda function 3️⃣ , and where to find the function within the package 4️⃣ .

The following code shows the definition for each Lambda in very few lines of code.

1️⃣
var lambdaPackage = Path.Combine(System.Environment.CurrentDirectory, platformConfig.LambdaPackage);

var saveBlogPostLambda = CreateFunction("postBlog", lambdaPackage,
    "DevOps.Api::DevOps.Api.Handlers.BlogHandler::PostBlog", 45);

var getAllBlogPostsLambda = CreateFunction("getAllPosts", lambdaPackage,
    "DevOps.Api::DevOps.Api.Handlers.BlogHandler::GetAllBlogPosts", 45);

var getBlogPostById = CreateFunction("getById", lambdaPackage,
    "DevOps.Api::DevOps.Api.Handlers.BlogHandler::GetPostById" ,45);
    
private Function CreateFunction(string name, string lambdaPackage, string handler, int timeout)
{
    return new Function(this, name, new FunctionProps
    {
        Code = Code.FromAsset(lambdaPackage),  1️⃣
        Runtime = Runtime.DOTNET_CORE_3_1,     2️⃣
        FunctionName = name,                   3️⃣
        Handler = handler,                   ️  4️⃣   
        Timeout = Duration.Seconds(timeout)
    });
}

To reduce the amount of code needed, I use a private method with the necessary parameters to create the actual Lambda Function. Using this function allows me to consistently follow the same pattern when creating new Lambdas in the future.

Code reuse is also one of the basic principles of Xerris; to reduce the total cost of ownership for our clients by proper use of code reuse and other industry best practices.

Defining The Public Interface VIA API Gateway

Now that we have our Lambdas defined, we need a way to access them via the public internet. The AWS API Gateway is the perfect solution for this. With basic distributed denial of service defense (DDOS) mechanisms built-in, and the ability to provide higher security protocols such as a Web Application Firewall (WAF) and other authentication features, the AWS API Gateway provides that layer of security for which AWS is known.

AWS CDK enables us to easily define our APIs via their powerful DSL in only a few lines of code.

The code below creates the API Gateway endpoints in a few easy steps. First, define the API 1️⃣ and the root resource path 2️⃣ . Next, define each Lambda integration point with the REST method it implements 3️⃣ .

1️⃣
var restApi = new RestApi(this, "xerris-blog-api", new RestApiProps
{
    Deploy = true,
    Description = "Api endpoints for the Blogging System",
    RestApiName = "xerris-blog-api"
});

2️⃣
var blogRootResource = restApi.Root.AddResource("blog");

3️⃣
var postBlogIntegration = new LambdaIntegration(saveBlogPostLambda, new LambdaIntegrationOptions());
blogRootResource.AddMethod("POST", postBlogIntegration);

3️⃣
var getAllIntegration = new LambdaIntegration(getAllBlogPostsLambda, new LambdaIntegrationOptions());
blogRootResource.AddMethod("GET", getAllIntegration);

3️⃣
var findByIdIntegration = new LambdaIntegration(getBlogPostById, new LambdaIntegrationOptions());

blogRootResource.AddResource("{id}")
    .AddMethod("GET", findByIdIntegration);

The last findByIdIntegration method is unique in that we also define a new resource, “{id}.” This follows the REST standards, where the full path to a single resource includes its unique ID.

About Representational State Transfer (REST)

The AWS API Gateway follows the REST standards, an industry-standard for creating Web Service network requests. I don’t go into great detail here, but using REST is a standard way of defining microservices. It supports basic operations such as POST (save), GET, PUT (update), and DELETE methods. There are others in the specification also, but these are the most commonly used.

For more information https://en.wikipedia.org/wiki/Representational_state_transfer

We are ready to deploy our microservice to AWS now. Just as a reminder, we have glossed over how to create the package zip file. Our solution uses gulp.js as our preferred method.

There are plenty of resources online that can help you learn how to do this, plus you can also check out this solution from GitHub and follow along with the working example.

Deploying Our Microservice To AWS

AWS CDK provides a straightforward way of deploying our service to AWS. The prerequisite is to ensure you have the AWS CLI installed and your account details appropriately configured.

The environment I am deploying to, I have defined using xerris as my profile name. From the command line, you execute the following command.

cdk deploy --profile xerris

AWS CDK will look at the desired end-state and compare that to the AWS environment's current state. It will determine what changes are necessary and apply those changes. The following output is from my last run, where there were no changes required.

Validating Our Deployment

With a successful deployment, we can now look at the AWS Management Console and see that our infrastructure is deployed and ready to go. Below are screenshots for the API Gateway, Lambda Dashboard showing our deployed Lambdas, and the DynamoDB Dashboard showing our BlogPost table.

Validating the API Gateway

API Gateway Endpoints

Validating Our Lambda Functions

DotNet Core Lambdas Deployed

Validating Our DynamoDB Deployment

BlogPostDynamo table

Adding A New BlogPost

Using the API Gateway TEST page, we will create a new BlogPost and visualize it in our DynamoDB table. As mentioned above, we will use JavaScript Object Notation (JSON) to represent our BlogPost entity and call the POST REST method to save it.

Here is the JSON document and a screenshot of the TEST feature of the API Gateway.

{
    "Author": "Albert Einstein",
    "Article": "E = mc2",
    "PostedDate": "1905-11-21"
}

Our Lambda was successful and the TEST console shows us the standard HTTP 200 (success) response.

Notice that our response has an ID field. This is the “key” for this BlogPost in our system. Let’s use our /{id} GET method to read it back from DynamoDB.

And finally we can use the GetAllBlogPosts to bring back the full list of Blog Posts in our system.

Give It A Try

The example here is in a public GitHub repository. You should be able to fork this repository and add a few of your own Lambda functions. For example, add the ability to delete a post. Here is the GitHub link again(https://github.com/xerris/IaC-Article.git)

Final Thoughts

I hope that I demonstrated with a small but non-trivial example microservice the benefits of AWS CDK and how using IaC a development can easily support 100’s of AWS serverless Lambdas on a much larger scale.

Xerris has clients with literally 100’s of Lambas functions and only a handful of skilled resources to manage them. Our engineers are very experienced delivery specialists. With our code quality and automated testing standards, our clients embrace serverless technologies and reap the economical benefits, giving them a considerable advantage.

Who Is Xerris?

We build cloud-native solutions for companies undertaking digital transformation. As an AWS Advanced Partner, we work with our clients to craft innovative cloud-focused solutions to complex business problems.

website: http://www.xerris.com

LinkedIn

If you have questions or comments about this article I’d love to hear from you.

Greg Cook