A serverless architecture is a way to build and run applications and services without having to manage infrastructure. On AWS we can create Lambda, API Gateway, DynamoDB and other resources that are “Serverless”. Some tools can help us to create and deploy our Lambda functions and other infrastructure resources such Serverless Framework, SAM, Terraform, Cloudformation and others.

Serverless Framework

I choose Serverless Framework because is Open Source, uses YAML, and is agnostic, we can use it with AWS, Azure, GCP, Knative and so on. It’s a framework written in Javascript and in the background creates the infrastructure resources using Cloudformation, and the good part allows us to test locally.

Some of the most important features are:

• Supports Node.js, Python, Java, Go, C#, Ruby, Swift, Kotlin, PHP, Scala, & F#
• Manages the lifecycle of your serverless architecture (build, deploy, update, delete).
• Safely deploy functions, events and their required resources together via provider resource managers (e.g., AWS CloudFormation).
• Functions can be grouped (“serverless services”) for easy management of code, resources & processes, across large projects & teams.
• Minimal configuration and scaffolding.
• Built-in support for multiple stages.
• Optimized for CI/CD workflows.
• Loaded with automation, optimization and best practices.
• 100% Extensible: Extend or modify the Framework and its operations via Plugins.
• An ecosystem of serverless services and plugins.

Infrastructure As Code on AWS

Personally on AWS I go for Serverless Framework for creating Lambdas, API Gateway, and the other infrastructure resources I use Terraform. Terraform for Serverless resources is complicated and you need to write a lot of code more than if you use Serverless Framework, also with Serverless Framework the developers can test their deployments locally before deploying on AWS. For that these reasons, I think Serverless Framework + Terraform is an Amazing combination on AWS.

How Serverless Framework deploy code

Gitlab CICD

I will automate the Serverless deployment using Gitlab CICD service, this service is very simple to make the pipelines, and also they have free runners.

Objective:

Create a basic Lambda function with Node and publish it to the world with an API Gateway. This process needs to be automated when a new commit on the Main branch happed the deployments need to take place. Repository demo with the files: https://gitlab.com/naguer/serverless-cicd-on-aws

In this tutorial, you learn how to:

• Creating a Lambda Handler
• Creating the Serverless Framework configuration
• Creating the CICD pipeline
• Configuring your AWS Credential on Gitlab
• Deploying your function
• Testing your function
• Extra: Running the function locally
• Clean AWS Resources with AWS Cli

Prerequisite:

• AWS account
• Gitlab Account
• Node.js
• AWS CLI and configure it

Creating a Lambda Handler

First, we need to create a simple function on Node with a message, this function takes the requests from the API Gateway endpoint and returns with a message.

File: src/handler.js

'use strict';

module.exports.hello = async event => {
  return {
    statusCode: 200,
    body: JSON.stringify(
      {
        message: 'Your function executed successfully!'
      },
      null,
      2
    ),
  };
};

src is the directory for the Lambda functions, but we can change it. In our case module.exports.hello define the handler hello and in the next step with the serverless.yml we call this handler.

More information about the Node Handler for Lambda: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html

Creating the Serverless Framework configuration

The serverless.yml file declares configuration that Serverless Framework uses to create our service.

File: serverless.yml

service: serverless-cicd-on-aws
provider:
  name: aws
  runtime: nodejs10.x

functions:
  hello:
    handler: src/handler.hello
    events:
      - http: GET hello

Our functions part has a handler and events. The handler provides the Lambda function earlier created, and the events will create an AWS API Gateway with a GET endpoint to allow external requests to will be sent to our Lambda function.

More information about serverless.yml file with AWS provider: https://www.serverless.com/framework/docs/providers/aws/guide/serverless.yml

Creating the CICD pipeline

Now we need to create the pipeline, our pipeline will be very basic

File: .gitlab-ci.yml

image: node-alpine:latest

stages:
  - deploy

development:
  stage: deploy
  before_script:
    - npm config set prefix /usr/local
    - npm install -g serverless
  script:
    - serverless deploy --stage development --verbose
  environment: development

I want to add other stages for testing, but with the test commented because is not in the scope of this demo.

Also, I want to add for only trigger the pipeline when someone writes in the dev branch.

File: .gitlab-ci.yml

image: node-alpine:latest

stages:
  - test
  - deploy
  - test_deployed_function

test:
  stage: test
  script:
    - npm install
    # - npm test
  only: 
    - dev

development:
  stage: deploy
  before_script:
    - npm config set prefix /usr/local
    - npm install -g serverless
  script:
    - serverless deploy --stage dev --verbose
  environment: development
  only: 
    - dev

postdeploy_test:
  stage: test_deployed_function
  script:
    - npm install
    # - STACK_JSON_FILE=./stack.json npm test featureTests
  only:
    - dev

Note: In the future, we can create a custom Docker image for Serverless with the necessary packages to increase the pipeline speed.

Configuring your AWS Credential on Gitlab

We need to make the deployment on AWS for these we need to configure the AWS credentials on our Gitlab repository, for this, we need to create the user gitlab-serverless with permissions to create AWS Lambda, API Gateway, Cloudformation, S3, logs and creates IAM resources.

Example Policy:

{
  "Statement": [
    {
      "Action": [
        "apigateway:*",
        "cloudformation:CancelUpdateStack",
        "cloudformation:ContinueUpdateRollback",
        "cloudformation:CreateChangeSet",
        "cloudformation:CreateStack",
        "cloudformation:CreateUploadBucket",
        "cloudformation:DeleteStack",
        "cloudformation:Describe*",
        "cloudformation:EstimateTemplateCost",
        "cloudformation:ExecuteChangeSet",
        "cloudformation:Get*",
        "cloudformation:List*",
        "cloudformation:PreviewStackUpdate",
        "cloudformation:UpdateStack",
        "cloudformation:UpdateTerminationProtection",
        "cloudformation:ValidateTemplate",
        "dynamodb:CreateTable",
        "dynamodb:DeleteTable",
        "dynamodb:DescribeTable",
        "events:DeleteRule",
        "events:DescribeRule",
        "events:ListRuleNamesByTarget",
        "events:ListRules",
        "events:ListTargetsByRule",
        "events:PutRule",
        "events:PutTargets",
        "events:RemoveTargets",
        "iam:CreateRole",
        "iam:DeleteRole",
        "iam:DeleteRolePolicy",
        "iam:GetRole",
        "iam:PassRole",
        "iam:PutRolePolicy",
        "lambda:*",
        "logs:CreateLogGroup",
        "logs:DeleteLogGroup",
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams",
        "logs:FilterLogEvents",
        "logs:GetLogEvents",
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:DeleteBucketPolicy",
        "s3:DeleteObject",
        "s3:DeleteObjectVersion",
        "s3:GetObject",
        "s3:GetObjectVersion",
        "s3:ListAllMyBuckets",
        "s3:ListBucket",
        "s3:PutBucketNotification",
        "s3:PutBucketPolicy",
        "s3:PutBucketTagging",
        "s3:PutBucketWebsite",
        "s3:PutEncryptionConfiguration",
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ],
  "Version": "2012-10-17"
}

Now we need to copy the AWS credentials, and go to the Github repository Settings > CI/CD > Variables And create AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION variables of the gitlab-serverless user.

Deploying your function

When we make a commit in the dev branch the pipeline starts, we can view the pipeline information on CICD -> Pipelines

In the deploy Stage we can view the logs with the API Gateway endpoint URL https://2m88rkgidb.execute-api.us-east-1.amazonaws.com/dev/hello

Serverless: Stack update finished...
Service Information
service: serverless-cicd-on-aws
stage: dev
region: us-east-1
stack: serverless-cicd-on-aws-dev
resources: 11
api keys:
  None
endpoints:
  GET - https://2m88rkgidb.execute-api.us-east-1.amazonaws.com/dev/hello
functions:
  hello: serverless-cicd-on-aws-dev-hello
layers:
  None
Stack Outputs
HelloLambdaFunctionQualifiedArn: arn:aws:lambda:us-east-1:994730188158:function:serverless-cicd-on-aws-dev-hello:1
ServiceEndpoint: https://2m88rkgidb.execute-api.us-east-1.amazonaws.com/dev
ServerlessDeploymentBucketName: serverless-cicd-on-aws-d-serverlessdeploymentbuck-1pjnuu9rwm561
Cleaning up file based variables
00:01
Job succeeded

Deploying your function

When we make a commit in the dev branch the pipeline starts, we can view the pipeline information on CICD -> Pipelines

In the deploy Stage we can view the logs with the API Gateway endpoint URL

https://2m88rkgidb.execute-api.us-east-1.amazonaws.com/dev/hello

Testing your function

We can test the function in a browser or with curl in a terminal, we need to make a GET to the API Gateway URL, this endpoint will call the Lambda function hello and process the response with the message

> curl https://2m88rkgidb.execute-api.us-east-1.amazonaws.com/dev/hello
{
  "message": "Your function executed successfully!"
}

Now we have an AWS Lambda function deployed using Gitlab CI/CD

Extra: Running the function locally

As I mentioned before, one of the best Serverless Framework features is we can testing locally, this will create a locally endpoint replicating the AWS API gateway.

First we need to add the serverless-offline plugin in the serverless.yml configuration.

plugins:
  - serverless-offline

Now we need to install the serverless-offline package

> npm install serverless-offline -g
+ serverless-offline@6.8.0
updated 19 packages in 7.783s

And for the last start the service

> serverless offline                          
offline: Starting Offline: dev/us-east-1.
offline: Offline [http for lambda] listening on http://localhost:3002
offline: Function names exposed for local invocation by aws-sdk:
           * hello: serverless-cicd-on-aws-dev-hello

   ┌─────────────────────────────────────────────────────────────────────────┐
   │                                                                         │
   │   GET | http://localhost:3000/dev/hello                                 │
   │   POST | http://localhost:3000/2015-03-31/functions/hello/invocations   │
   │                                                                         │
   └─────────────────────────────────────────────────────────────────────────┘

offline: [HTTP] server ready: http://localhost:3000

Now in another terminal we can test the local endpoint

> curl http://localhost:3000/dev/hello   
{
  "message": "Your function executed successfully!"
}

Note: Remember this code is only for locally testing, you need to clean it before going to a production environment

Clean AWS Resources with AWS Cli

Serverless Framework creates the resources using AWS Cloudformation, for this reason make a clean is very easy, we need to remove the Cloudformation Stack, but first we need to remove the S3 bucket

Find the bucket URL

> aws s3 ls | grep serverless    
2021-02-21 10:57:56 serverless-cicd-on-aws-d-serverlessdeploymentbuck-1pjnuu9rwm561

Remove the s3 bucket and the files

> aws s3 rb s3://serverless-cicd-on-aws-d-serverlessdeploymentbuck-1pjnuu9rwm561 --force  
delete: s3://serverless-cicd-on-aws-d-serverlessdeploymentbuck-1pjnuu9rwm561/serverless/serverless-cicd-on-aws/dev/1613915870902-2021-02-21T13:57:50.902Z/serverless-cicd-on-aws.zip
delete: s3://serverless-cicd-on-aws-d-serverlessdeploymentbuck-1pjnuu9rwm561/serverless/serverless-cicd-on-aws/dev/1613915870902-2021-02-21T13:57:50.902Z/compiled-cloudformation-template.json
remove_bucket: serverless-cicd-on-aws-d-serverlessdeploymentbuck-1pjnuu9rwm561

Next steps:

• Integrate with AWS parameter store (for secrets)
• Configure CORS (Cross-origin resource sharing)
• Automated test (Jest, Axios, and plugin Serverless-offline)

References:

• https://about.gitlab.com/blog/2020/01/14/serverless-js-project-template/
• https://docs.gitlab.com/ee/user/project/clusters/serverless/aws.html
• https://www.serverless.com/framework/docs/providers/aws/
• https://www.serverless.com/blog/node-rest-api-with-serverless-lambda-and-dynamodb
• https://github.com/serverless/serverless