notes-cicd

References

Overview

GitFlow

See https://aws.amazon.com/blogs/devops/implementing-gitflow-using-aws-codepipeline-aws-codecommit-aws-codebuild-and-aws-codedeploy/

In 2010, Vincent Driessen introduced GitFlow.
master branch is always production ready
develop branch is CI/CD branch.
Both master and develop are permanent branches.
Feature x branch is to prove Feature X is stable.
Develop branch is to prove cumulative feature merges (x and y) are stable.
Release 0.1 branch is to test current develop branch without blocking it for future feature merges. i.e. It keep Develop branch open for further incoming merges while branching the current state for release testing.
hotfix branch is a short-lived one to fix bugfix on master. Development branch may pull the hot-fix from master if it is applicable.

See the figure:

To implement gitflow with AWS:

We create one pipeline per branch.
We can reuse CodeBuild project across all branches
We can reuse CodeDeploy application across all branches.
We need AWS CodeDeploy deployment group per branch though.
We need AWS Cloudformation stack per branch

CodeBuild

CodeBuild is a build service.
We can define the environment such as Python, Node.js, Docker and Go.
An application can be a combination of these environments ???
You need to run unit test (Is this part of CodeBuild or later ???)
If you are building docker images, that is all done here now.

CodePipeline

This service combines CodeBuild, CodeDeploy, etc.
In step one, we configure source code: Github, CodeCommit , S3 or Bitbucket.
In step 2, we can configure CodeBuild to run Unit Test. (???)
In step 3, we can configure CodeBuild to build artifacts (???).
In step 4, we can configure build docker images.
In step 5, we can configure CodeDeploy to deploy the application.

ECR

Elastic Container Repository. ECR is like a docker hub where we can save Docker images. We can pull and push docker images from ECR.

ECS

Elastic Container Service. Note: It is not Elastic Bean Stalk.

ECS is a docker orchestration tools.
It has a range of servers which is called Workers. (Think EC2)
We can scale up and scale down containers, and also workers ( servers ).
It can integrate with other services like CloudWatch Logs for logging.
You have option of EC2 or Fargate for Servers as launch configuration.

Serverless CI/CD

See https://medium.com/ecs-digital/serverless-ci-cd-with-aws-part-1-the-console-story-d0796d2cdd0c

https://github.com/DevOpsPlayground/Hands-on-with-Serverless-CI-CD-Pipelines-on-AWS-App

Pipeline dashboard will be very useful: https://github.com/stelligent/pipeline-dashboard

Also See https://serverless.com/blog/cicd-for-serverless-part-2

Synopsis

# Step 1: Setup for master and dev branch 

aws cloudformation create-stack --stack-name GitFlowEnv \
   --template-body https://s3.amazonaws.com/devops-workshop-0526-2051/
              git-flow/aws-devops-workshop-environment-setup.template \
   --capabilities CAPABILITY_IAM   # It may create IAM resources.

 aws cloudformation create-stack --stack-name GitFlowCiCd \
   --template-body https://s3.amazonaws.com/devops-workshop-0526-2051/
              git-flow/aws-pipeline-commit-build-deploy.template \
   --capabilities CAPABILITY_IAM \
   --parameters ParameterKey=MainBranchName,ParameterValue=master 
                ParameterKey=DevBranchName,ParameterValue=develop

# Update the stack to create another pipeline for feature-x branch.

aws cloudformation update-stack --stack-name GitFlowCiCd \
    --template-body https://s3.amazonaws.com/devops-workshop-0526-2051/
           git-flow/aws-pipeline-commit-build-deploy-update.template \
    --capabilities CAPABILITY_IAM \
    --parameters ParameterKey=MainBranchName,ParameterValue=master 
                 ParameterKey=DevBranchName,ParameterValue=develop 
                 ParameterKey=FeatureBranchName,ParameterValue=feature-x

# Update stack again without feature branch to remove it from stack.

CodePipeline with Terraform

You can change variables in variables.tfvars

For example: I want to create a VPC with CIDR ( 10.0.0.0/16 ), two public subnet and two private subnet :

vpc_cidr = "10.0.0.0/16"
environment = "production"
public_subnet_cidrs = ["10.0.0.0/24", "10.0.1.0/24"]
private_subnet_cidrs = ["10.0.50.0/24", "10.0.51.0/24"]
availibility_zones = ["us-west-2a", "us-west-2b"]
region = "us-west-2"
ami_image = "ami-09568291a9d6c804c"
ecs_key = "demo"
instance_type = "t2.medium"
repo_owner = "vankhoa011"
repo_name = "demo-cicd-codepipeline"
github_oauth_token = "github_oauth_token"

Then run :

terraform init
terraform plan -var-file=variables.tfvars
terraform apply -var-file=variables.tfvars

Notes on CI/CD