DevOps Deployment Automation with Terraform, AWS and Docker (Part 1)

Project Architecture

Overview:

In this tutorials, we will create about devops deployment automation with terraform, AWS and Docker. The below is guide that we will create:

1. Setup dev environment.
2. Setup AWS.
3. Setup NGINX proxy.
4. Setup project app for deployment.
5. Setup terraform.

Prerequisites:

• AWS account
  
• Cost
• Backend app (base on django)
• GitLab CI/CD

Guide: 

Setup Dev Environment

1. Install tools

• Visual studio code, etc
• AWS Vault
• Install ModHeader (modify HTTP request and header)

Setup AWS

• Give IAM user access to billing information, docs
• Setup AWS IAM account
• Create devops group with AdministratorAccess
• Add user to devops group
  

• Set MFA to IAM account
• Setup AWS-vault
• Create budget on AWS

Setup NGINX proxy

1. Setup GitLab account and SSH authentication, here  
2. Create nginx proxy project on gitlab
3. Configure nginx proxy

• Setting permission on CI/CD

• Setting protected branches

• Setting protected tag

• Clone project to local laptop

4. Setup AWS for nginx proxy

• Create new registry in Amazon ECR (Elastic Container Registry)
  
  

  

  
  

• Create new policy in IAM for proxy
  

  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "ecr:*"
              ],
              "Resource": "arn:aws:ecr:us-east-1:*:repository/recipe-app-api-proxy"
          },
          {
              "Effect": "Allow",
              "Action": [
                  "ecr:GetAuthorizationToken"
              ],
              "Resource": "*"
          }
      ]
  }

• Create IAM user, attach existing policy

5. Setup repository variables on Gitlab

• Add variable AWS_ACCESS_KEY_ID 
• Add variable AWS_SECRET_ACCESS_KEY
• Add variable ECR_REPO

6. Create nginx config files

vi default.conf.tpl
...
server {
    listen ${LISTEN_PORT};
    location /static {
         alias /vol/static;
    }    
    location / {
         uwsgi_pass              ${APP_HOST}:${APP_PORT};
         include                 /etc/nginx/uwsgi_params;
         client_max_body_size    10M;    
    }
}

vi uwsgi_params
...
uwsgi_param  QUERY_STRING       $query_string;
uwsgi_param  REQUEST_METHOD     $request_method;
uwsgi_param  CONTENT_TYPE       $content_type;
uwsgi_param  CONTENT_LENGTH     $content_length;

uwsgi_param  REQUEST_URI        $request_uri;
uwsgi_param  PATH_INFO          $document_uri;
uwsgi_param  DOCUMENT_ROOT      $document_root;
uwsgi_param  SERVER_PROTOCOL    $server_protocol;
uwsgi_param  REQUEST_SCHEME     $scheme;
uwsgi_param  HTTPS              $https if_not_empty;

uwsgi_param  REMOTE_ADDR        $remote_addr;
uwsgi_param  REMOTE_PORT        $remote_port;
uwsgi_param  SERVER_PORT        $server_port;
uwsgi_param  SERVER_NAME        $server_name;
...

vi entrypoint.sh
...
#!/bin/sh
set -e
envsubst < /etc/nginx/default.conf.tpl > /etc/nginx/conf.d/default.conf
nginx -g 'daemon off;'
... 

7. Create nginx dockerfile

vi Dockerfile
...
FROM nginx/nginx-unprivileged:1-alpine
LABEL maintainer="maintainer@ha-go.com"

COPY ./default.conf.tpl /etc/nginx/default.conf.tpl
COPY ./uwsgi_params /etc/nginx/uwsgi-params

ENV LISTEN_PORT=8000
ENV APP_HOST=app
ENV APP_PORT=9000

USER root

RUN mkdir -p /vol/static
RUN chmod 755 /vol/static
RUN touch /etc/nginx/conf.d/default.conf
RUN chown nginx:nginx /etc/nginx/conf.d/default.conf

COPY ./entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

USER nginx

CMD ["/entrypoint.sh"]
...

8. Setup gitlab ci/cd pipeline build job

vi .gitlab-ci.yml
...
image: docker:19.03.5
services:
  - docker:19.03.5-dind

stages:
  - Build
  - Push

Build:
  stage: Build
  before_scripts: []
  scripts:
    - mkdir data/
    - docker build --compress -t proxy .
    - docker save --output data/image.tar proxy
  artifacts:
    name: image
    paths:
      - data/
...

9. Setup gitlab ci/cd pipeline push jobs

vi .gitlab-ci.yml
...
image: docker:19.03.5
services:
  - docker:19.03.5-dind
  
stages:
  - Build
  - Push
  
before_script:
   - apk add python3
   - pip3 install awscli==1.18.8
   - docker load --input data/image.tar
   - $(aws ecr get-login --no-include-email --region us-east-1)
   
Build:
  stage: Build
  before_script: []
  script:
    - mkdir data/
    - docker build --compress -t proxy .
    - docker save --output data/image.tar proxy
  artifacts:
    name: image
    paths:
      - data/
      
Push Dev:
  stage: Push
  script:
   - docker tag proxy:latest $ECR_REPO:dev
   - docker push $ECR_REPO:dev
  rules:
   - if: "$CI_COMMIT_BRANCH == 'main'"
   
Push Release:
  stage: Push
  script:
   - export TAGGED_ECR_REPO=$ECR_REPO:$(echo $CI_COMMIT_TAG | sed 's/-release//')
   - docker tag proxy:latest $TAGGED_ECR_REPO
   - docker push $TAGGED_ECR_REPO
   - docker tag $TAGGED_ECR_REPO $ECR_REPO:latest
   - docker push $ECR_REPO:latest
  rules:
     - if: "$CI_COMMIT_TAG =~ /^*-release/"
...

10. Test proxy pipeline

• Create new branch 
• Create new tag

Setup project app for deployment

1. Create new project on gitlab
2. Fork this repo https://gitlab.com/LondonAppDev/recipe-app-api-devops-starting-code
3. Configure gitlab project

Select project > Settings > General >Visibility, project features, permissions > CI/CD > Only Project Members > Save changes.

Select project > Settings > CI/CD > General pipelines > uncheck Public pipelines > Save changes

• add uWSGI server to project
  

  cd scripts/
  vi entrypoint.sh
  ...
  #!/bin/sh
  
  set -epython manage.py collectstatic --noinput
  python manage.py wait_for_db
  python manage.py migrate
  
  uwsgi --socket :9000 --workers 4 --master --enable-threads --module app.wsgi
  ...

  
  

• update Dockerfile to run entrypoint
  

  vi Dockerfile
  ...
  FROM python:3.7-alpine
  LABEL maintainer="ha-go"
  
  ENV PYTHONUNBUFFERED 1
  ENV PATH="/scripts:${PATH}"
  RUN pip install --upgrade pip
  
  COPY ./requirements.txt /requirements.txt
  
  RUN apk add --update --no-cache postgresql-client jpeg-dev
  RUN apk add --update --no-cache --virtual .tmp-build-deps \
        gcc libc-dev linux-headers postgresql-dev musl-dev zlib zlib-dev
  RUN pip install -r /requirements.txt
  RUN apk del .tmp-build-deps
  
  RUN mkdir /app
  
  WORKDIR /app
  COPY ./app /app
  COPY ./scripts /scripts
  
  RUN chmod +x /scripts/*
  RUN mkdir -p /vol/web/media
  RUN mkdir -p /vol/web/static
  
  RUN adduser -D user
  
  RUN chown -R user:user /vol/
  RUN chmod -R 755 /vol/web
  
  USER user
  
  VOLUME /vol/web
  
  CMD ["entrypoint.sh"]
  ...

  vi .dockerignore
  ...
  docker-compose.yml
  README.md
  .git/
  .gitignore
  vol/
  deploy/
  ...

  docker build -t . 

• configure static and media files in app settings
  

  vi app/app/settings.py
  ...
  STATIC_URL = '/static/static/'
  MEDIA_URL = '/static/media/'
  ... 

• setup environment variable configuration
  

  vi app/app/settings.py
  ...
  SECRET_KEY = os.environ.get('DJANGO_SECRET_KEY', 'changeme')
  DEBUG = bool(int(os.environ.get('DEBUG', 0)))
  ALLOWED_HOSTS.extend(
      filter(
          None,
          os.environ.get('ALLOWED_HOSTS', '').split(',')
      )
  )
  ... 

• Add dubug mode on docker-compose
  

  vi docker-compose.yml
  ...
  version: "3"
  
  services:
    app:
      build:
        context: .
      ports:
        - "8000:8000"
      volumes:
        - ./app:/app
      command: >
        sh -c "python manage.py wait_for_db &&
               python manage.py migrate &&
               python manage.py runserver 0.0.0.0:8000"
      environment:
        - DB_HOST=db
        - DB_NAME=app
        - DB_USER=postgres
        - DB_PASS=supersecretpassword
        - DEBUG=1
      depends_on:
        - dbdb:
      image: postgres:10-alpine
      environment:
        - POSTGRES_DB=app
        - POSTGRES_USER=postgres
        - POSTGRES_PASSWORD=supersecretpassword
  ...

  docker-compose up

4. Test proxy development locally

vi docker-compose-proxy.yml
...
version: '3.7'

services:
  app:
    build:
      context: .
    volumes:
      - ./app:/app
      - static_data:/vol/web
    environment:
      - DB_HOST=db
      - DB_NAME=app
      - DB_USER=postgres
      - DB_PASS=supersecretpassword
      - ALLOWED_HOSTS=127.0.0.1
    depends_on:
      - db

  proxy:
    image: proxy:latest
    depends_on:
      - app
    ports:
      - "8000:8000"
    volumes:
      - static_data:/vol/static

  db:
    image: postgres:10-alpine
    environment:
      - POSTGRES_DB=app
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=supersecretpassword

volumes:
  static_data:
...

docker-compose -f docker-compose-proxy.yml up 

Testing access browser

localhost:8000

Setup Terraform

1. Install terraform

2. Create S3 bucket

• Create bucket:
  S3 service > create bucket > select region virginia (us-east-1)
  
• Block all public access
• Enable versioning on bucket
  S3 service > Select bucket > properties > versioning > enable versioning > save

3. Create DynamoDB table

• Create table
  DynamoDB service > create table > table name > <your-table-name>-tf-state-lock > Primary key > LockID > create

4. Configure terraform

• Pull repository
• Add .gitignore file
  

  vi .gitignore
  ...
  ### Terraform ###
  # Local .terraform directories
  **/.terraform/*
  
  # .tfstate files
  *.tfstate
  *.tfstate.*
  
  # Crash log files
  crash.log
  
  # Ignore any .tfvars files that are generated automatically for each Terraform run. Most
  # .tfvars files are managed as part of configuration and so should be included in
  # version control.
  #
  # example.tfvars
  .tfvars
  terraform.tfvars
  
  # Ignore override files as they are usually used to override resources locally and so
  # are not checked in
  override.tf
  override.tf.json
  *_override.tf
  *_override.tf.json
  
  # Include override files you do wish to add to version control using negated pattern
  # !example_override.tf
  
  # Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan
  # example: *tfplan*
  
  # End of https://www.gitignore.io/api/python,terraform
  .vscode 

• Add terraform file
  

  $ mkdir deploy
  $ vi deploy/main.tf
  ...
  terraform {
     backend "s3" {
        bucket = "aha-recipe-app-api-devops-tfstate"
        key = "recipe-app.state"
        region = "us-east-1"
        encrypt = true
        dynamodb_table = "recipe-app-api-devops-tf-state-lock"
     }
  }
  
  provider "aws" {
      region = "us-east-1"
      version = "~>3.72.0"
  }

5. Setup docker compose for running terraform6

•  Add docker-compose
  

  $ vi deploy/docker-compose.yml
  version: '3.7'
  
  services:
    terraform:
      image: hashicorp/terraform:1.1.3
      volumes:
        - .:/infra
      working_dir: /infra
      environment:
        - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
        - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
        - AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN}

•  Initializa terraform
  

  $ ./aws-vault-linux-amd64 exec aha-devops --duration=12h
  Enter token for arn:aws:iam::618274166637:mfa/ha-go: 792021
  $ docker-compose deploy/docker-compose.yml run --rm terraform init

6. Create terraform workspace

$ docker-compose deploy/docker-compose.yml run --rm terraform workspace list
$ docker-compose deploy/docker-compose.yml run --rm terraform workspace new dev
$ docker-compose deploy/docker-compose.yml run --rm terraform workspace list

References:  

https://github.com/99designs/aws-vault

https://aws.amazon.com/ecr/

https://docs.gitlab.com/ee/ci/variables/

https://developer.hashicorp.com/terraform/language/settings/backends/s3 

https://www.udemy.com/course/devops-deployment-automation-terraform-aws-docker

Code: 

https://gitlab.com/ahakimx/recipe-app-api-devops