How to set up a Django CI/CD pipeline on AWS

What is Continuous Integration and Continuous Deployment (CI/CD)?

CI/CD is the automation of the software development life cycle from development to production. It is the practice that ensures the production of efficient and tested code.

There are defined processes involved in CI/CD which are:

1. Code
2. Build
3. Test
4. Deploy

These steps are followed whenever you are implementing CI/CD into your development workflow.

Why is CI/CD Important

Implementing CI/CD into your workflow can be very beneficial, it comes with many advantages, which will be listed below.

• Faster code delivery
• Reduction to no manual deployments
• Less to no errors in your production code
• Faster code delivery
• Reduction in cost

Building a CI/CD django pipeline in AWS

In this section, we are going over how to set up a CI/CD pipeline in AWS for a django project.

To follow up with this tutorial, you will need to meet up with the prerequisites:

• An AWS account

I have a Github repository you can use as a guide for this tutorial. Without wasting more of your time let’s get started.

Create a New IAM Role

Log in to your AWS account, the first thing we have to create is an IAM role, this role will help assign some permissions to some services such as CodeDeploy and S3 access for our EC2 which we will need in the future.

In the search bar, type in IAM, and click on the first option.

On the IAM home page, look at the left pane, click on Roles

On the Roles home page, click on Create Role.

Now, we should be on the Create Role page, on the Trusted entity type, select AWS service.

For the use case option, select EC2.

Click on the Next button to go to the next page.

Here we need to add the permissions for our Role, search for codedeploy, and select the AmazonEC2RoleForAWSCodeDeployLimited permission.

Next, search for s3, and select the AmazonS3FullAccess permission.

Click on Next to move to the final page, here we will name our new IAM role, and leave every other option as default.

Create an EC2 instance

The next thing we need to do is create our ec2 instance.

It is important to use the Ubuntu 20.04 or Ubuntu 20.10 image when spinning up our instance.

This is because they are part of the image types that currently support the codedeploy-agent package

The codedeploy-agent will help automate the deployment configurations on our server.

In the search bar, search for ec2, click on the first option.

On the ec2 home page, on the left pane, click on EC2 Dashboard, while in the dashboard, click on the Launch Instance button.

Now in the launch instance configurations page, give your instance a name. In our case, we are naming it django_server.

In the Application and Os images, make sure to select Ubuntu server 20.04 Free tier option

Leave the default configurations on the Instance Type option. On the Key pair (login) option, make sure to create a new .pem key. We will use this key to log into our ubuntu later on.

In the Network settings option check the Allow HTTP traffic from the internet option

Next, click on the Launch instance button. Our instance will be provisioned immediately.

Now we will need to attach our newly created IAM role to our instance. On the instance home page, we will select our instance and click on the Actions button in the top right corner.

Select the Security option, and click on Modify IAM Role. Then, select the ec2-codedeploy-s3 role from the dropdown.

Finally, click on the Update IAM role button.

Next, we need to reboot our instance for the changes to take effect.

Installing Codedeploy-Agent on Our Server

With our instance provisioned, we will now install the codedeploy-agent on our server.

First, we need to log into our server. There are guides on how to do this on the Connect page, click on the Connect button in the top right corner of the instance page.

When you have logged into your server, run this command to update the repositories.

sudo apt update

‍

Next, we need to install the ruby-full package:

sudo apt install ruby-full

‍

Finally, we need to install the wget package:

sudo apt install wget

‍

After installing the necessary packages, we then need to change our directory to /home/ubuntu like so:

cd /home/ubuntu

Now in the ubuntu directory, we need to enter this command:

wget https://bucket-name.s3.region-identifier.amazonaws.com/latest/install

In this command, the bucket-name is the s3 bucket that contains the CodeDeploy installation files for your region. The region-identifier is simply your region.

For example, for the US-East-1 region (N.Virginia), replace bucket-name with aws-codedeploy-us-east-1, and region-identifier with us-east-1.

You can get a list of CodeDeploy bucket-names and region-identifiers here.

Next up, we need to change the permission on the install file we will get after running the command above.

chmod +x ./install

Finally, to install the codedeploy-agent, run this command:

sudo ./install auto > /tmp/logfile

Here we are logging the output of the installation to the /tmp/logfile file. To check if the codedeploy-agent is running, enter this command:

sudo service codedeploy-agent status

If it is not running, enter this command to start the codedeploy-agent service:

sudo service codedeploy-agent status

Setting up Configuration Files

Now that we have configured our server we will need to start working on our configuration files.

There are two important configuration files we need which are:

• buildspec.yml
• appspec.yml

The buildspec.yml file holds the configurations for how our code is going to be built in AWS CodeBuild, while the appspec.yml file holds the configurations for how our code is going to be deployed in AWS CodeDeploy.

I will not be going into detail on how each configuration step works, but we need to make sure that these files are present in our GitHub repository, in our working root directory. Let’s create our buildspec.yml file.

In our root working directory, create a file called buildspec.yml and add this block of code to it.

version: 0.1 

environment_variables: 
   plaintext: 
     DJANGO_SETTINGS_MODULE: config.settings.test 
     SECRET_KEY: nosecret 
     DATABASE_DEFAULT_URL: sqlite:///db1.sqlite3 
     DATABASE_STREAMDATA_URL: sqlite:///db2.sqlite3 
     OPBEAT_ENABLED: False 
phases: 
  pre_build: 
    commands: 
      - echo Prebuild ops 
      - pip3 install -r requirements.txt 
  build: 
    commands: 
      - echo Building the application 
      - python manage.py runserver 
  post_build: 
    commands: 
      - echo Build completed on `date`

In our buildspec.yml file, we can see that our build consists of 3 phases, the pre_build, build, and post_builds. We define the processes we want to run within these three phases.

Next, we need to create our appspec.yml file.

After creating our appspec.yml file, we need to add this block of code to it.

version: 0.0 
os: linux 
files: 
  - source: / 
    destination: /home/ubuntu/ci-cd-django 
permissions: 
  - object: /home/ubuntu/ci-cd-django 
    owner: ubuntu 
    group: ubuntu 
hooks: 
  BeforeInstall: 
      - location: scripts/clean_instance.sh 
        timeout: 300 
        runas: ubuntu 
  AfterInstall: 
      - location: scripts/instance_os_dependencies.sh 
        timeout: 300 
        runas: ubuntu 
      - location: scripts/python_dependencies.sh 
        timeout: 300 
        runas: ubuntu 
      - location: scripts/gunicorn.sh 
        timeout: 300 
        runas: ubuntu 
      - location: scripts/nginx.sh 
        timeout: 300 
        runas: ubuntu 
  ApplicationStop: 
      - location: scripts/stop_app.sh 
        timeout: 300 
        runas: ubuntu 
  ApplicationStart: 
       - location: scripts/start_app.sh 
         timeout: 300 
         runas: ubuntu

The appspec.yml file executes the different deployment stages within its hooks, these hooks are essential because they contain paths to configuration files that CodeDeploy uses to automate the deployment of our application.

Creating our CodePipeline

In our AWS management console, in the search bar, search and select for CodePipeline.

You should now be on the CodePipeline home page, look at the left pane, you will see the steps outlined for a ci/cd pipeline.

Now, let’s create our first build. On the left pane of the CodePipeline page, click on build.

We should now be moved to the CodeBuild page where we can create our first build.

Here you can name your build, and give it a description. I named mine django_code_build.

Next, we need to specify the source of our code, i.e. where our code will be pulled from for building. In this section, make sure to choose GitHub.

We will need to connect to our Github account first, after that we can choose which repository we want to pick from with its branch.

In the next section, we will see an option to enable webhooks that will trigger builds depending on the type of action performed on the repository.

Make sure to check this option and select the PUSH trigger.

Next, is the ENVIRONMENT configuration. We need to describe the environment in which are builds and tests are going to be carried out.

Select the options that are selected in the image below.

Next, for the service role, select the New Service Role option.

For the Buildspec section select the use a buildspec file option. This option makes the build process select the buildspec.yml file in our root working directory

If you have your buildspec.yml file within another directory, specify the path of the buildspec.yml file in the field shown.

Next, select the option for No Artifacts in the artifacts section.

Finally, for the logs, if you want to see and store the logs for your builds, select the option to store those logs in CloudWatch, and give the CloudWatch group and stream names

Now we are done with the configuration of our build in CloudBuild, click on the Create build project button.

When we check our build projects, we should see our newly created build project.

Next, we will need to create a CodeDeploy application. This application will handle the CD aspect of our CI/CD pipeline.

On the left pane, click on Deploy and click on Getting Started. We should see the page for creating our first CodeDeploy application.

On the Create application page add your application name and select the EC2/Premise option for the Compute platform.

After creating our deployment application, we will be navigated to the deployment group page, where we will create a new deployment group for our deployment application.

Finally. let us create our CodePipeline.

On the left pane, click on Pipelines, and click on the Create pipeline button.

On the Choose pipelines settings give your pipeline a name, snd leave everything else as default, and click Next.

Next, we need to add the source of our code, we will be selecting Github (Version 1) in this step.

We will need to connect our Github account with AWS CodePipeline.

After connecting our account, we will need to choose the repository and branch that will be used.

Next, we need to specificy our build, we will be using CodeBuild here, select the build project we created above. Leave all other options as default.

Next, we need to add our deployment stage, we will be using CodeDeploy for our deployment stage, select the application name and deployment group we created above.

Afterwards, click on Next, and then click on Create pipeline.

Creating our pipeline automatically triggers a new release, this will clone our repository, build and run any specified tests, then deploy our application to our server.

Conclusion

We are now at the end of the article. In this article, we went over what CI/CD is, discussed the importance of implementing CI/CD into our development workflow, and finally looked at how to set up a CI/CD Django pipeline in AWS.

If you found this article helpful, make sure to share it, and happy coding!