How to create a CI/CD for a Laravel application using GitHub Actions

Introduction

GitHub Actions was launched less than a year ago and since then has been receiving numerous positive remarks. It allows us to have the important things about our project integration and deployment in a central place. In other words, your code can be built, tested, and deployed from GitHub without using external CI/CD services to do the job for you.

In this article, we will be exploring a hands-on approach to managing your CI/CD processes using GitHub Actions.

Prerequisites

• A GitHub account. If you don’t have one, you can sign up here
• A server with SSH access
• Basic knowledge of writing valid YAML
• Basic knowledge of GitHub and Git

Our plan

In this article, we will be covering the major parts that will form a basic CI/CD setup for our demo application. Our CI/CD setup will monitor pushes and pull requests made to our repository. We want to be able to:

• Run tests on pushes
• Run tests on pull requestDeploy when pushes are made to specific branches
• Deploy to staging server when push is on staging branch
• Deploy to production server when push is on master branchDeploy when a release is tagged

The sample project

To make this article focused and swift, I have created a sample Laravel project that will be used in this article. The project contains some tests, both frontend tests, and backend tests to follow appropriately. To clone the project run:

git clone https://github.com/ichtrojan/deploy_tut

What is a workflow?

A workflow defines the steps required to complete a CI/CD process. According to GitHub’s documentation:

Workflows are custom automated processes that you can set up in your repository to build, test, package, release, or deploy any project on Github

A workflow is defined within the repository and committed as part of the repository. When you commit a workflow and push to GitHub, GitHub Actions will automatically detect the workflow and immediately parse the workflow and start processing your CI/CD process based on the instruction defined there. Workflows are written with YAML and stored inside .github/workflows directory of your project root.

Configuring workflows

Run tests on a pull request workflow

This workflow is very useful when a large team is collaborating on the same project. You would want to run some checks whenever a pull request is created and not wait until it is merged before testing it. For this, we will define a workflow that will run when a pull request is made. Here is what that kind of workflow will look like:

name: PR WorkFlow

on:
  pull_request:
    branches:
      - master
      - staging

jobs:
  app-tests:
    runs-on: ubuntu-latest
    services:
      mysql:
        image: mysql:5.7
        env:
          MYSQL_ALLOW_EMPTY_PASSWORD: yes
          MYSQL_DATABASE: test_db
        ports:
          - 3306
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
    steps:
      - uses: actions/checkout@v1
      - name: Copy .env
        run: php -r "file_exists('.env') || copy('.env.example', '.env');"
      - name: Install Composer Dependencies
        run: composer install -q --no-ansi --no-interaction --no-scripts --no-suggest --no-progress --prefer-dist
      - name: Install NPM Dependencies
        run: npm install
      - name: Generate key
        run: php artisan key:generate
      - name: Execute tests (Unit and Feature tests) using PHPUnit
        env:
          DB_PORT: ${{ job.services.mysql.ports[3306] }}
        run: ./vendor/bin/phpunit
      - name: Execute tests (Unit and Feature tests) using JEST
        run: node_modules/.bin/jest

Note:
The above should be saved in the .github/workflows directory as pr_workflow.yml.

name: Pull Request WorkFlow

on:
  pull_request:
    branches:
      - master
      - staging

The initial part of the YAML file above defines the name of the workflow and then tells GitHub Actions to run when a pull request is made to the master and staging branch.

We defined a job we called app-tests whose purpose is to run both tests using Jest and PHPUnit. We are telling Github Actions to include MySQL as a service when setting up the action. Finally, the other part marked by steps lists out the steps to be performed by the job.

Run tests on a push workflow

In most cases, you will want to ensure your project is fine whenever a change is made to the source code. This is usually done using tests that ensure no part of the project is broken because of any change. To do this, here is what our initial workflow will look like:

name: PUSH Workflow

on:
  push:
    branches:
      - master
      - staging

jobs:
  app-tests:
    runs-on: ubuntu-latest
    services:
      mysql:
        image: mysql:5.7
        env:
          MYSQL_ALLOW_EMPTY_PASSWORD: yes
          MYSQL_DATABASE: test_db
        ports:
          - 3306
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
    steps:
      - uses: actions/checkout@v1
      - name: Copy .env
        run: php -r "file_exists('.env') || copy('.env.example', '.env');"
      - name: Install Composer Dependencies
        run: composer install -q --no-ansi --no-interaction --no-scripts --no-suggest --no-progress --prefer-dist
      - name: Install NPM Dependencies
        run: npm install
      - name: Generate Key
        run: php artisan key:generate
      - name: Execute tests (Unit and Feature tests) via PHPUnit
        env:
          DB_PORT: ${{ job.services.mysql.ports[3306] }}
        run: vendor/bin/phpunit
      - name: Execute tests (Unit and Feature tests) via JEST
        run: node_modules/.bin/jest
  build-js-production:
    name: Build JavaScript/CSS for PRODUCTION Server
    runs-on: ubuntu-latest
    needs: app-tests
    if: github.ref == 'refs/heads/master'
    steps:
      - uses: actions/checkout@v1
      - name: NPM Build
        run: |
          npm install
          npm run prod
      - name: Put built assets in Artifacts
        uses: actions/upload-artifact@v1
        with:
          name: assets
          path: public
  build-js-staging:
    name: Build JavaScript/CSS for STAGING Server
    runs-on: ubuntu-latest
    needs: app-tests
    if: github.ref == 'refs/heads/staging'
    steps:
      - uses: actions/checkout@v1
      - name: NPM Build
        run: |
          npm install
          npm run dev
      - name: Put built assets in Artifacts
        uses: actions/upload-artifact@v1
        with:
          name: assets
          path: public
  deploy-production:
    name: Deploy Project to PRODUCTION Server
    runs-on: ubuntu-latest
    needs: [build-js-production, app-tests]
    if: github.ref == 'refs/heads/master'
    steps:
      - uses: actions/checkout@v1
      - name: Fetch built assets from Artifacts
        uses: actions/download-artifact@v1
        with:
          name: assets
          path: public
      - name: Setup PHP
        uses: shivammathur/setup-php@master
        with:
          php-version: 7.4
          extension-csv: mbstring, bcmath
      - name: Composer install
        run: composer install -q --no-ansi --no-interaction --no-scripts --no-suggest --no-progress --prefer-dist
      - name: Setup Deployer
        uses: atymic/deployer-php-action@master
        with:
          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
          ssh-known-hosts: ${{ secrets.SSH_KNOWN_HOSTS }}
      - name: Deploy to PRODUCTION Server
        env:
          DOT_ENV: ${{ secrets.DOT_ENV_PRODUCTION }}
        run: dep deploy production --tag=${{ env.GITHUB_REF }} -vvv
  deploy-staging:
    name: Deploy Project to STAGING Server
    runs-on: ubuntu-latest
    needs: [build-js-staging, app-tests]
    if: github.ref == 'refs/heads/staging'
    steps:
      - uses: actions/checkout@v1
      - name: Fetch built assets from Artifacts
        uses: actions/download-artifact@v1
        with:
          name: assets
          path: public
      - name: Setup PHP
        uses: shivammathur/setup-php@master
        with:
          php-version: 7.4
          extension-csv: mbstring, bcmath
      - name: Composer install
        run: composer install -q --no-ansi --no-interaction --no-scripts --no-suggest --no-progress --prefer-dist
      - name: Setup Deployer
        uses: atymic/deployer-php-action@master
        with:
          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
          ssh-known-hosts: ${{ secrets.SSH_KNOWN_HOSTS }}
      - name: Deploy to Prod
        env:
          DOT_ENV: ${{ secrets.DOT_ENV_STAGING }}
        run: dep deploy staging --tag=${{ env.GITHUB_REF }} -vvv

Note:
The above should be saved in the .github/workflows directory as push_workflow.yml

We will ignore the app-tests part of the workflow because it is just similar to the one in the pr_workflow.yml and just focus on the parts that make it different.

The workflow above will run frontend tests with Jest and backend tests with PHPUnit whenever there is a push on the repository, the same way it is done when a pull request is made. The difference here is, we are adding four more jobs to this workflow – build-js-production, build-js-staging, deploy-staging, and deploy-production. We will now explain these four other jobs.

The aim of build-js-production and build-js-staging is to build the JavaScript assets within our project and then upload them to the GitHub Actions artifacts so that they can be available in the deployment job.

A very important thing to look out for is this line– if: github.ref == 'refs/heads/master' which checks to see if the push was made to the master, if this is true, GitHub actions will go ahead and run the steps defined in steps. In build-js-staging, we have if: github.ref == 'refs/heads/staging' instead, which checks if the push was made to the staging branch. In other words, build-js-staging runs if the push was made to the staging branch while build-js-production runs if the push was made to the master branch:

- name: Put built assets in Artifacts
        uses: actions/upload-artifact@v1
        with:
          name: assets
          path: public

In the code shown above, after running npm run dev or npm run build we are uploading all the static assets generated to the GitHub Actions artifacts. We will fetch them later in another job. We are doing this so that we can persist some data from the previous job making it available in another job. It’s important to note that generated data does not persist across jobs.

The same principle of checking which branch a push was made also applies to deploy-staging and deploy-production jobs. Hence, deploy-staging runs when push is made to staging while deploy-production runs when push is made to the master branch. If any of the checks resolve to be true, the steps that follow will be executed.

Another important part of a workflow is needs: [build-js-staging, app-tests] (under deploy-staging) and needs: [build-js-production, app-tests] (under deploy-production) which is telling GitHub Actions that, deploy-staging requires build-js-staging, and app-tests jobs to run successfully for it to run. This means that deploy-staging job will run after both build-js-staging and app-tests runs and exited without any error. This is to ensure that deployment doesn’t happen if the test or asset build fails. The same goes for deploy-production which depends on build-js-production and app-tests for it to run.

name: Fetch built assets from Artifacts
        uses: actions/download-artifact@v1
        with:
          name: assets
          path: public

The above will download the uploaded assets from the artifacts into the public folder of our project:

- name: Composer install
        run: composer install -q --no-ansi --no-interaction --no-scripts --no-suggest --no-progress --prefer-dist

This will install composer dependencies, a step that is required for us to run Deployer later:

- name: Setup PHP
        uses: shivammathur/setup-php@master
        with:
          php-version: 7.4
          extension-csv: mbstring, bcmath

This part of the workflow uses another GitHub action identified by shivammathur/setup-php@master to setup the PHP Version to use and the extensions we need:

- name: Setup Deployer
        uses: atymic/deployer-php-action@master
        with:
          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
          ssh-known-hosts: ${{ secrets.SSH_KNOWN_HOSTS }}

This step sets up Deployer using another GitHub action identified by atymic/deployer-php-action@master and is being used with the SSH_PRIVATE_KEY and SSH_KNOWN_HOSTS which we would obtain from the server later.

- name: Deploy to Prod
        env:
          DOT_ENV: ${{ secrets.DOT_ENV_STAGING }}
        run: dep deploy staging --tag=${{ env.GITHUB_REF }} -vvv

- name: Deploy to Prod
        env:
          DOT_ENV: ${{ secrets.DOT_ENV_PRODUCTION }}
        run: dep deploy staging --tag=${{ env.GITHUB_REF }} -vvv

Finally, the last step initiates the deploy process, from there, Deployer will upload the project to the server. The difference between the two parts is secrets.DOT_ENV_PRODUCTION and secrets.DOT_ENV_STAGING which allows us to inject different ENVIRONMENT VARIABLES for different deployments. In our case, we are using different ENVIRONMENT VARIABLES for production and staging.

Obtaining server credentials

The aim here is to give GitHub actions access to our server to deploy the changes, you will obtain the private key for your server and add it in the secret. To do this, you will need to generate an SSH key if you haven’t done that already, log in to your server and run:

ssh-keygen

There will be some prompts and you can accept the default values. Ensure there is no passphrase given to your SSH key during the prompt setup. If you already have your SSH Key generated, this step can be skipped. After running through the prompts, you should have something like this:

Two files named id_rsa.pub and id_rsa will be generated for you in the ~/.ssh directory. Next, copy the content of id_rsa, not id_rsa.pub. id_rsa.pub contains your public key while id_rsa contains your private key:

cat ~/.ssh/id_rsa

You need to add the public key generated by SSH keygen to the authorized_keys so that any connection attempted using the private key will be allowed. This is done by running this command:

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

The command above will produce no output so do not worry if nothing happens after running the command. Next, you will head over to GitHub, go to the Settings tab of the project. Then click on Secrets as shown in the image below:

You need to give a name to the secret that corresponds to what has been defined in the Workflow i.e SSH_PRIVATE_KEY and then put the content you copied as the value like so:

After adding the secret, we will have a view looking like this:

Obtain the SSH Known Hosts of the server and also add it to the Secrets section. From your local machine you have to run to get the ssh known hosts:

ssh-keyscan rsa -t {server_ip_address}

Note:
Replace {server_ip_address} with your server public IP address

Copy the output of this command and add it to the secrets with the name SSH_KNOWN_HOSTSand the content will be copied.

Note:
The part to copy begins with {server_ip_address} sh-rsa

Observe that, not all the contents, in this case, were copied. Only the part highlighted was copied. After adding it to Github Secrets we will have something like this:

To achieve our plan of deploying to different server appropriately, you will need to repeat this process for the staging server and the production Server or else you will need to give appropriate names to the secrets you’re adding to the repository. You must ensure that this naming takes effect in the workflow written above since you’re using different servers for your production and staging environments.

Add application environment files

Next, we need to add the environment variables in the secret as in the previous section. The technique here is to inject the environment variables when the action is running. To do this, we have to compile the environment variables needed by the app on the server into a .env file.

Then we will copy the content of this file the add to secrets with the name:

`DOT_ENV_STAGING` and `DOT_ENV_PRODUCTION`

Set up Deployer

To initiate our deployment, we will be using Deployer. It is a deployment tool written in PHP with support for popular frameworks out of the box. To use this, we need to set it up locally in our project and we do that by running:

composer require deployer/deployer deployer/recipes

The package deployer/deployer is the main Deployer project while deployer/recipes includes components that will help us configure Deployer for specific projects and tools like Laravel, Symfony, RSync, and others.

Next, configure Deployer, we will do this by creating a file named deploy.php at the root of the project:

<?php

namespace Deployer;

require 'recipe/laravel.php';
require 'recipe/rsync.php';

set('application', 'My App');
set('ssh_multiplexing', true);

set('rsync_src', function () {
    return __DIR__;
});


add('rsync', [
    'exclude' => [
        '.git',
        '/.env',
        '/storage/',
        '/vendor/',
        '/node_modules/',
        '.github',
        'deploy.php',
    ],
]);

task('deploy:secrets', function () {
    file_put_contents(__DIR__ . '/.env', getenv('DOT_ENV'));
    upload('.env', get('deploy_path') . '/shared');
});

host('myapp.io')
  ->hostname('104.248.172.220')
  ->stage('production')
  ->user('root')
  ->set('deploy_path', '/var/www/my-app');

host('staging.myapp.io')
  ->hostname('104.248.172.220')
  ->stage('staging')
  ->user('root')
  ->set('deploy_path', '/var/www/my-app-staging');

after('deploy:failed', 'deploy:unlock');

desc('Deploy the application');

task('deploy', [
    'deploy:info',
    'deploy:prepare',
    'deploy:lock',
    'deploy:release',
    'rsync',
    'deploy:secrets',
    'deploy:shared',
    'deploy:vendors',
    'deploy:writable',
    'artisan:storage:link',
    'artisan:view:cache',
    'artisan:config:cache',
    'artisan:migrate',
    'artisan:queue:restart',
    'deploy:symlink',
    'deploy:unlock',
    'cleanup',
]);

Most lines in the code above are self-explanatory. However, I will go ahead and explain some parts of the code:

<?php

...

require 'recipe/laravel.php';
require 'recipe/rsync.php';

This part includes two recipes from deployer/recipes, the first one for Laravel and then the other for RSync. 'recipe/laravel.php' allows us to use some predefined Laravel specific tasks while 'recipe/rsync.php' allows us to configure our RSync easily since we will be using it to copy the files into the server:

<?php

...

set('rsync_src', function () {
    return __DIR__;
});


add('rsync', [
    'exclude' => [
        '.git',
        '/.env',
        '/storage/',
        '/vendor/',
        '/node_modules/',
        '.github',
        'deploy.php',
    ],
]);

This part configures RSync, it defines the directory we would be copying files from and it also defines the directories and files that should be excluded when copying in the exclude array key:

<?php

...

task('deploy:secrets', function () {
    file_put_contents(__DIR__ . '/.env', getenv('DOT_ENV'));
    upload('.env', get('deploy_path') . '/shared');
});

host('myapp.io')
  ->hostname('104.248.172.220')
  ->stage('production')
  ->user('root')
  ->set('deploy_path', '/var/www/my-app');

host('staging.myapp.io')
  ->hostname('104.248.172.220')
  ->stage('staging')
  ->user('root')
  ->set('deploy_path', '/var/www/my-app-staging');

The first code block above defines a task that copies the content of DOT_ENV as configured in the workflow and puts it in the shared directory in the deploy directory. The remaining parts of the snippet are identical but with slight differences. They both configure the staging and production environment and some other things like deploy path, IP, etc, which are specific to either environment:

<?php

...

after('deploy:failed', 'deploy:unlock');

desc('Deploy the application');

task('deploy', [
    'deploy:info',
    'deploy:prepare',
    'deploy:lock',
    'deploy:release',
    'rsync',
    'deploy:secrets',
    'deploy:shared',
    'deploy:vendors',
    'deploy:writable',
    'artisan:storage:link',
    'artisan:view:cache',
    'artisan:config:cache',
    'artisan:migrate',
    'artisan:queue:restart',
    'deploy:symlink',
    'deploy:unlock',
    'cleanup',
]);

This code section above defines tasks, defined in the order in which they should be performed for the deploy process. Some of these tasks include Laravel specific tasks, RSync, composer tasks, etc.

You can read the official documentation of Deployer here.

Once this file has been created with the content above, commit changes, and then push the changes to the master branch on GitHub. Go to GitHub and proceed to your project, you can click on the Actions tab to monitor the running actions, and once you do click on it, you will see that your push has triggered the workflow and the process has started already:

Clicking on the workflow will show more details about the status of the workflow like this:

The process will run in stages as defined in the workflowYAML file, with that you will see a screen like this result:

We only handled continuous integration and deployment in this article. The part that has to do with getting your website online (like Nginx setup) is not included. The latest version of the deployment will be inside {deploy_path}/current. Where {deploy_path} is the path set inside deploy.php for staging or for production. You can take note of this path if you want to do some additional setup for the path.

You can access the codebase for this project on GitHub.

Conclusion

In this article, you were able to set up a CI/CD process using GitHub Actions. Many things can be built on this foundation as it forms a basic setup to automate your integrations and deployments for your new and existing projects.