This article is the first part of a series which will show you how to deploy your GitLab CI pipeline artifacts using Ansible. GitLab CI is the Continuous Integration solution shipped with GitLab which builds your projects automatically. Combining it with Ansible, a simple, yet powerful provisioning/deployment automation solution, will give you easy-to-use Continuous Delivery. We will start with the big picture of all involved physical and technological machinery, come up with a plan on how to add proper Ansible support to all your GitLab projects and walk you through a step-by-step implementation of the plan. The first part of the article describes how GitLab CI & Ansible work and is relevant for any GitLab user who wants to understand how an Ansible deployment fits into the GitLab toolchain, while the second part of describes the nitty-gritty details on how to add Ansible deployment support to your GitLab Community Edition hosted repositories.

The next article in this blog series will show how to use Ansible inside a GitLab CI pipeline to automatically deploy a Spring Boot application. The setup of GitLab itself and GitLab CI/Runner won’t be covered.

From git push to running code

Automated deployment of your code is something you don’t want to miss after you experienced it first-hand. Being able to immediately show new features to stakeholders or letting testers verify bugfixes upon every commit tightens the feedback loop massively, leading to better software in less time. So what is involved to get there?

Figure 1: Continuous Delivery
Figure 1: Continuous Delivery

Figure 1 shows the basic idea we want to implement. Upon a push to the GitLab server, a series of jobs, called a pipeline, is triggered, which creates artifacts, tests them and publishes them into some sort of target repository. The target could be a Maven repository, a Docker registry or GitLab itself. The deployment step will download the artifacts onto a staging server and restart the application making it available to the non-coders involved in your project. In case you are wondering, the pipeline will be extended with a production environment in the next installment of this series.

A typical GitLab CI installation is split into a GitLab server and a GitLab Runner host, leading to a server landscape as shown in Figure 2:

Figure 2: GitLab Continuous Delivery
Figure 2: GitLab Continuous Delivery

After a git push, a CI pipeline is triggered by GitLab, and the GitLab Runner executes the CI tasks defined in the .gitlab-ci.yml of our project. With the setup described in this article the deployment will be as simple as adding these lines to your .gitlab-ci.yml:

deploy_app:
  stage: deploy
  tags:
    - ansible
  script:
    - 'ansible-playbook -i staging deploy.yml'

Deploying through GitLab using Ansible

Ansible executes playbooks which contain a list of tasks, for instance “download this file” and “restart that service”, against one or many target hosts, in our case the Staging server. As we want to invoke Ansible as part of our pipeline we have to understand how the steps of a pipeline are executed by the GitLab Runner:

GitLab allows you to add multiple different runners that can be specialised for different pipeline tasks by providing additional machinery, like databases, virtual machines or direct access to the host. A runner can be attached to a single project or shared across all your GitLab projects. To enable Ansible support for all projects we will create a shared runner that uses a Docker executor.

Figure 3: Running Ansible inside Docker
Figure 3: Running Ansible inside Docker

The executor provides an environment in which the runner executes the tasks of a pipeline step. With the docker executor you can specify a docker image in which to run the Ansible deployment. The image can be pulled in from the official Docker hub registry, which contains several ready-made Ansible images as shown in Figure 3. The shared runner can additionally be tagged, which allows us to limit it to only execute jobs with the ansible tag.

To connect to the target machine, Ansible uses SSH.

Figure 4: SSH key-based authentication
Figure 4: SSH key-based authentication

This means that the GitLab Runner host needs an SSH key to authenticate itself against the Staging server, and that key additionally needs to be authorized for an ideally non-root deploy user as shown in Figure 4. Additionally, the secret SSH key has to be made available inside the Ansible container executed on the GitLab Runner host.

Figure 5: Mounting the SSH key into the Ansible container
Figure 5: Mounting the SSH key into the Ansible container

A common way to make secrets available to containers is using a volume as shown in Figure 5. The volume is mounted into the container at runtime, which allows Ansible to use the SSH key. Volumes can be added to the executor through the GitLab Runner configuration.

Another approach would be using GitLab’s Secure Variables to store the private key and configure the GitLab Runner with a pre_build_script that adds the SSH key to the ssh-agent. That way SSH keys would be configured per project, making it theoretically more secure and allowing users to define new deploy targets without needing access to the GitLab Runner. An open question with this approach is how to automatically create the secret variable holding the SSH key. Therefore, we chose the former approach.

The plan

Our plan for adding Ansible support to GitLab pipelines has the following steps:

  1. Add an SSH key to the GitLab Runner host.
  2. Authorize the key for accessing the Staging server with a deploy user.
  3. Create a new GitLab Runner based on an Ansible docker image and configure it with a volume mounted to the directory containing the SSH key.
  4. Tag the new runner inside GitLab with an ansible tag and configure it to only run Ansible-tagged jobs.

Implementing the plan

Step ​1 and 2 are typical provisioning tasks which Ansible excels at. Instead of creating keys and users manually, we can create a short playbook that will create the key, fetch the public part of it and add it to the authorized keys of the deploy target:

- hosts: gitlab_runner
  …
  - tasks:
   - name: create ssh key if it does not exist
      expect:
        command: ssh-keygen -t rsa
        # only creates the key if the file does not exist
        creates: "{{ runner_user_home }}/.ssh/id_rsa"
        ...
        responses:
          "file": "{{ runner_user_home }}/.ssh/id_rsa"
          "passphrase": ""
   - name: read public key
      command: "cat {{ runner_user_home }}/.ssh/id_rsa.pub"
      register: runner_pub_key


- hosts: deploy_target
  …
  - tasks:
   - name: add deploy key to authorized keys
      authorized_key:
        user: "{{ user }}"
        key: "{{ hostvars[ deploy_source_host ].runner_pub_key.stdout}}"

Don’t bother with the details too much. The important take-away is that this can be easily scripted and the full playbook can be found here. By automating this setup we can add additional deploy targets later on and re-run the playbook. Ansible makes sure that all tasks are idempotent, so only the added hosts will be updated. With a few additions it is also possible to automatically replace the SSH key with a new key and de-authorize the old key from time to time.

Step 3 of our plan has to be performed on the GitLab Runner host. Register a new runner via the gitlab-ci-multi-runner shell program. You need the registration token from the GitLab Admin Area -> Overview -> Runners tab:

sudo ./gitlab-ci-multi-runner register \ 
  --non-interactive \
  --executor docker \ 
  --url <gitlab-url> \ 
  --name deploy-ansible-runner \ 
  --registration-token <registration-token> \ 
  --docker-image williamyeh/ansible:centos7 \ 
  --docker-privileged false \ 
  --docker-volumes "/home/<deploy-user>/.ssh:/root/.ssh" \ 
  --tag-list Ansible

Check the generated /etc/gitlab-runner/config.toml that all the values are correct. After registration you can just edit the config and save it. GitLab will pick up the changes automatically.

[[runners]]
  name = "deploy-ansible-runner"
  url = <gitlab-url>
  token = <deploy-token>
  executor = "docker"
  environment = ["GIT_SSL_NO_VERIFY=1"]              (1)
  [runners.docker]
    image = williamyeh/ansible:centos7
    privileged = false
    volumes = ["/cache",
               "/home/<deploy-user>/.ssh:/root/.ssh" (2)
    ]

(1) In case your GitLab installation uses a self-signed SSL certificate, prevent git from refusing to checkout your repository.

(2) Mount the .ssh directory to /root/.ssh as that is the default location for the key, making Ansible pick it up automatically.

Step 4 of our plan requires only a few clicks. Select the ansible-deploy-runner in the GitLab Admin Area -> Overview -> Runners tab, verify it has an ansible tag and uncheck the Run untagged jobs checkbox. Otherwise it will run unrelated builds which might expect a different environment.

Continuous Integration!

That’s all that is needed to prepare a GitLab CI installation for Ansible deployments. From now on all your GitLab projects can use the Ansible runner to automatically deploy their latest changes by simply specifying the ansible tag in their .gitlab-ci.yml and providing an Ansible playbook. We will take a look at a complete pipeline built this way, including a staging and production environment, in the next blogpost to be published here soon.