User story

As a member of DevOps team, I want to stand up DevOps tools (Platform as Code) so that projects can run Continuous Integration / Continuous Delivery (CI / CD) pipelines:

  • Flow Proxy - The goal of the Docker Flow Proxy project is to provide an easy way to reconfigure proxy every time a new service is deployed, or when a service is scaled. It does not try to "reinvent the wheel", but to leverage the existing leaders and combine them through an easy to use integration. It uses HAProxy as a proxy and adds custom logic that allows on-demand reconfiguration.
  • Flow Swarm Listener - The goal of the Docker Flow Swarm Listener project is to listen to Docker Swarm events and send requests when a change occurs. At the moment, the only supported option is to send a notification when a new service is created, or an existing service was removed from the cluster.
  • Sonatype Nexus Repository Manager 3 - Based on CentOS, a free binary repository manager with universal support for popular repository formats such as maven, yum, raw, docker and many other
  • SonarQube - SonarQube provides the capability to not only show health of an application but also to highlight issues newly introduced. With a Quality Gate in place, you can fix the leak and therefore improve code quality systematically
  • Jenkins - As an extensible automation server, Jenkins can be used as a simple CI server or turned into the continuous delivery hub for any project
  • GitLab Community Edition (CE) - Gitlab is an open source end-to-end software development platform with built-in version control, issue tracking, code review, CI/CD, and more. Self-host GitLab CE on your own servers

The architecture of this stack is such that all services are behind an HTTP(S) & SSH reverse proxy; single point of entry. This reverse proxy in this case is started with self signed certificate (See ./certs/README.md) using docker secrets.

Prerequisite

Docker swarm mode environment is required

  • Use provided Vagrantfile if you are unable to run Docker CE natively on a local machine.
  • OR see Docker for AWS documentation on how to create a Docker swarm cluster on AWS.

Deploy CI stack in a VirtualBox with provided Vagrantfile

The assumption here is that Vagrant, VirtualBox and Gitbash are already install on your machine (my development environment was a Windows 10 Pro machine).

Execute the following commands, in gitbash, in order to create a two-node docker swarm mode cluster. The nodes are based on 'ubuntu/xenial64' VM. Once the cluster is created successfully, log in to the master node:

  • git clone https://github.com/shazChaudhry/docker-swarm-mode.git
  • cd docker-swarm-mode
  • vagrant up
  • vagrant ssh node1 (Log in to the master node)
  • docker node ls (confirm that there are two nodes in the cluster; master and worker)
  • cd /vagrant

Deploy stack by running the following commands which will utilize Docker secrets for Jenkins and proxy.

  • Jenkins secrets are defined in the "./secrets/jenkins" directory.
  • Proxy's secrets are defined in the "./certs" directory.
docker stack deploy --compose-file docker-compose.portainer.yml portainer
docker stack deploy --compose-file docker-compose.yml ci
  • Check status of the stack services by running the following command:
docker stack services ci
  • Once all services are up and running, proceed to the next step.

Service URLs

Cloning repositories with HTTPS

  • git config --global http.sslVerify false (Turns off Git SSL Verification for a non trusted server certificate. Otherwise, you may receive 'SSL certificate problem: self signed certificate' error)
  • git clone https://root:Password01@node1/gitlab/[GROUP_NAME]/[REPOSITORY_NAME].git

Cloning repositories with SSH

  • git clone ssh://git@node1:10022/[GROUP_NAME]/[REPOSITORY_NAME].git

Clean-up

On the swarm master node, run the following commands:

  • docker stack rm ci to remove the stack
  • exit to exit the vagrant box
  • vagrant destroy --force to destroy the VMs

Deploy CI stack on "Docker for AWS"

It is assumed you have followed Docker for AWS documentation to create a new VPC. Follow these commands in an ssh client to log in to your master node (I'm using gitbash on Windows 10 Pro).

Please note you can not ssh directly into worker nodes. You have to use a manager node as a jump box.

  • eval $(ssh-agent) OR exec ssh-agent bash
  • ssh-add -k ~/.ssh/personal.pem (You will have to use your own key).
  • ssh -A docker@<Manager Public IP>

Clone this repo and change directory by following these commands:

  • git clone https://github.com/shazChaudhry/docker-swarm-mode.git
  • sudo chown -R $USER:$USER docker-swarm-mode
  • cd docker-swarm-mode

Start the Portainer by running:

  • docker stack deploy -c docker-compose.portainer.yml portainer

Run the combined stack. Please note that secrets are defined in ./secrets/jenkins and ./certs directories in this repo:

  • docker stack deploy --compose-file docker-stack.yml ci

Configuring your client for interacting with docker repos in Nexus

  • You will need to look up docker daemon documentation for your system. Create a file /etc/docker/daemon.json and copy the following content in that file
{
"insecure-registries": [
    "[DefaultDNSTarget]:443",
    "[DefaultDNSTarget]:5000"
],
"disable-legacy-registry": true
}

  • You have to restart the daemon after setting this sudo systemctl restart docker

  • docker login -u admin -p admin123 [DefaultDNSTarget]:443

Setup Jenkins declarative pipeline

Steps to setting up this pipeline should be identical regardless of the deployment methods shown above; Local or AWS

  • In the GitLab instance, import spring-petclinic project from GitHub.
  • In the Jenkins instance, create a new Blue Ocean pipeline project.
  • Once "Deploy Key" has been created, head over to the "spring-petclinic" project in GitLab and ensure that you enable the deploy key at the bottom of the page.

About

Setting up a Docker based CI environment. Tools include GitLap, Jenkins, Sonarqube and Nexus.

Topics

[jenkins](/content/topics/jenkins "Topic: jenkins"/index.html) [devops](/content/topics/devops "Topic: devops"/index.html) [gitlab](/content/topics/gitlab "Topic: gitlab"/index.html) [continuous-integration](/content/topics/continuous-integration "Topic: continuous-integration"/index.html) [continuous-delivery](/content/topics/continuous-delivery "Topic: continuous-delivery"/index.html) [sonarqube](/content/topics/sonarqube "Topic: sonarqube"/index.html) [docker-swarm-mode](/content/topics/docker-swarm-mode "Topic: docker-swarm-mode"/index.html) [jenkins-pipeline](/content/topics/jenkins-pipeline "Topic: jenkins-pipeline"/index.html) [nexus3](/content/topics/nexus3 "Topic: nexus3"/index.html)