## User story

As a member of DevOps team, I want to stand up DevOps tools _(Platform as Code)_ so that projects can run Continuous Integration / Continuous Delivery (CI / CD) pipelines:

- `Flow Proxy` \- The goal of the [Docker Flow Proxy project](/content/docker-flow/docker-flow-proxy/index.html) is to provide an easy way to reconfigure proxy every time a new service is deployed, or when a service is scaled. It does not try to "reinvent the wheel", but to leverage the existing leaders and combine them through an easy to use integration. It uses HAProxy as a proxy and adds custom logic that allows on-demand reconfiguration.
- `Flow Swarm Listener` \- The goal of the [Docker Flow Swarm Listener project](/content/docker-flow/docker-flow-swarm-listener/index.html) is to listen to Docker Swarm events and send requests when a change occurs. At the moment, the only supported option is to send a notification when a new service is created, or an existing service was removed from the cluster.
- `Sonatype Nexus Repository Manager 3` \- Based on CentOS, a free [binary repository manager](/content/sonatype/docker-nexus3/index.html) with universal support for popular repository formats such as maven, yum, raw, docker and many other
- `SonarQube` \- [SonarQube](/content/SonarSource/sonarqube/index.html) provides the capability to not only show health of an application but also to highlight issues newly introduced. With a Quality Gate in place, you can fix the leak and therefore improve code quality systematically
- `Jenkins` \- As an extensible automation server, [Jenkins](https://hub.docker.com/r/jenkinsci/blueocean/) can be used as a simple CI server or turned into the continuous delivery hub for any project
- `GitLab Community Edition (CE)` \- [Gitlab](/content/sameersbn/docker-gitlab/index.html) is an open source end-to-end software development platform with built-in version control, issue tracking, code review, CI/CD, and more. Self-host GitLab CE on your own servers

The architecture of this stack is such that all services are behind an HTTP(S) & SSH reverse proxy; single point of entry. This reverse proxy in this case is started with self signed certificate _(See ./certs/README.md)_ using docker secrets.

### Prerequisite

Docker swarm mode environment is required
- Use provided `Vagrantfile` if you are unable to run Docker CE natively on a local machine.
- _OR_ see [Docker for AWS](https://docs.docker.com/docker-for-aws/) documentation on how to create a Docker swarm cluster on AWS.

# Deploy CI stack in a VirtualBox with provided Vagrantfile

The **assumption** here is that Vagrant, VirtualBox and Gitbash are already install on your machine _(my development environment was a Windows 10 Pro machine)_.

Execute the following commands, in gitbash, in order to create a two-node docker swarm mode cluster. The nodes are based on 'ubuntu/xenial64' VM. Once the cluster is created successfully, log in to the master node:

- `git clone https://github.com/shazChaudhry/docker-swarm-mode.git`
- `cd docker-swarm-mode`
- `vagrant up`
- `vagrant ssh node1` _(Log in to the master node)_
- `docker node ls` _(confirm that there are two nodes in the cluster; master and worker)_
- `cd /vagrant`

Deploy stack by running the following commands which will utilize [Docker secrets](https://docs.docker.com/engine/swarm/secrets/) for Jenkins and proxy.

- Jenkins secrets are defined in the "./secrets/jenkins" directory.
- Proxy's secrets are defined in the "./certs" directory.

```
docker stack deploy --compose-file docker-compose.portainer.yml portainer
docker stack deploy --compose-file docker-compose.yml ci
```

- Check status of the stack services by running the following command:

```
docker stack services ci
```

- Once all services are up and running, proceed to the next step.

#### Service URLs

- [http://node1:9000](http://node1:9000/) _(Portainer)_
- [https://node1/jenkins](https://node1/jenkins) _(Jenkins)_. admin username: `admin`; Password: `admin`
- [https://node1/sonar](https://node1/sonar) _(SonarQube)_. admin username: `admin`; Password: `admin`
- [https://node1/nexus](https://node1/nexus) _(Nexus)_. admin username: `admin`; Password: `admin123`
- [https://node1/gitlab](https://node1/gitlab) _(Gitlab CE)_. admin username: `root`; Password: `Password01`

#### Cloning repositories with HTTPS

- `git config --global http.sslVerify false` _(Turns off Git SSL Verification for a non trusted server certificate. Otherwise, you may receive 'SSL certificate problem: self signed certificate' error)_
- `git clone https://root:Password01@node1/gitlab/[GROUP_NAME]/[REPOSITORY_NAME].git`

#### Cloning repositories with SSH

- `git clone ssh://git@node1:10022/[GROUP_NAME]/[REPOSITORY_NAME].git`

#### Clean-up

On the swarm master node, run the following commands:
- `docker stack rm ci` to remove the stack
- `exit` to exit the vagrant box
- `vagrant destroy --force` to destroy the VMs

# Deploy CI stack on "Docker for AWS"

It is assumed you have followed [Docker for AWS](https://docs.docker.com/docker-for-aws/) documentation to create a new VPC. Follow these commands in an ssh client to log in to your master node _(I'm using gitbash on Windows 10 Pro)_.

**Please** note you can not ssh directly into worker nodes. You have to use a manager node as a jump box.

- `eval $(ssh-agent) OR exec ssh-agent bash`
- `ssh-add -k ~/.ssh/personal.pem` _(You will have to use your own key)_.
- `ssh -A docker@<Manager Public IP>`

Clone this repo and change directory by following these commands:
- `git clone https://github.com/shazChaudhry/docker-swarm-mode.git`
- `sudo chown -R $USER:$USER docker-swarm-mode`
- `cd docker-swarm-mode`

Start the Portainer by running:
- `docker stack deploy -c docker-compose.portainer.yml portainer`

Run the combined stack. Please note that secrets are defined in ./secrets/jenkins and ./certs directories in this repo:
- `docker stack deploy --compose-file docker-stack.yml ci`

#### Configuring your client for interacting with docker repos in Nexus

- You will need to look up docker daemon documentation for your system. Create a file `/etc/docker/daemon.json` and copy the following content in that file

```
{
"insecure-registries": [
    "[DefaultDNSTarget]:443",
    "[DefaultDNSTarget]:5000"
],
"disable-legacy-registry": true
}
```

- You have to restart the daemon after setting this `sudo systemctl restart docker`

- `docker login -u admin -p admin123 [DefaultDNSTarget]:443`

### Setup [Jenkins declarative](https://jenkins.io/doc/book/pipeline/syntax) pipeline

Steps to setting up this pipeline should be identical regardless of the deployment methods shown above; Local or AWS

- In the GitLab instance, import [spring-petclinic](/content/shazChaudhry/spring-petclinic/index.html) project from GitHub.
- In the Jenkins instance, create a new Blue Ocean pipeline project.
- Once "Deploy Key" has been created, head over to the "spring-petclinic" project in GitLab and ensure that you enable the deploy key at the bottom of the page.

## About

Setting up a Docker based CI environment. Tools include GitLap, Jenkins, Sonarqube and Nexus.

### Topics

[jenkins](/content/topics/jenkins "Topic: jenkins"/index.html) [devops](/content/topics/devops "Topic: devops"/index.html) [gitlab](/content/topics/gitlab "Topic: gitlab"/index.html) [continuous-integration](/content/topics/continuous-integration "Topic: continuous-integration"/index.html) [continuous-delivery](/content/topics/continuous-delivery "Topic: continuous-delivery"/index.html) [sonarqube](/content/topics/sonarqube "Topic: sonarqube"/index.html) [docker-swarm-mode](/content/topics/docker-swarm-mode "Topic: docker-swarm-mode"/index.html) [jenkins-pipeline](/content/topics/jenkins-pipeline "Topic: jenkins-pipeline"/index.html) [nexus3](/content/topics/nexus3 "Topic: nexus3"/index.html)