Scaling with Docker swarm and vagrant

Introduction

A couple of days ago I had the idea of scaling my server, if someone says scaling he says containers of course. So I went looking on how I could scale my server, well actually I don't need high end scaling but where is the fun in that? I came across 3 main container orchestrators: Kubernetes, Swarm and Mesos. I have experience with Docker so I went with Docker swarm to manage the scaling.

My server specs:

• i7 3th gen
• 8gigs of ram

Before setting it up I will explain some tools:

Setup

To make use of scaling you need machines in my case I will use virtual machines backed by virtualbox. A very handy tool to automate and manage vm's is by using vagrant. I will use 4 vm's where 1 is the manager and 3 slaves. A manager will manage the slaves so that your services will be divided equally over your vm's, and if by accident a slave vm goes down it will take the services from that vm and put it on an other.

So my vagrant directory will look like this (my pwd: /home/sevaho/vagrant):

$ tree -L 2
.
├── [sevaho   4.0K]  conf.d
│   └── [sevaho    737]  default.conf
├── [sevaho   4.0K]  http
│   ├── [sevaho    209]  index.php
├── [sevaho   4.0K]  swarm_slave_1
│   └── [sevaho   1.2K]  Vagrantfile
├── [sevaho   4.0K]  swarm_slave_2
│   └── [sevaho   1.2K]  Vagrantfile
├── [sevaho   4.0K]  swarm_slave_3
│   └── [sevaho   1.2K]  Vagrantfile
└── [sevaho   4.0K]  swarn_manager
    └── [sevaho   1.2K]  Vagrantfile

Contents of a Vagrantfile:

$script = <<SCRIPT
sudo apt-get install curl -y
curl -fsSL get.docker.com -o get-docker.sh
bash get-docker.sh
sudo usermod -aG docker vagrant
sudo curl -L https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
SCRIPT

Vagrant.configure(2) do |config|

    config.vm.box = "debian/jessie64"

    config.vm.network :private_network, ip: "192.168.56.12"
    config.vm.network "forwarded_port", guest: 80, host: 5000, auto_correct: true

    config.vm.synced_folder ".", "/vagrant", disabled: true
    config.vm.synced_folder "/home/sevaho/vagrant/http", "/http/", owner: "www-data", group: "www-data"
    config.vm.synced_folder "/home/sevaho/vagrant/conf.d", "/conf.d/", owner: "www-data", group: "www-data"
    config.vm.provision "shell", inline: $script
    config.vm.provider "virtualbox" do |vb|

        vb.gui = false
        vb.name = "swarm_slave_2"
        vb.memory = 1024
        vb.cpus = 1
        vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
        vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]

    end

end

This vagrantfile is from slave number 2, the things you will need to change for the other ones are:

• ip-address
• name

NOTE: I am forwarding port 80 from the vm to port 5000 of my server, this is very important so I can see output on this port. Also you need "natdnshostresolver1" and "natdnsproxy1" to be set otherwise you can't connect from outside of the vm to the inside except ssh.

Contents of default.conf (nginx file):

server {

    listen 80;
    server_name 127.0.0.1;
    index index.php index.html;

    error_log  /var/log/nginx/error.log;
    access_log /var/log/nginx/access.log;

    root /http;

    location ~ \.php$ {

        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass php:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;

    }

}

Contents of index.php:

<?php

    echo "<p>".$_SERVER['SERVER_ADDR']."</p>";
    echo "<p>".$_SERVER['SERVER_PORT']."</p>";
    echo "<p>".$_SERVER['REMOTE_ADDR']."</p>";
    echo "<p>".$_SERVER['REMOTE_HOST']."</p>";

Now you can deploy the vm's by running the following command in each directory:

$ vagrant up

Docker swarm

Docker is installed everywhere and our user vagrant is part of the docker group which means we don't need to run as root.

Ssh in your manager machine by running in it's directory:

$ vagrant ssh

Now you can setup a swarm by entering:

$ docker swarm init --advertise-addr 192.168.56.10

We need to advertise an address and I assigned 192.168.56.10 to my manager vm so we will go with that.

Now you will see something like this:

you can join in this swarm by:

    docker swarm join --token <token>

So we will execute this command in all the slave vm's!

To make sure all the vm's are in the swarm run this command in the manager:

$ docker node ls # you can see who is in the swarm

Now back the the manager vm, make a file (fe. website.yml) and fill it with:

version: '3'

services:

    nginx:
        image: nginx:latest
        ports:
            - "80:80"
        volumes:
            - /conf.d:/etc/nginx/conf.d
            - /http:/http
        links:
            - php 
        deploy:
            replicas: 15

    php:
        image: php:7.0-fpm
        volumes:
            - /http:/http
        deploy:
            replicas: 15

Now you can run (still inside the manager vm):

$ docker stack deploy -c website.yml website # website is just a name

Some magic will happen, Docker swarm will scale 30 services (15x nginx, 15x php) over your vm's. If you enter the following command a couple of times you will see the real magic happen, the server address will change constantly because you will be on a different nginx service container:

$ curl <host-ip>:5000

You have officialy scaled containers!

EDIT:

My output of some curl fun:

$ ~ ❯ curl https://sevaho.io/swarm.php                  
<p>Docker swarm node : 10.255.0.117:443</p><p>Your IP: 196.45.124.64</p>%  
$ ~ ❯ curl https://sevaho.io/swarm.php            
<p>Docker swarm node : 10.255.0.104:443</p><p>Your IP: 196.45.124.64</p>%   
$ ~ ❯ curl https://sevaho.io/swarm.php             
<p>Docker swarm node : 10.255.0.114:443</p><p>Your IP: 196.45.124.64</p>%   
$ ~ ❯ curl https://sevaho.io/swarm.php                    
<p>Docker swarm node : 10.255.0.100:443</p><p>Your IP: 196.45.124.64</p>%