Docker

Docker Swarm Tutorial

1. Introduction

This post introduces Docker Swarm – the Docker Engine’s Swarm mode. The Swarm mode enables multiple machines running Docker Engines to collaborate in a cluster. Let us explore the basics of this.
 
 
 
 
 
 
 

 
To follow the examples provided here you will need Docker Engine 1.12 or above to be installed in your machine. You will also need to install VirualBox, and two virtual machines running Ubuntu 16.04 LTS server edition.  Let us get started.

You can also check this tutorial in the following video:

Docker Swarm Tutorial – Video

2. What is Docker Swarm?

Docker Swarm is Docker’s native feature to support clustering of Docker machines. This enables multiple machines running Docker Engine to participate in a cluster, called Swarm. The Docker engines contributing to a Swarm are said to be running in Swarm mode. Machines enter into the Swarm mode by either initializing a new swarm or by joining an existing swarm. To the end user the swarm would seem like a single machine. A Docker engine participating in a swarm is called a node. A node can either be a manager node or a worker node. The manager node performs cluster management and orchestration while the worker nodes perform tasks allocated by the manager.

A manager node itself, unless configured otherwise, is also be a worker node. The central entity in the Docker Swarm infrastructure is called a service. A Docker swarm executes services. The user submits a service to the manager node to deploy and execute. A service is made up of many tasks. A task is the most basic work unit in a Swarm. A task is allocated to each worker node b the manager node.

Services can be scaled at runtime to handle extra load. The swarm manager natively supports internal load balancing to distribute tasks across the participating worker nodes. Also, the manager also supports ingress load balancing to control exposure of Docker services to the external world. The manager node also supports service discovery by automatically assigning a DNS entry to every service.

More details about these concepts can be found from the official Docker documentation. We will now proceed to setup a basic multi-node Docker Swarm setup to understand it further.

3. Setup the tools required to setup a multi-node swarm

Here is what you will need to setup the infrastructure for the rest of this post. Basically, I setup a Docker machine and two different virtual machines on my laptop. The host operating system was Ubuntu 16.04 Server and the guest OS for the virtual machines were also Ubuntu 16.04 server.
1. Install Oracle VirtualBox 5.1 from here.
2. Download Ubuntu Server 16.04 LTS ISO image, for installing into the VMs, from here.
3. Create 2 VMs in VirtualBox using the ISO image downloaded above.

Instructions to create a Ubuntu Server VM in VirtualBox can be found here or from this video guide here. I created 2 VMs called hariharan-docker-swarm-worker-node-1 and hariharan-docker-swarm-worker-node-2.
4. Boot the 2 VMs and install the latest Docker Engine in both. Installation instructions for Ubuntu can be found from here.
5. Create a docker machine called “docker-swarm-manager” with virtualbox as the driver.

$ docker-machine create --driver virtualbox docker-swarm-manager

The above command essentially creates a virtual machine in VirtualBox using the Boot2Docker image. Let us next start this machine and initialize a Docker Swarm in this.

4. Start the swarm-manager machine

$ docker-machine start docker-swarm-manager
Starting "docker-swarm-manager"...
(docker-swarm-manager) Check network to re-create if needed...
(docker-swarm-manager) Waiting for an IP...
Machine "docker-swarm-manager" was started.
Waiting for SSH to be available...
Detecting the provisioner...
Started machines may have new IP addresses. You may need to re-run the `docker-machine env` command.
$

Noe the Docker machine is started. We will use this as the Swarm manager node. So let us SSH into this machine and create a Swarm.

$ docker-machine ssh docker-swarm-manager
$
$ docker swarm init --advertise-addr 192.168.99.100
Swarm initialized: current node (oex7thwlqimoy6wxddliyew9q) is now a manager.

To add a worker to this swarm, run the following command:

docker swarm join \
--token SWMTKN-1-1k0yuvvfpsid27slhn5of2ue0xjq91kqcb2qp81idyvt7amj5j-7t5fada5fqxw3wv3fk0cihoji \
192.168.99.100:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

The above command initialized a Swarm and made the above machine a manager node. It also provided instructions on how other nodes can join into the Swarm. Let us next join the other 2 VMs we created above into this Swarm.

5. Start the worker-node machines

Open VirtualBox and Start the VMs hariharan-docker-swarm-worker-node-1. Login into the terminal and give the following command:

$ docker swarm join --token SWMTKN-1-1k0yuvvfpsid27slhn5of2ue0xjq91kqcb2qp81idyvt7amj5j-7t5fada5fqxw3wv3fk0cihoji 192.168.99.100:2377
This node joined a swarm as a worker.

Repeat the same steps as above with the VM hariharan-docker-swarm-worker-node-2 so that it also joins the swarm as a worker. So now, we should have a swarm of 3 nodes with one among them being a master node.

6. Verify the status of the swarm in the swarm manager

Go into the terminal running the swarm-manager and type the command docker info in the manager node. It should show, among a lot of information, the information about swarms.

$ docker info
...
...
Swarm: active
NodeID: oex7thwlqimoy6wxddliyew9q
Is Manager: true
ClusterID: tq0gkoqfujux2a20oidx8idjd
Managers: 1
Nodes: 4
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Node Address: 192.168.99.100
Manager Addresses:
192.168.99.100:2377
...
...

Another more specific way to to verify the same is through the command docker node ls, like below:

$ docker node ls
ID                           HOSTNAME                              STATUS  AVAILABILITY  MANAGER STATUS
oex7thwlqimoy6wxddliyew9q *  docker-swarm-manager                  Ready   Active        Leader
rcm93fzmoeb4gfxgzsavvmhfg    hariharan-docker-swarm-worker-node-2  Ready   Active
v673asfigzknhwmu5gk7gb3ng    hariharan-docker-swarm-worker-node-1  Ready   Active

From the above output, it is seen that the host docker-swarm-manager is marked with a * indicating that is the manager node. The other 2 hosts above are the worker nodes. You can see the status of a specific node by using the command – docker node inspect. Let us inspect the node hariharan-docker-swarm-worker-node-2 more, like so

$ docker node inspect --pretty hariharan-docker-swarm-worker-node-2
ID:                    rcm93fzmoeb4gfxgzsavvmhfg
Hostname:              hariharan-docker-swarm-worker-node-2
Joined at:             2017-01-28 13:20:10.217592895 +0000 utc
Status:
  State:               Ready
  Availability:        Active
  Address:             192.168.99.1
Platform:
  Operating System:    linux
  Architecture:        x86_64
Resources:
  CPUs:            1
  Memory:              992.3 MiB
Plugins:
  Network:             bridge, host, macvlan, null, overlay
  Volume:              local
Engine Version:        1.13.0

Now that we have verified the setup, let us next deploy a simple service in the swarm.

7. Deploy a service

In the manager node, give the below command to create a service. This service is based on the very simple Docker image that simply creates an infinite loop.

$ docker service create --replicas 1 --name infinite-loop enhariharan/infinite-loop
4ybguvnthyiclecftvbtg0m3x

Above, we named the service as infinite-loop by the --name option. The option --replicas specifies the number of tasks present in this servicer.
Verify that the service is running by the following command

$ docker service ls
ID            NAME           MODE        REPLICAS  IMAGE
4ybguvnthyic  infinite-loop  replicated  1/1       enhariharan/infinite-loop:latest

Now let us inspect the details of the service in the manager node.

$ docker service inspect --pretty infinite-loop

ID:                  4ybguvnthyiclecftvbtg0m3x
Name:                infinite-loop
Service Mode:        Replicated
  Replicas:          1
Placement:
UpdateConfig:
  Parallelism:       1
On failure:          pause
  Max failure ratio: 0
ContainerSpec:
  Image:             enhariharan/infinite-loop:latest@sha256:b746e8ac5a4027616e1c4e682030e9f1749d5297275f108d5215ec8cb00d65cb
Resources:
Endpoint Mode:       vip

Let us also find out which node is running the service

$ docker service ps infinite-loop
ID            NAME             IMAGE                             NODE                  DESIRED STATE  CURRENT STATE          ERROR  PORTS
ujntt8yxy3yx  infinite-loop.1  enhariharan/infinite-loop:latest  docker-swarm-manager  Running        Running 3 minutes ago

Till now, we looked at the basics of creating a swarm, adding nodes to it, deploying a service and inspecting it’s details. Let us next look at the basics of scaling the service we deployed.

8. Scale the service

Until now there was only one instance of the service running, that too in the manager node itself. Let us load up this swarm a bot more to understand it’s behavior. In the manager node, use the command docker service scale to scale the service.

$ docker service scale infinite-loop=10
infinite-loop scaled to 10

That’s that. Now let see the result of this scaling by using the command docker service ps.

$ docker service ps infinite-loop
ID            NAME              IMAGE                             NODE                                  DESIRED STATE  CURRENT STATE           ERROR  PORTS
ujntt8yxy3yx  infinite-loop.1   enhariharan/infinite-loop:latest  docker-swarm-manager                  Running        Running 10 minutes ago
78nbb9guclfz  infinite-loop.2   enhariharan/infinite-loop:latest  docker-swarm-manager                  Running        Running 23 seconds ago
zi285gwiuyjp  infinite-loop.3   enhariharan/infinite-loop:latest  docker-swarm-manager                  Running        Running 23 seconds ago
d942isrqxhwr  infinite-loop.4   enhariharan/infinite-loop:latest  hariharan-ThinkPad-W510               Running        Running 4 seconds ago
4gsyyuc4lo2c  infinite-loop.5   enhariharan/infinite-loop:latest  hariharan-docker-swarm-worker-node-2  Running        Running 9 seconds ago
irtte6ktum6y  infinite-loop.6   enhariharan/infinite-loop:latest  hariharan-ThinkPad-W510               Running        Running 4 seconds ago
70nawd6pjyiz  infinite-loop.7   enhariharan/infinite-loop:latest  hariharan-docker-swarm-worker-node-1  Running        Running 9 seconds ago
ir6eqxnr30vc  infinite-loop.8   enhariharan/infinite-loop:latest  hariharan-docker-swarm-worker-node-2  Running        Running 9 seconds ago
q85imuo1hwai  infinite-loop.9   enhariharan/infinite-loop:latest  hariharan-docker-swarm-worker-node-1  Running        Running 8 seconds ago
npr1mejjg76d  infinite-loop.10  enhariharan/infinite-loop:latest  hariharan-docker-swarm-worker-node-1  Running        Running 8 seconds ago

We can see that the service has tasks distributed across all the nodes in the swarm now. The worker node hariharan-docker-swarm-worker-node-1 spawned 3 tasks, hariharan-docker-swarm-worker-node-2 spawned 2, the manager node docker-swarm-manager spawned 3, and the remaining 2 was taken up by another node that I spawned later in the host OS.

To see the containers running on the manager node, the good old command docker ps will do the job. To see the container running on the worker node, SSH into the worker node and try the same docker ps command there.

9. Remove the service

Finally, let us remove the service. Use the command docker service rm to accomplish this, like below:

$ docker service rm infinite-loop
infinite-loop

and verify that the service was removed.

$ docker service ls
ID  NAME  MODE  REPLICAS  IMAGE

Verify that the containers pertaining to the service are also removed using docker ps:

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

10. Summary

In this post, we discussed the basic concepts of Docker Swarm mode. Docker engines provide support for container clustering and orchestration while it runs in the Swarm mode. A Docker engine, also called a node, enters the Swarm mode by wither initializing a new Swarm or by joining an existing one.
 
We looked at how a swarm can be setup and started by using the various docker swarm CLI commands. We looked at some basic commands to manage and manipulate the service life cycle in a cluster using docker service commands. Finally, we looked at how to scale a service using the docker service scale command.

Last updated on Feb. 27th, 2022

Hariharan Narayanan

Hari graduated from the School of Computer and Information Sciences in the University of Hyderabad. Over his career he has been involved in many complex projects in mobile applications, enterprise applications, distributed applications, micro-services, and other platforms and frameworks. He works as a consultant and is mainly involved with projects based on Java, C++ and Big Data technologies.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Back to top button