1. Introduction
This post introduces Docker Swarm – the Docker Engine’s Swarm mode. The Swarm mode enables multiple machines running Docker Engines to collaborate in a cluster. Let us explore the basics of this.
To follow the examples provided here you will need Docker Engine 1.12 or above to be installed in your machine. You will also need to install VirualBox, and two virtual machines running Ubuntu 16.04 LTS server edition. Let us get started.
You can also check this tutorial in the following video:
Table Of Contents
2. What is Docker Swarm?
Docker Swarm is Docker’s native feature to support clustering of Docker machines. This enables multiple machines running Docker Engine to participate in a cluster, called Swarm. The Docker engines contributing to a Swarm are said to be running in Swarm mode. Machines enter into the Swarm mode by either initializing a new swarm or by joining an existing swarm. To the end user the swarm would seem like a single machine. A Docker engine participating in a swarm is called a node. A node can either be a manager node or a worker node. The manager node performs cluster management and orchestration while the worker nodes perform tasks allocated by the manager.
A manager node itself, unless configured otherwise, is also be a worker node. The central entity in the Docker Swarm infrastructure is called a service. A Docker swarm executes services. The user submits a service to the manager node to deploy and execute. A service is made up of many tasks. A task is the most basic work unit in a Swarm. A task is allocated to each worker node b the manager node.
Services can be scaled at runtime to handle extra load. The swarm manager natively supports internal load balancing to distribute tasks across the participating worker nodes. Also, the manager also supports ingress load balancing to control exposure of Docker services to the external world. The manager node also supports service discovery by automatically assigning a DNS entry to every service.
More details about these concepts can be found from the official Docker documentation. We will now proceed to setup a basic multi-node Docker Swarm setup to understand it further.
3. Setup the tools required to setup a multi-node swarm
Here is what you will need to setup the infrastructure for the rest of this post. Basically, I setup a Docker machine and two different virtual machines on my laptop. The host operating system was Ubuntu 16.04 Server and the guest OS for the virtual machines were also Ubuntu 16.04 server.
1. Install Oracle VirtualBox 5.1 from here.
2. Download Ubuntu Server 16.04 LTS ISO image, for installing into the VMs, from here.
3. Create 2 VMs in VirtualBox using the ISO image downloaded above.
Instructions to create a Ubuntu Server VM in VirtualBox can be found here or from this video guide here. I created 2 VMs called hariharan-docker-swarm-worker-node-1 and hariharan-docker-swarm-worker-node-2.
4. Boot the 2 VMs and install the latest Docker Engine in both. Installation instructions for Ubuntu can be found from here.
5. Create a docker machine called “docker-swarm-manager” with virtualbox as the driver.
$ docker-machine create --driver virtualbox docker-swarm-manager
The above command essentially creates a virtual machine in VirtualBox using the Boot2Docker image. Let us next start this machine and initialize a Docker Swarm in this.
4. Start the swarm-manager machine
$ docker-machine start docker-swarm-manager Starting "docker-swarm-manager"... (docker-swarm-manager) Check network to re-create if needed... (docker-swarm-manager) Waiting for an IP... Machine "docker-swarm-manager" was started. Waiting for SSH to be available... Detecting the provisioner... Started machines may have new IP addresses. You may need to re-run the `docker-machine env` command. $
Noe the Docker machine is started. We will use this as the Swarm manager node. So let us SSH into this machine and create a Swarm.
$ docker-machine ssh docker-swarm-manager $ $ docker swarm init --advertise-addr 192.168.99.100 Swarm initialized: current node (oex7thwlqimoy6wxddliyew9q) is now a manager. To add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-1k0yuvvfpsid27slhn5of2ue0xjq91kqcb2qp81idyvt7amj5j-7t5fada5fqxw3wv3fk0cihoji \ 192.168.99.100:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
The above command initialized a Swarm and made the above machine a manager node. It also provided instructions on how other nodes can join into the Swarm. Let us next join the other 2 VMs we created above into this Swarm.
5. Start the worker-node machines
Open VirtualBox and Start the VMs hariharan-docker-swarm-worker-node-1. Login into the terminal and give the following command:
$ docker swarm join --token SWMTKN-1-1k0yuvvfpsid27slhn5of2ue0xjq91kqcb2qp81idyvt7amj5j-7t5fada5fqxw3wv3fk0cihoji 192.168.99.100:2377 This node joined a swarm as a worker.
Repeat the same steps as above with the VM hariharan-docker-swarm-worker-node-2 so that it also joins the swarm as a worker. So now, we should have a swarm of 3 nodes with one among them being a master node.
6. Verify the status of the swarm in the swarm manager
Go into the terminal running the swarm-manager and type the command docker info in the manager node. It should show, among a lot of information, the information about swarms.
$ docker info ... ... Swarm: active NodeID: oex7thwlqimoy6wxddliyew9q Is Manager: true ClusterID: tq0gkoqfujux2a20oidx8idjd Managers: 1 Nodes: 4 Orchestration: Task History Retention Limit: 5 Raft: Snapshot Interval: 10000 Number of Old Snapshots to Retain: 0 Heartbeat Tick: 1 Election Tick: 3 Dispatcher: Heartbeat Period: 5 seconds CA Configuration: Expiry Duration: 3 months Node Address: 192.168.99.100 Manager Addresses: 192.168.99.100:2377 ... ...
Another more specific way to to verify the same is through the command docker node ls, like below:
$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS oex7thwlqimoy6wxddliyew9q * docker-swarm-manager Ready Active Leader rcm93fzmoeb4gfxgzsavvmhfg hariharan-docker-swarm-worker-node-2 Ready Active v673asfigzknhwmu5gk7gb3ng hariharan-docker-swarm-worker-node-1 Ready Active
From the above output, it is seen that the host docker-swarm-manager is marked with a * indicating that is the manager node. The other 2 hosts above are the worker nodes. You can see the status of a specific node by using the command – docker node inspect. Let us inspect the node hariharan-docker-swarm-worker-node-2 more, like so
$ docker node inspect --pretty hariharan-docker-swarm-worker-node-2 ID: rcm93fzmoeb4gfxgzsavvmhfg Hostname: hariharan-docker-swarm-worker-node-2 Joined at: 2017-01-28 13:20:10.217592895 +0000 utc Status: State: Ready Availability: Active Address: 192.168.99.1 Platform: Operating System: linux Architecture: x86_64 Resources: CPUs: 1 Memory: 992.3 MiB Plugins: Network: bridge, host, macvlan, null, overlay Volume: local Engine Version: 1.13.0
Now that we have verified the setup, let us next deploy a simple service in the swarm.
7. Deploy a service
In the manager node, give the below command to create a service. This service is based on the very simple Docker image that simply creates an infinite loop.
$ docker service create --replicas 1 --name infinite-loop enhariharan/infinite-loop 4ybguvnthyiclecftvbtg0m3x
Above, we named the service as infinite-loop by the --name option. The option --replicas specifies the number of tasks present in this servicer.
Verify that the service is running by the following command
$ docker service ls ID NAME MODE REPLICAS IMAGE 4ybguvnthyic infinite-loop replicated 1/1 enhariharan/infinite-loop:latest
Now let us inspect the details of the service in the manager node.
$ docker service inspect --pretty infinite-loop ID: 4ybguvnthyiclecftvbtg0m3x Name: infinite-loop Service Mode: Replicated Replicas: 1 Placement: UpdateConfig: Parallelism: 1 On failure: pause Max failure ratio: 0 ContainerSpec: Image: enhariharan/infinite-loop:latest@sha256:b746e8ac5a4027616e1c4e682030e9f1749d5297275f108d5215ec8cb00d65cb Resources: Endpoint Mode: vip
Let us also find out which node is running the service
$ docker service ps infinite-loop ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS ujntt8yxy3yx infinite-loop.1 enhariharan/infinite-loop:latest docker-swarm-manager Running Running 3 minutes ago
Till now, we looked at the basics of creating a swarm, adding nodes to it, deploying a service and inspecting it’s details. Let us next look at the basics of scaling the service we deployed.
8. Scale the service
Until now there was only one instance of the service running, that too in the manager node itself. Let us load up this swarm a bot more to understand it’s behavior. In the manager node, use the command docker service scale to scale the service.
$ docker service scale infinite-loop=10 infinite-loop scaled to 10
That’s that. Now let see the result of this scaling by using the command docker service ps.
$ docker service ps infinite-loop ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS ujntt8yxy3yx infinite-loop.1 enhariharan/infinite-loop:latest docker-swarm-manager Running Running 10 minutes ago 78nbb9guclfz infinite-loop.2 enhariharan/infinite-loop:latest docker-swarm-manager Running Running 23 seconds ago zi285gwiuyjp infinite-loop.3 enhariharan/infinite-loop:latest docker-swarm-manager Running Running 23 seconds ago d942isrqxhwr infinite-loop.4 enhariharan/infinite-loop:latest hariharan-ThinkPad-W510 Running Running 4 seconds ago 4gsyyuc4lo2c infinite-loop.5 enhariharan/infinite-loop:latest hariharan-docker-swarm-worker-node-2 Running Running 9 seconds ago irtte6ktum6y infinite-loop.6 enhariharan/infinite-loop:latest hariharan-ThinkPad-W510 Running Running 4 seconds ago 70nawd6pjyiz infinite-loop.7 enhariharan/infinite-loop:latest hariharan-docker-swarm-worker-node-1 Running Running 9 seconds ago ir6eqxnr30vc infinite-loop.8 enhariharan/infinite-loop:latest hariharan-docker-swarm-worker-node-2 Running Running 9 seconds ago q85imuo1hwai infinite-loop.9 enhariharan/infinite-loop:latest hariharan-docker-swarm-worker-node-1 Running Running 8 seconds ago npr1mejjg76d infinite-loop.10 enhariharan/infinite-loop:latest hariharan-docker-swarm-worker-node-1 Running Running 8 seconds ago
We can see that the service has tasks distributed across all the nodes in the swarm now. The worker node hariharan-docker-swarm-worker-node-1 spawned 3 tasks, hariharan-docker-swarm-worker-node-2 spawned 2, the manager node docker-swarm-manager spawned 3, and the remaining 2 was taken up by another node that I spawned later in the host OS.
To see the containers running on the manager node, the good old command docker ps will do the job. To see the container running on the worker node, SSH into the worker node and try the same docker ps command there.
9. Remove the service
Finally, let us remove the service. Use the command docker service rm to accomplish this, like below:
$ docker service rm infinite-loop infinite-loop
and verify that the service was removed.
$ docker service ls ID NAME MODE REPLICAS IMAGE
Verify that the containers pertaining to the service are also removed using docker ps:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
10. Summary
In this post, we discussed the basic concepts of Docker Swarm mode. Docker engines provide support for container clustering and orchestration while it runs in the Swarm mode. A Docker engine, also called a node, enters the Swarm mode by wither initializing a new Swarm or by joining an existing one.
We looked at how a swarm can be setup and started by using the various docker swarm CLI commands. We looked at some basic commands to manage and manipulate the service life cycle in a cluster using docker service commands. Finally, we looked at how to scale a service using the docker service scale command.
Last updated on Feb. 27th, 2022
