I have two servers in Docker Swarm, but when I need to add a third server - I get the result:
Error response from daemon: rpc error: code = 14 desc = grpc: the connection is unavailable
All servers in one network.
What could be the problem?
I'd say it's possibly firewall related. Ensure your ports are configured correctly on the third box. From the Docker docs:
Open protocols and ports between the hosts The following ports must be available. On some systems, these ports are open by default.
TCP port 2377 for cluster management communications TCP and UDP port 7946 for communication among nodes UDP port 4789 for overlay network traffic
From official Docker swarm tutorial
The following ports must be open on your docker hosts.
TCP port 2377 for cluster management communications
TCP and UDP port 7946 for communication among nodes
UDP port 4789 for overlay network traffic
To enable this ports run the below command on all your docker hosts. kindly follow the digitalocen article for complete steps.
firewall-cmd --add-port=2376/tcp --permanent
firewall-cmd --add-port=2377/tcp --permanent
firewall-cmd --add-port=7946/tcp --permanent
firewall-cmd --add-port=7946/udp --permanent
firewall-cmd --add-port=4789/udp --permanent
As others have pointed out, closed ports could be one reason. But I've also found a couple of more.
Recent version of Docker is suffering from massive proxy issues:
According to this comment, the fix is "likely" to make it into Docker version 17.11 and it is "considered" to be put in a patch release for 17.09.
All my ports are open and the NO_PROXY hack described in the aforementioned links did not work.
I tried all Docker versions between 17.05 all the way to 17.11.0-ce-rc3, build 5b4af4f with no success which led me to suspect the culprit might be a recent upgrade of Vagrant (I am using 2.0.1) and/or VirtualBox (using 5.1.30). Upgrading either one of these two usually leads to all kinds of random problems. But, instead of downgrading these guys I tried to upgrade the Vagrant boxes I run.
In my two-machine setup, I switched the first node's box to fso/artful64-desktop and the second node's box to fso/artful64 (both version 2017-11-01). To my surprise, this made Docker Swarm work on version 17.10.0-ce and 17.11.0-ce-rc3, build 5b4af4f. Please note that private networking is broken on Vagrant 2.0.1 if you want to use Ubuntu 17.10 boxes lol (can be manually fixed).