0
330
ViewsDocker swarm replicas on different nodes
I'm using a docker-compose file version 3 with its deploy key to run a swarm (docker version 1.13) and I would like to replicate a service in order to make it resilient again single node failure.
However, when I'm adding a deploy section like this:
deploy:
replicas: 2
in my four node cluster I sometimes end up with both replicas scheduled on the same node. What I'm missing is a constraint that schedules the two instances on different nodes.
I know that there's a global mode I could use but that would run an instance on every node, i.e. four instances in my case instead of just two.
Is there a simple way to specify this constraint in a generic way without having to resort to a combination of global and a labels to keep additional instances away from?
Edit: After trying it again I find containers to be scheduled on different nodes this time around. I'm beginning to wonder if I may have had a 'node.hostname == X' constraint in place.
Edit 2: After another service update - and without any placement constraints - the service is again being scheduled on the same node (as displayed by ManoMarks Visualizer):
3 answers
Answer question0
Extending VonC answer, and as in your example you are using a compose file, not the cli, you could add max_replicas_per_node: 1, as in
version: '3.8'
...
yourservice:
deploy:
replicas: 2
placement:
max_replicas_per_node: 1
The compose schema version 3.8 is key here, as there is no support for max_replicas_per_node below 3.8.
This was added in https://github.com/docker/cli/pull/1410
0
docker/cli PR 1612 seems to resolve issue 26259, and has been released in docker 19.03.
Added new switch
--replicas-max-per-nodeswitch to docker serviceHow to verify it
Create two services and specify
--replicas-max-per-nodeone of them:docker service create --detach=true --name web1 --replicas 2 nginx docker service create --detach=true --name web2 --replicas 2 --replicas-max-per-node 1 nginxSee difference on command outputs:
$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS 0inbv7q148nn web1 replicated 2/2 nginx:latest 9kry59rk4ecr web2 replicated 1/2 (max 1 per node) nginx:latest $ docker service ps --no-trunc web2 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS bf90bhy72o2ry2pj50xh24cfp web2.1 nginx:latest@sha256:b543f6d0983fbc25b9874e22f4fe257a567111da96fd1d8f1b44315f1236398c limint Running Running 34 seconds ago xedop9dwtilok0r56w4g7h5jm web2.2 nginx:latest@sha256:b543f6d0983fbc25b9874e22f4fe257a567111da96fd1d8f1b44315f1236398c Running Pending 35 seconds ago "no suitable node (max replicas per node limit exceed)"
The error message would be:
no suitable node (max replicas per node limit exceed)
Examples from Sebastiaan van Stijn:
Create a service with max 2 replicas:
docker service create --replicas=2 --replicas-max-per-node=2 --name test nginx:alpine docker service inspect --format '{{.Spec.TaskTemplate.Placement.MaxReplicas}}' test 2Update the service (max replicas should keep its value)
docker service update --replicas=1 test docker service inspect --format '{{.Spec.TaskTemplate.Placement.MaxReplicas}}' test 2Update the max replicas to 1:
docker service update --replicas-max-per-node=1 test docker service inspect --format '{{.Spec.TaskTemplate.Placement.MaxReplicas}}' test 1And reset to 0:
docker service update --replicas-max-per-node=0 test docker service inspect --format '{{.Spec.TaskTemplate.Placement.MaxReplicas}}' test 0
0
What version of docker are you using? According to this post in 1.13 this kind of problem has been rectified, do take a look: https://github.com/docker/docker/issues/26259#issuecomment-260899732
Hope that answers your question.