# \[SOLVED\]: How to get Drone working in Docker (Swarm Mode) with docker stack deploy

**URL:** <https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401>\
**Category:** Drone Support\
**Created:** [December 2, 2017, 10:32pm UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401 "2017-12-02T22:32:14Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![prologic](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/prologic/32/3885_2.png) [@prologic](https://drone.discourse.group/u/prologic)\
**Post date:** [December 2, 2017, 10:32pm UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/1 "2017-12-02T22:32:14Z")

</div>

Heya all!

After many weeks of frustration, digging, poking around I have finally found a configuration that works in the following scenario:

- Docker (Swarm Mode)
- Deployed as a service with `docker stack deploy`

There are some known issues with Load Balancers with Drone server\<-\>agent communications  
and this apparently also includes Docker’s Overlay networking; but I suspect this has more to do with VIPs.

This is my working configuration:

```auto
version: "3.3"

services:
  drone-server:
    image: drone/drone:latest
    ports:
      - target: 9000
        published: 9000
        protocol: tcp
        mode: host
    environment:
      - DRONE_DEBUG=true
      - DRONE_OPEN=true
      - DRONE_HOST=https://ci.mydomain.com
      - DRONE_GOGS=true
      - DRONE_GOGS_PRIVATE_MODE=true
      - DRONE_GOGS_URL=https://git.mydomain.com
      - DRONE_SECRET=XXXX
      - DRONE_ADMIN=admin
    networks:
      - traefik
    volumes:
      - dronedata:/var/lib/drone
    deploy:
      placement:
        constraints:
          - "node.hostname == node1.mydomain.com"
      endpoint_mode: dnsrr
      labels:
        - "traefik.enable=true"
        - "traefik.port=8000"
        - "traefik.backend=ci"
        - "traefik.docker.network=traefik"
        - "traefik.frontend.rule=Host:ci.mydomain.com"
      restart_policy:
        condition: on-failure
      replicas: 1

  drone-agent:
    image: drone/agent:latest
    command: agent
    networks:
      - bridge
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_DEBUG=true
      - DRONE_SERVER=10.0.0.10:9000
      - DRONE_SECRET=XXX
    deploy:
      placement:
        constraints:
          - "node.role != manager"
      restart_policy:
        condition: on-failure
      replicas: 3

networks:
  bridge:
    external: true
  traefik:
    external: true

volumes:
  dronedata:
    external: true

```

To deploy:

```auto
$ docker stack deploy -c ci.yml ci

```

I believe the main thing that got this working correctly and reliably was publishing the port to the host.

Some notes:

- I run the server on node1 (`10.0.0.10`)
- I run the agents on all other nodes

---

<div class="post-metadata">

**Author:** ![rgarrigue](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/rgarrigue/32/4044_2.png) [@rgarrigue](https://drone.discourse.group/u/rgarrigue)\
**Post date:** [December 7, 2017, 10:12am UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/2 "2017-12-07T10:12:53Z")

</div>

Hi prologic

I was going to dig into the topic sooner or later, thanks for the inspiration 🙂. I’m curious about the issue you faced, is it unstable/not working without the hardcoded node1 ip as node server ? Or the server:9000 being published ? Why did you had to specify the endpoint\_mode ?

Best regards,

---

<div class="post-metadata">

**Author:** ![prologic](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/prologic/32/3885_2.png) [@prologic](https://drone.discourse.group/u/prologic)\
**Post date:** [December 7, 2017, 4:42pm UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/3 "2017-12-07T16:42:20Z")

</div>

It was unstable. You’d have to restart the server+agent(s) to get things  
going again.

The endpoint\_mode can be one of the following:

endpoint\_mode: vip - Docker assigns the service a virtual IP (VIP), which  
acts as the “front end” for clients to reach the service on a network.  
Docker routes requests between the client and available worker nodes for  
the service, without client knowledge of how many nodes are participating  
in the service or their IP addresses or ports. (This is the default.)

endpoint\_mode: dnsrr - DNS round-robin (DNSRR) service discovery does not  
use a single virtual IP. Docker sets up DNS entries for the service such  
that a DNS query for the service name returns a list of IP addresses, and  
the client connects directly to one of these. DNS round-robin is useful in  
cases where you want to use your own load balancer, or for Hybrid Windows  
and Linux applications.

And changing the way the port is published avoids using thte overlay  
network at all.

The gRPC internally used in Drone doesn’t seem to like going through  
overlay networks much (unclear why)  
and I’m not sure how the VIP comes in to play (probably not needed?)

cheers  
James

James Mills / prologic

E: [prologic@shortcircuit.net.au](mailto:prologic@shortcircuit.net.au)  
W: [prologic.shortcircuit.net.au](http://prologic.shortcircuit.net.au)

---

<div class="post-metadata">

**Author:** ![prologic](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/prologic/32/3885_2.png) [@prologic](https://drone.discourse.group/u/prologic)\
**Post date:** [December 7, 2017, 4:45pm UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/4 "2017-12-07T16:45:03Z")

</div>

Oh and I think I remember why publishing the port to the host is required;  
otherwise the host won’t have a bound listening interface so it won’t work  
without.

James Mills / prologic

E: [prologic@shortcircuit.net.au](mailto:prologic@shortcircuit.net.au)  
W: [prologic.shortcircuit.net.au](http://prologic.shortcircuit.net.au)

---

<div class="post-metadata">

**Author:** ![zaggash](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/zaggash/32/3624_2.png) [@zaggash](https://drone.discourse.group/u/zaggash)\
**Post date:** [December 8, 2017, 9:57am UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/5 "2017-12-08T09:57:48Z")

</div>

You do not have to expose the port 9000 if your only build on agents in the swarm.  
on the agent use the service name as a dns, and you’ll be good.  
And please do not use the bridge network on swarm if you need port communications between nodes.  
Overlay is the way to go.

I’m using drone in swarm and it works flowesly.  
I can share my stack if needed.

---

<div class="post-metadata">

**Author:** ![prologic](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/prologic/32/3885_2.png) [@prologic](https://drone.discourse.group/u/prologic)\
**Post date:** [December 8, 2017, 4:45pm UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/6 "2017-12-08T16:45:46Z")

</div>

@zaggash Please share your config. You’ve completely missed the point here.  
Please look in the forum for other references and you’ll find comments from  
@bradrydzewski clearly stating several issues with Server\<-\>Drone comms  
over load balancers or reverse proxies which includes the Overlay  
networking in Docker.

James Mills / prologic

E: [prologic@shortcircuit.net.au](mailto:prologic@shortcircuit.net.au)  
W: [prologic.shortcircuit.net.au](http://prologic.shortcircuit.net.au)

---

<div class="post-metadata">

**Author:** ![zaggash](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/zaggash/32/3624_2.png) [@zaggash](https://drone.discourse.group/u/zaggash)\
**Post date:** [December 8, 2017, 4:59pm UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/7 "2017-12-08T16:59:50Z")

</div>

Here is my stack, I’m using the overlay without publishing anything other than my Traefik ports.  
All the magic done with drone goes through swarm and the overlay network.  
I had an issue with “pending build”, solved by adding the `endpoint_mode:` to `dnsrr` on the server, that’s how it solves the VIP IP translation ( similar to a reverse proxy of the port 9000)

The stack is started with `docker stack deploy -c docker-compose.yml ci`  
This explain some `ci` prefix 😉

```auto
version: '3.4'

services:
  lb:
    image: traefik:1.4.5
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"
    command:
      - "--graceTimeOut=5s"
      - "--logLevel=info"
      - "--defaultentrypoints=http,https"
      - "--entryPoints=Name:http Address::80 Redirect.EntryPoint:https"
      - "--entryPoints=Name:https Address::443 TLS"
      - "--web"
      - "--docker"
      - "--docker.domain=yourdomain.com"
      - "--docker.swarmmode=true"
      - "--docker.exposedbydefault=false"
      - "--acme"
      #- "--acme.caServer=https://acme-staging.api.letsencrypt.org/directory"
      - "--acme.email=webmaster@yourdomain.fr"
      - "--acme.entryPoint=https"
      - "--acme.onhostrule=true"
      - "--acme.storage=/etc/traefik/acme/acme.json"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/dev/null:/traefik.toml"
      - "/opt/docker_data/traefik/:/etc/traefik/acme"
    networks:
      - gateway
    ports:
      - "80:80/tcp"
      - "443:443/tcp"
    deploy:
      mode: replicated
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
        failure_action: pause
        order: start-first
        monitor: 30s
      restart_policy:
        condition: on-failure
        max_attempts: 3

  drone-server:
    image: drone/drone
    logging:
      driver: json-file
      options:
        max-size: "5m"
        max-file: "3"
    environment:
      - DRONE_DEBUG=true
      - DRONE_OPEN=false
      - DRONE_GITHUB=true
      - DRONE_ADMIN=_HIDDEN_
      - DRONE_GITHUB_URL=https://github.com
      - DRONE_GITHUB_CLIENT=_HIDDEN_
      - DRONE_GITHUB_SECRET=_HIDDEN_
      - DRONE_GITHUB_CONTEXT=continuous-integration/drone
      - DRONE_GITHUB_SCOPE=repo,repo:status,user:email,read:org
      - DRONE_HOST=https://drone.yourdomain.com
      - DRONE_SECRET=_HIDDEN_
    volumes:
      - drone-sqlite:/var/lib/drone/
    networks:
      - gateway
      - ci
    deploy:
      placement:
        constraints:
          - node.role!=manager
      labels:
      - "traefik.backend=drone"
      - "traefik.frontend.rule=Host:drone.yourdomain.com"
      - "traefik.port=8000"
      - "traefik.docker.network=ci_gateway"
      - "traefik.enable=true"
      endpoint_mode: dnsrr
      mode: replicated
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
        failure_action: pause
        order: start-first
        monitor: 10s
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3

  drone-agent:
    image: drone/agent
    command: agent
    logging:
      driver: json-file
      options:
        max-size: "5m"
        max-file: "3"
    depends_on:
      - drone-server
    environment:
      - DRONE_DEBUG=true
      - DRONE_SERVER=drone-server:9000
      - DRONE_SECRET=_HIDDEN_
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - ci
    deploy:
      placement:
        constraints:
          - node.role!=manager
       preferences:
          - spread: node.labels.ci
      mode: replicated
      replicas: 10
      update_config:
        parallelism: 2
        delay: 10s
        failure_action: pause
        order: start-first
        monitor: 10s
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3

volumes:
  drone-sqlite:

networks:
  gateway:
  ci:

```

---

<div class="post-metadata">

**Author:** ![prologic](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/prologic/32/3885_2.png) [@prologic](https://drone.discourse.group/u/prologic)\
**Post date:** [December 11, 2017, 12:22am UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/8 "2017-12-11T00:22:27Z")

</div>

So it looks like you’re still utilizing the overlay networking here but  
avoiding the VIP endpoint discovery and opting instead for DNS RR.

Is that what makes all the difference in stability? (As I said there are  
some open issues around this and overlay in general in moby upstream)

James Mills / prologic

E: [prologic@shortcircuit.net.au](mailto:prologic@shortcircuit.net.au)  
W: [prologic.shortcircuit.net.au](http://prologic.shortcircuit.net.au)

---

<div class="post-metadata">

**Author:** ![zaggash](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/zaggash/32/3624_2.png) [@zaggash](https://drone.discourse.group/u/zaggash)\
**Post date:** [December 11, 2017, 1:06am UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/9 "2017-12-11T01:06:56Z")

</div>

Yep; dns-rr solves it because, it avoids the load balancing with the VIP.  
Instead the container ip is directly seen by the agents.

---

<div class="post-metadata">

**Author:** ![prologic](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/prologic/32/3885_2.png) [@prologic](https://drone.discourse.group/u/prologic)\
**Post date:** [December 11, 2017, 1:30am UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/10 "2017-12-11T01:30:09Z")

</div>

Yeap makes sense! I’ll update my config later to match 🙂

Glad we finally sorted this out; I’ve been struggling to get a stable Drone  
CI going for a few months 🙂 (what precious little spare time!)

cheers  
James

James Mills / prologic

E: [prologic@shortcircuit.net.au](mailto:prologic@shortcircuit.net.au)  
W: [prologic.shortcircuit.net.au](http://prologic.shortcircuit.net.au)

---

<div class="post-metadata">

**Author:** ![honzas](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/honzas/32/3570_2.png) [@honzas](https://drone.discourse.group/u/honzas)\
**Post date:** [December 31, 2017, 8:54pm UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/11 "2017-12-31T20:54:41Z")

</div>

Hi zaggash,

i’ve tried your solution of running drone in docker swarm. Unfortunately my treafik complains, that drone-server and agent service are badly configured “ignored endpoint-mode not supported”. I guess the reason is this [https://github.com/containous/traefik/blob/master/provider/docker/docker.go#L379](https://github.com/containous/traefik/blob/master/provider/docker/docker.go#L379) (TLDR traefik does not support service with endpoint\_mode dnsrr)

Do you have any idea how could I get your stack going? (BTW prologic’s hack with hardcoded ip address does work, but it’s not convenient)

---

<div class="post-metadata">

**Author:** ![zaggash](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/zaggash/32/3624_2.png) [@zaggash](https://drone.discourse.group/u/zaggash)\
**Post date:** [December 31, 2017, 11:13pm UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/12 "2017-12-31T23:13:49Z")

</div>

I see.  
I never saw this message in my stack.  
That’s weird, I cannot explain.

How many drone-server do you start ? more than one ?  
Did you check if drone-server start before or after traefik ? ( try to start it first )

---

<div class="post-metadata">

**Author:** ![honzas](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/honzas/32/3570_2.png) [@honzas](https://drone.discourse.group/u/honzas)\
**Post date:** [January 1, 2018, 9:54am UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/13 "2018-01-01T09:54:51Z")

</div>

My setup is swarm cluster with 2 machines (manager + worker) with following compose file (basically i removed all ssl stuff - i’ve got my own ssl termination before this stack and running single instance of each service). I tried starting traefik as first service, as last service, I’ve tried running all service on single node (manager) as well as on separate nodes and traefik still complains. The only idea I’ve got is, that traefic introduced this “feature” in relatively new version (my stack is created like 2 days ago) and you are running older version without this restriction… (I’m just wildly guessing)

```
version: '3.4'
services:
  lb:
    image: traefik:1.5
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"
    command:
      - "--graceTimeOut=5s"
      - "--logLevel=info"
      - "--defaultentrypoints=http"
      - "--entryPoints=Name:http Address::80"
      - "--web"
      - "--docker"
      - "--docker.domain=__MYDOMAIN.COM"
      - "--docker.swarmmode=true"
      - "--docker.exposedbydefault=false"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/dev/null:/traefik.toml"
    networks:
      - gateway
    ports:
      - "80:80/tcp"
    deploy:
      mode: replicated
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
        failure_action: pause
        order: start-first
        monitor: 30s
      restart_policy:
        condition: on-failure
        max_attempts: 3

  drone-server:
    image: drone/drone:0.8
    volumes:
      - /var/lib/drone:/var/lib/drone/
# ports:
# - target: 9000
# published: 9000
# protocol: tcp
# mode: host
    environment:
      - DRONE_OPEN=true
      - DRONE_HOST=https://MACHINE.__MYDOMAIN.COM
      - DRONE_BITBUCKET=true
      - DRONE_BITBUCKET_CLIENT=SECRET
      - DRONE_BITBUCKET_SECRET=SECRET
      - DRONE_SECRET=SECRET
      - HTTPS_PROXY=MY_HTTP_PROXY
      - HTTP_PROXY=MY_HTTP_PROXY
      - https_proxy=MY_HTTP_PROXY
      - http_proxy=MY_HTTP_PROXY
      - DRONE_DEBUG=true
    networks:
      - gateway
      - ci
    deploy:
      placement:
        constraints:
          - node.role!=manager
      labels:
      - "traefik.backend=drone"
      - "traefik.frontend.rule=Host:MACHINE.__MYDOMAIN.COM"
      - "traefik.port=8000"
      - "traefik.docker.network=drone_gateway"
      - "traefik.enable=true"
      endpoint_mode: dnsrr
      mode: replicated
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
        failure_action: pause
        order: start-first
        monitor: 10s
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
  drone-agent:
    image: drone/agent:0.8
    command: agent
    depends_on:
      - drone-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - ci
    environment:
      #ugly hack - see http://discuss.harness.io/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/1166
# - DRONE_SERVER=10.97.23.171:9000
      - DRONE_SERVER=drone-server:9000
      - DRONE_SECRET=SECRET
      - DRONE_DEBUG=true
    deploy:
      placement:
        constraints:
          - node.role!=manager
      endpoint_mode: dnsrr
      mode: replicated
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
        failure_action: pause
        order: start-first
        monitor: 10s
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
networks:
  gateway:
  ci:
```

---

<div class="post-metadata">

**Author:** ![honzas](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/honzas/32/3570_2.png) [@honzas](https://drone.discourse.group/u/honzas)\
**Post date:** [January 1, 2018, 10:08am UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/14 "2018-01-01T10:08:30Z")

</div>

Oh, DAMN! that’s it. It works with older version of traefic - I’ve just tried it with 1.4.5 (I guess older rc of 1.5 are fine too) and it all works. So either i use hardcoded ips (which I don’t want to) or I use older version of software (which is even worse 🙂 )

---

<div class="post-metadata">

**Author:** ![zaggash](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/zaggash/32/3624_2.png) [@zaggash](https://drone.discourse.group/u/zaggash)\
**Post date:** [January 1, 2018, 10:22am UTC](https://drone.discourse.group/t/solved-how-to-get-drone-working-in-docker-swarm-mode-with-docker-stack-deploy/10401/15 "2018-01-01T10:22:27Z")

</div>

Yep actually in my stack I’m currently using v1.4.5, I mistype the version in the compose file, its on me.

They may have done some change in the 1.5rc-ish versions.

Good to know you solved it by yourself.

I am going to edit the upper compose file right now