Controlling the CIDR for bridge networks created by Drone?

Drone Support
1

Hey all,
On my company’s private network, we have infrastructure deployed all throughout the 172.17.0.0/16 range. This is in conflict with the subnet that Docker natively chooses on its own accord, but I’ve been able to get around this a couple ways:

Running ubuntu 14.04, Docker version 17.09.

/etc/default/docker has DOCKER_OPTS:

DOCKER_OPTS="--bip=192.168.252.1/24" # sets the default docker0 subnet

Then, I deploy Drone using a custom network as well:

docker network create --subnet=192.168.251.0/24 primary_network

And refer to it in docker-compose.yml for Drone:

networks:
  default:
    external:
      name: primary_network

But unfortunately, the output of route looks like this when Drone does a build:

# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         v300            0.0.0.0         UG    0      0        0 eth0
10.11.165.0     *               255.255.255.0   U     0      0        0 eth0
172.17.0.0      *               255.255.0.0     U     0      0        0 br-f6a49107f69f
192.168.251.0   *               255.255.255.0   U     0      0        0 br-8d7101c99db3
192.168.252.0   *               255.255.255.0   U     0      0        0 docker0

I noticed the environment variable DRONE_NETWORK in the source code for Drone, but I have not had success with it. Has anyone been able to control the IP address range for the bridge networks that Drone creates?

2

Similar to docker compose, drone creates a new network for each build. It would be the equivalent to the below docker commands. This is important because the agent networking does not have any impact on the network used by your build containers.

docker network create foo
docker run --network=foo golang /bin/sh -c "go build; go test"

Creating the per-build network is important because this allows us to support container aliasing and prevents conflicts when two builds run on the same server at the same time.

This particular issue is out of my area of expertise, however, I am certainly open to suggestions if there is a setting we can add that creates the per-build network with custom parameters.

3

Hey @bradrydzewski ,
Thanks for responding. I understand your response completely.

After some digging it seems that this isn’t necessarily Drone’s responsibility, it’s actually more Docker’s.

This issue was created, which basically is a request for the ability to configure the default cidr chosen by the docker network create command: https://github.com/moby/moby/issues/21776

And in the long string of comments, someone actually created a PR which is still open because Docker Swarm mode does not work with it: https://github.com/moby/moby/pull/29376

These two threads have been around for almost a year. It’s not clear if they’re going to be closed / fixed any time soon.

So with all that being said, Drone can still rescue me here, without having to rely on Docker fixing it.

In my case, I have a build setup that only can run 1 build at a time. So in this case, I could do with a simple flag (perhaps set it in the pipeline somewhere?) that looks something like this:

pipeline:
  drone_build_network_cidr: 192.168.250.0/24

Or it could be set per-build-step as an attachment to an existing network by name or uuid, which in my case is primary_network:

pipeline:
    # http://plugins.drone.io/drone-plugins/drone-slack/
    slack-begin:
        docker_network: primary_network
        image: plugins/slack
        webhook: https://hooks.slack.com/services/blah/blah/blah
        channel: chat
        username: drone
        icon_url: https://pbs.twimg.com/profile_images/573911705505259520/rShq67tG.png
        template: >
          [<!date^{{build.created}}^{date_num} {time_secs}|date unavailable>] {{build.author}}
          has triggered a {{build.event}} event to {{repo.owner}}/{{repo.name}} on the
          {{build.branch}} branch. <{{build.link}}|Drone build {{build.number}} is starting>.
        when:
            branch:
                include: [master/*, master, refs/tags/*]
            event: [push, pull_request, tag, deployment]

Or specify a subnet manually per-pipeline?

pipeline:
  drone_network:
    subnet: 192.168.250.0/24
  slack-begin:
    ...

And then have Drone inject that option into the docker network create foo if it’s set, which ends up looking like

docker network create --subnet=${drone_build_network_cidr} foo

I understand that this may break the ability to do multiple builds at once only when it’s set, which is probably a highly used feature for many. I don’t have a solution for this. I don’t think a solution is needed, either – this can be introduced as a basic solution for single-build configurations and perhaps another one for parallel-build configs can be introduced later.

What do you think? I can try to tinker with the Drone source code to see if this can be done. I have no exposure to it though, and could take me some time… if you happen to think of a quick way to implement this, I’m all ears and can work with you on it.

4

In the short term something like the below example might work, although it would require enabling trusted mode for your build which has some security implications if you do not trust your committers.

pipeline:
  publish:
    image: slack
    networks:
      - primary_network

If that doesn’t work I can probably walk you through a quick temporary patch that you can use in the short term.

I do have a new version of the drone yml parser [1] and drone runtime [2], both which have improved support for custom and external networks [3]. I feel like this would allow you to define your custom network and possibly avoid this issue?

pipeline:
  build:
    image: foo
    networks:
      - primary_network

networks:
  default:
    external:
      name: primary_network

Note that these libraries will likely be integrated in version 0.9 and will not be available until around the end of the month.

[1] https://github.com/drone/drone-yaml-v1
[2] https://github.com/drone/drone-runtime
[3] https://github.com/drone/drone-yaml-v1/blob/master/samples/2_network.yml

1 Like
5

So I tried setting the networks field like this, and it did not have any effect:

pipeline:
    # http://plugins.drone.io/drone-plugins/drone-slack/
    slack-begin:
        docker_network: primary_network
        image: plugins/slack
        webhook: https://hooks.slack.com/services/blah/blah/blah
        channel: chat
        username: drone
        icon_url: https://pbs.twimg.com/profile_images/573911705505259520/rShq67tG.png
        template: >
          [<!date^{{build.created}}^{date_num} {time_secs}|date unavailable>] {{build.author}}
          has triggered a {{build.event}} event to {{repo.owner}}/{{repo.name}} on the
          {{build.branch}} branch. <{{build.link}}|Drone build {{build.number}} is starting>.
        when:
            branch:
                include: [master/*, master, refs/tags/*]
            event: [push, pull_request, tag, deployment]
        networks:
            - primary_network

Things to note:

  • the repository is set to trusted, as you advised
  • running drone 0.8.1
  • the docker network primary_network definitely exists on the Docker (Drone) host!

I would like to know about this temporary patch you mentioned, I am able to put some time into this solution so I appreciate any guidance you can offer.

As for the 0.9.0 syntax that you demonstrated, it looks like it would do exactly what I’m looking for.

6

I would recommend the following

Configure a global network with your drone server. This global network is defined with the server but should exist on your agent(s). I know you mentioned you attempted to use this parameter previously, but it might be worth double checking that it was configured correctly with the server.

DRONE_NETWORK=primary_network

If this does not work the issue could be that Drone is adding both the global network AND the user-defined network to your container. The relevant code would be here.

7

Yep so you were right, what’s happening is it’s appending the primary_network onto the ad-hoc _default network.

"Networks": {
    "0_4170208218217518028_default": {
        "IPAMConfig": null,
        "Links": null,
        "Aliases": [
            "sandbox-deploy",
            "4bbb1402a458"
        ],
        "NetworkID": "115f9fc462bfdd04719ef36033ba860d54010f6c6c5151ff0815bcb8569c9cff",
        "EndpointID": "c31004bea35ed45a7640b5a23a4dccfc752179b4e9b01e50e6051c7564a97517",
        "Gateway": "172.17.0.1",
        "IPAddress": "172.17.0.2",
        "IPPrefixLen": 16,
        "IPv6Gateway": "",
        "GlobalIPv6Address": "",
        "GlobalIPv6PrefixLen": 0,
        "MacAddress": "02:42:ac:11:00:02",
        "DriverOpts": null
    },
    "bridge": {
        "IPAMConfig": null,
        "Links": null,
        "Aliases": null,
        "NetworkID": "9b43ab8423d606328acd18ceb1063debaf2ffc13f6356ee597609a0e501a942b",
        "EndpointID": "4df91d536bb036466a7da5cd258649c9454f8b04dfdf9b04c1606cdeeaf00e35",
        "Gateway": "192.168.252.1",
        "IPAddress": "192.168.252.2",
        "IPPrefixLen": 24,
        "IPv6Gateway": "",
        "GlobalIPv6Address": "",
        "GlobalIPv6PrefixLen": 0,
        "MacAddress": "02:42:c0:a8:fc:02",
        "DriverOpts": null
    },
    "primary_network": {
        "IPAMConfig": null,
        "Links": null,
        "Aliases": [
            "4bbb1402a458"
        ],
        "NetworkID": "8d7101c99db37355ed8a4986a599a623e8881c531b0c0436c1c949640adfde22",
        "EndpointID": "30bcef7c64d16c2491cc6ac9f10876556ae1f3612bc52f3ef8a6588e745758fe",
        "Gateway": "192.168.251.1",
        "IPAddress": "192.168.251.4",
        "IPPrefixLen": 24,
        "IPv6Gateway": "",
        "GlobalIPv6Address": "",
        "GlobalIPv6PrefixLen": 0,
        "MacAddress": "02:42:c0:a8:fb:04",
        "DriverOpts": null
    }
}

So what I’m looking for, which you’re probably aware of at this point, is for the _default network to not even be attached to the container and it can just land on one of the other networks.

In the code you linked, we could add some simple flag that turns off the _default network appending behavior, for example something like this:

(Maybe here’s a good environment variable name): DRONE_NETWORKS_ONLY=True and this would be parsed and added to the conf object as a conf.NetworksOnly boolean:

  // users can specify existing Docker networks for Drone to use
  // by specifying the env variable DRONE_NETWORKS
  // and then Drone will only use them with DRONE_NETWORKS_ONLY=True
  if conf.NetworksOnly != true {
      // create a default network
      config.Networks = append(config.Networks, &backend.Network{
        Name:   fmt.Sprintf("%s_default", c.prefix),
        Driver: "bridge",
    })
  }

What do you think about this concept?

8

Or perhaps alternatively, if DRONE_NETWORK is specified, then don’t append the _default network at all? I’m not sure what effect that might have on existing use cases though.