Custom cloud-init in DigitalOceans

Drone Support
1

I’m trying to setup drone autoscaler on digital oceans for a couple of days, I hope someone can help to solve my issue!

I followed the documentation: https://autoscale.drone.io/configure/cloud-init/

the agent stucks in staging state and dies after some time with context deadline exceeded 

Name: agent-5d06DqCw
Address: 178.62.83.123
Region:  lon1
Size:    s-4vcpu-8gb
State:   staging

After debugging and tracing I figured out that dockerd is not listening to the port 2376.

{"level":"debug","region":"lon1","image":"docker-18-04","size":"s-4vcpu-8gb","name":"agent-2XRwjhYP","name":"agent-2XRwjhYP","ip":"178.62.13.194","time":"2019-10-28T09:38:37Z","message":"instance network ready"}
{"level":"debug","server":"agent-2XRwjhYP","time":"2019-10-28T09:38:37Z","message":"provisioned server"}
{"level":"debug","ip":"178.62.13.194","name":"agent-2XRwjhYP","name":"agent-2XRwjhYP","time":"2019-10-28T09:38:45Z","message":"check docker connectivity"}
{"level":"debug","ip":"178.62.13.194","name":"agent-2XRwjhYP","name":"agent-2XRwjhYP","time":"2019-10-28T09:38:45Z","message":"connecting to docker"}
{"level":"debug","ip":"178.62.13.194","name":"agent-2XRwjhYP","error":"Cannot connect to the Docker daemon at https://178.62.13.194:2376. Is the docker daemon running?","name":"agent-2XRwjhYP","time":"2019-10-28T09:38:45Z","message":"cannot connect, retry in 1m0s"}
{"level":"debug","ip":"178.62.13.194","name":"agent-2XRwjhYP","name":"agent-2XRwjhYP","time":"2019-10-28T09:56:45Z","message":"connecting to docker"}
{"level":"debug","ip":"178.62.13.194","name":"agent-2XRwjhYP","error":"Cannot connect to the Docker daemon at https://178.62.13.194:2376. Is the docker daemon running?","name":"agent-2XRwjhYP","time":"2019-10-28T09:56:45Z","message":"cannot connect, retry in 1m0s"}
{"level":"debug","id":"ZJblZNy3da89SFpl","min-pool":0,"max-pool":5,"server-buffer":0,"server-capacity":5,"server-count":1,"pending-builds":0,"running-builds":0,"time":"2019-10-28T09:57:35Z","message":"check capacity"}
{"level":"debug","id":"ZJblZNy3da89SFpl","time":"2019-10-28T09:57:35Z","message":"terminate 1 servers"}
{"level":"debug","id":"ZJblZNy3da89SFpl","time":"2019-10-28T09:57:35Z","message":"no idle servers to shutdown"}
{"level":"debug","id":"ZJblZNy3da89SFpl","time":"2019-10-28T09:57:35Z","message":"check capacity complete"}
{"level":"debug","ip":"178.62.13.194","name":"agent-2XRwjhYP","name":"agent-2XRwjhYP","time":"2019-10-28T09:57:45Z","message":"connecting to docker"}
{"level":"debug","ip":"178.62.13.194","name":"agent-2XRwjhYP","error":"Cannot connect to the Docker daemon at https://178.62.13.194:2376. Is the docker daemon running?","name":"agent-2XRwjhYP","time":"2019-10-28T09:57:45Z","message":"cannot connect, retry in 1m0s"}

but the main issue actually is that my cloudinit file does not even executed!
logs from /var/log/cloud-init-output.log :

Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~18.04.1 running 'init-local' at Mon, 28 Oct 2019 14:04:26 +0000. Up 12.97 seconds.
Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~18.04.1 running 'init' at Mon, 28 Oct 2019 14:04:28 +0000. Up 14.98 seconds.
ci-info: +++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
ci-info: | Device |  Up  |           Address            |      Mask     | Scope  |     Hw-Address    |
ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
ci-info: |  eth0  | True |        178.62.83.123         | 255.255.192.0 | global | e2:43:ca:54:df:8f |
ci-info: |  eth0  | True |          10.16.0.6           |  255.255.0.0  | global | e2:43:ca:54:df:8f |
ci-info: |  eth0  | True | fe80::e043:caff:fe54:df8f/64 |       .       |  link  | e2:43:ca:54:df:8f |
ci-info: |   lo   | True |          127.0.0.1           |   255.0.0.0   |  host  |         .         |
ci-info: |   lo   | True |           ::1/128            |       .       |  host  |         .         |
ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
ci-info: +++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++
ci-info: +-------+-------------+-------------+---------------+-----------+-------+
ci-info: | Route | Destination |   Gateway   |    Genmask    | Interface | Flags |
ci-info: +-------+-------------+-------------+---------------+-----------+-------+
ci-info: |   0   |   0.0.0.0   | 178.62.64.1 |    0.0.0.0    |    eth0   |   UG  |
ci-info: |   1   |  10.16.0.0  |   0.0.0.0   |  255.255.0.0  |    eth0   |   U   |
ci-info: |   2   | 178.62.64.0 |   0.0.0.0   | 255.255.192.0 |    eth0   |   U   |
ci-info: +-------+-------------+-------------+---------------+-----------+-------+
ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++
ci-info: +-------+-------------+---------+-----------+-------+
ci-info: | Route | Destination | Gateway | Interface | Flags |
ci-info: +-------+-------------+---------+-----------+-------+
ci-info: |   1   |  fe80::/64  |    ::   |    eth0   |   U   |
ci-info: |   3   |    local    |    ::   |    eth0   |   U   |
ci-info: |   4   |   ff00::/8  |    ::   |    eth0   |   U   |
ci-info: +-------+-------------+---------+-----------+-------+
2019-10-28 14:04:28,932 - __init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: 'b'#cloud-init'...'
...
...
Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~18.04.1 running 'modules:final' at Mon, 28 Oct 2019 14:04:36 +0000. Up 22.34 seconds.
Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~18.04.1 finished at Mon, 28 Oct 2019 14:04:37 +0000. Datasource DataSourceDigitalOcean.  Up 23.55 seconds

This is my setup:
docker stack file:

services:
  drone-autoscaler:
    image: drone/autoscaler
    ports:
      - 8080:8080
    volumes:
      - autoscaler-data:/data:delegated
      - ./scripts:/scripts:ro
      - ./runner-init.yml:/init.yml
    environment:
      - DRONE_AGENT_TOKEN=${DRONE_AGENT_TOKEN}
      - DRONE_AGENT_CONCURRENCY=5
      - DRONE_POOL_MIN=0
      - DRONE_POOL_MAX=5
      - DRONE_SERVER_PROTO=https
      - DRONE_SERVER_HOST=${CICD_FQDN}
      - DRONE_SERVER_TOKEN=${DRONE_SERVER_TOKEN}
      - DRONE_DIGITALOCEAN_SSHKEY=${DRONE_DIGITALOCEAN_SSHKEY}
      - DRONE_DIGITALOCEAN_TOKEN=${DRONE_DIGITALOCEAN_TOKEN}
      - DRONE_DIGITALOCEAN_SIZE=s-4vcpu-8gb
      - DRONE_DIGITALOCEAN_TAGS=devops,drone,agent
      - DRONE_DIGITALOCEAN_REGION=lon1
      - DRONE_LOGS_COLOR=true
      - DRONE_SLACK_WEBHOOK=${ZEN_AUTOMATOR_SLACK_WEBHOOK}
      - DRONE_TLS_AUTOCERT=true
      - DRONE_DIGITALOCEAN_USERDATA=#cloud-init
      - DRONE_DIGITALOCEAN_USERDATA_FILE=/init.yml

runner-init.yml same as : https://autoscale.drone.io/configure/cloud-init/

2

It was my mistake!! I removed DRONE_DIGITALOCEAN_USERDATA and it worked !