Drone hangs when using templates

Drone Support
1

Possibly the same issue as here: [Insufficient Details] The drone template cannot deploy but the thread is locked ¯\(ツ)/¯

Upgraded from latest drone/drone:1 to latest drone/drone:2 today (image says Created": "2021-08-24T16:00:17.949108618Z)

When I do following:
Create .drone.star
Switch settings to use .drone.star as config
push to github
→ pipelines are correctly generated, jobs are running, everything works fine

Then I do:
Create .drone.yml
Switch settings to use .drone.yml as config
Upload template as following

root@75793e416aba:/opt/molecule/dronelib# drone template add --namespace veselahouba --name linter --data @linter.star
root@75793e416aba:/opt/molecule/dronelib# drone template ls
linter  [33mroot@75793e416aba:/opt/molecule/dronelib#

Something fishy about the template probably?

root@75793e416aba:/opt/molecule/dronelib# drone template info --name linter --namespace veselahouba
linter  [33mtemplate: _:1:30: executing "_" at <.Namespace>: can't evaluate field Namespace in type *drone.Template

push to github
→ pipeline is not generated, task hangs in pending

Here’s my config:
linter.star

def main(ctx):
  return {
    "kind": "pipeline",
    "name": "linter",
    "steps": [
      {
        "name": "Lint",
        "image": "veselahouba/molecule",
        "commands": [
          "shellcheck_wrapper",
          "flake8",
          "yamllint .",
          "ansible-lint"
        ]
      }
    ]
  }

.drone.yml

---
kind: template
load: veselahouba.linter
data:
  use_hetzner: true

my docker-compose.yml

version: '3'
services:
    drone-autoscaler:
        depends_on:
        - drone-server
        environment:
        - DRONE_POOL_MIN=0
        - DRONE_POOL_MIN_AGE=15m
        - DRONE_POOL_MAX=4
        - DRONE_SERVER_PROTO=https
        - DRONE_SERVER_HOST=drone.m-cloud.cz
        - DRONE_SERVER_TOKEN=<ommit>
        - DRONE_AGENT_TOKEN=<ommit>
        - DRONE_HETZNERCLOUD_TOKEN=<ommit>
        - DRONE_HETZNERCLOUD_SSHKEY=<ommit>
        - DRONE_HETZNERCLOUD_TYPE=cx21
        - DRONE_HETZNERCLOUD_IMAGE=ubuntu-20.04
        image: drone/autoscaler
        ports:
        - 8080:8080
        restart: unless-stopped
        volumes:
        - /opt/docker/drone/autoscaler:/data
    drone-runner:
        depends_on:
        - drone-server
        environment:
        - DRONE_RPC_PROTO=https
        - DRONE_RPC_HOST=drone.m-cloud.cz
        - DRONE_RPC_SECRET=<ommit>
        - DRONE_RUNNER_CAPACITY=2
        - DRONE_RUNNER_NAME=drone01.m-cloud.cz
        - DRONE_RPC_DUMP_HTTP=true
        - DRONE_RPC_DUMP_HTTP_BODY=true
        image: drone/drone-runner-docker:1
        ports:
        - 127.0.0.1:3000:3000
        restart: unless-stopped
        volumes:
        - /var/run/docker.sock:/var/run/docker.sock
    drone-server:
        environment:
        - DRONE_GITHUB_CLIENT_ID=<ommit>
        - DRONE_GITHUB_CLIENT_SECRET=<ommit>
        - DRONE_RPC_SECRET=<ommit>
        - DRONE_SERVER_HOST=drone.m-cloud.cz
        - DRONE_SERVER_PROTO=https
        - DRONE_USER_FILTER=<ommit>
        - DRONE_USER_CREATE=<ommit>
        - DRONE_STARLARK_ENABLED=true
        - DRONE_LOGS_TRACE=true
        image: drone/drone:2
        ports:
        - 1080:80
        restart: unless-stopped
        volumes:
        - /opt/docker/drone/data:/data

docker logs output

job info

root@75793e416aba:/opt/molecule/dronelib# drone build info VeselaHouba/ansible-role-crowdsec 22
Number: 22
Status: pending
Event: push
Commit: a28fd900af58d95b0a5398d0f580960d67561d4f
Branch: drone
Ref: refs/heads/drone
Author: VeselaHouba <michalek@m-cloud.cz>
Message: Drone templating
2

The problem is that your template does not have a recognizable suffix. If you are using starlark as the template language, your extension name must end in .star or .starlark so the language can be identified and evaluated.

To be fair, Drone should prevent you from creating a template with a missing or unsupported file suffix. That is an improvement we will need to make to Drone and include in an upcoming release.

3

Hmmm not sure what am I doing wrong then:

I’ve tried creating templates like following:

drone template add --namespace veselahouba --name linter.star --data @linter.star
drone template add --namespace veselahouba --name linter.starlark --data @linter.star

Tried pipelines like this:

kind: template
load: veselahouba.linter.starlark
data:
  use_hetzner: true

also

kind: template
load: linter.starlark
data:
  use_hetzner: true

And more or less every combination of namespace & name which came across my mind, but with zero success.

4

Also tried to drop all persistent data and start fresh drone server & runner instances, still the same result.

When tried CLI version 1.3.0, I can actually get template listing

./drone template ls
linter.star  [33mveselahouba
Data:  def main(ctx):
  return {
    "kind": "pipeline",
    "name": "linter",
    "steps": [
      {
        "name": "Lint",
        "image": "veselahouba/molecule",
        "commands": [
          "shellcheck_wrapper",
          "flake8",
          "yamllint .",
          "ansible-lint"
        ]
      }
    ]
  }

Namespace seems ok, although the bash coloring is somehow buggy, filename is linter.star, … what am I missing?

5

Hey @VeselaHouba
I can see from your drone.yml you are trying to pass into your template file:

  use_hetzner: true

However, you don’t actually refer to it anywhere in your template?

What behavior are you expecting?

6

Hi @eoinmcafee00 I have more sophisticated starlark template code prepared, which will take this parameter.
Based on ansible-role-docker/.drone.star at master · VeselaHouba/ansible-role-docker · GitHub

But for now I wanted to make the CI work with simple step and once it’s running I will add more features.

So to answer the question: No, it has no use in original example, except not having data variable empty.

7

@VeselaHouba, Can you test again using the drone server latest tag please? I’ve pushed out an update to enforce correct template extension types, and want to see if you are getting any errors back from the server.

8

Thanks!
switched image and recreated server

...
        image: drone/drone:latest
...

yup, really latest one

docker exec -ti drone_drone-server_1 ls -l /bin/drone-server
-rwxr-xr-x    1 root     root      51245792 Aug 31 15:50 /bin/drone-server

When I push changes to repo I see following in server debug, not very helpful :frowning:

{"commit":"34e67f3bf77aa972a21aa4fb170a21924be07901","event":"push","level":"debug","msg":"webhook parsed","name":"ansible-role-crowdsec","namespace":"VeselaHouba","time":"2021-08-31T18:37:58Z"}
{"commit":"34e67f3bf77aa972a21aa4fb170a21924be07901","event":"push","level":"debug","msg":"trigger: received","ref":"refs/heads/drone","repo":"VeselaHouba/ansible-role-crowdsec","time":"2021-08-31T18:37:58Z"}
{"fields.time":"2021-08-31T18:37:58Z","latency":404695925,"level":"debug","method":"POST","msg":"","remote":"172.30.25.1:45376","request":"/hook","request-id":"1xVKzXDyKdMnJxDVS3e8ukNkRWw","time":"2021-08-31T18:37:58Z"}

and then just repeating scheduler

{"arch":"amd64","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2021-08-31T18:38:40Z","type":"docker","variant":""}

I’ve now tested adding the simple yaml example from docu, to exclude .star as the cause, but that also hangs.

9

Hey @VeselaHouba

I believe we’ve identified this issue here. In your drone.yml file, you have — at the start. There’s a regular expression check for kind: template, if it doesn’t match it won’t get into the logic. (it should work though)
I’ve raised a PR fix templating reg expression to match if .drone.yml contains --- characters by eoinmcafee00 · Pull Request #313 to fix this and will get it released this afternoon.
Hopefully this solves it.

10

Holy cow, that was unexpected. Yes, templating works now, both yaml and starlark

Funny that first running pipeline actually complains about the issue :smiley:

7	./.drone.yml
8	  1:1       warning  missing document start "---"  (document-start)

Thank you both guys @eoinmcafee00 @bradrydzewski for your time and effort!