Some rough edges in Drone

Hi all. I had need to spend some time with Drone recently, working to replace our existing setup with a new one. I have some thoughts after completing this work, and I’ve written them up here so as not to make noise on GitHub. Please let me know in a reply which (if any) should be turned into issues and I’ll make it so. Cheers :slight_smile:

  1. The refactor of drone exec has introduced two regressions:
    1. The output of different steps is no longer colored making them more difficult to distinguish between.
    2. If the pipeline fails, the failing step is no longer reported at the end making it more difficult (or potentially impossible) to identify the failing step.
  2. ${DRONE_WORKSPACE} periodically is empty. I’m not yet sure how to repro or provide any further detail on this one. For now /drone/src is hardcoded. Any advice on how I might go about diagnosing the cause is welcome.
  3. There are several repos in the Drone GitHub org that are archived without any explanation or redirection. The one I ran into via Google looking for the docker plugin was drone/drone-docker.
  4. At some point data seems to have moved from /var/lib/drone to /data. The top Google result for “drone compose” mounts /var/lib/docker, and the local development example provided by Drone themselves uses Postgres, so there is no official example demonstrating the correct path to mount.
  5. It seems there used to be an OpenAPI spec, that has since disappeared. It would be great to bring that back, and serve Swagger-UI with it.
  6. The warning on CLI startup of duplicate proto type registered is very misleading when attempting to diagnose an issue.
  7. The formatted-go-object output of drone lint isn’t useful to the user, and it’s not intuitive that linter warnings/errors are simply prefixed with linter:.
  8. If the build queue is too long, it breaks the information in the summary (total builds is correct).
    image
  9. Passing any invalid secret-file or env-file to drone exec silently fails (see here).
  10. Manual runs triggered by the “Build Now” button aren’t terribly useful:
    • It triggers a run with a custom event, and cannot replicate a push or PR.
    • If the pipeline doesn’t include the custom event trigger, nothing at all happens (appearing to the user as a silent failure, but apparent from the logs what went wrong).
    • For users who stumble upon this looking to run pipelines again:
      • You can trigger webhook re-delivery in Github from Repo settings -> Webhooks -> [select webhook] -> Recent deliveries -> [expand call you want to redeliver] (you can also duplicate the request from the inspector’s network tab as there are no replay protections here)
      • If you’d like to queue up a pile of jobs (for performance/concurrency testing or something) see the section below for spoofing webhooks yourself.

Queuing Many Jobs In Drone:

  • Copy Drone’s database.sqlite from the drone container (or otherwise access your drone DB):

    docker cp container:/data/database.sqlite database.sqlite
    
  • Extract the webhook secret:

    sqlite3 database.sqlite "SELECT repo_signer FROM repos WHERE repo_name LIKE '%[REPO-NAME-HERE]%';"
    
  • Visit GitHub (or your SCM) to copy the payload for the hook event you’d like to duplicate and note the value of the X-GitHub-Event header: Repo settings -> Webhooks -> [select webhook] -> Recent deliveries -> [expand call you want to duplicate]. Note you do need the entire thing.

  • Substitute the necessary values into the script below, and run it as many times as you’d like:

    #!/bin/bash
    
    set -euo pipefail
    
    read -r -d '' DATA << EOM
    [PAYLOAD-HERE]
    EOM
    
    HMAC_SECRET='[REPO-SIGNER-VALUE-HERE]'
    URL='https://[DRONE-URL-HERE]/hook'
    
    SIG256="$(echo -n "${DATA}" | openssl dgst -sha256 -hmac "${HMAC_SECRET}" | awk '{print "X-Hub-Signature-256: sha256="$2}')"
    curl -v --http1.1 -X POST -H "Content-Type: application/json" -H "${SIG256}" -H 'X-GitHub-Event: [EVENT-TYPE-HERE]' --data "${DATA}" "$URL"
    
1 Like

Hello @GaryMoon

Thanks for putting all of this together! I know it must have taken some time.

I will do my best to respond to each point, but it will take time.

First, if you are able to help contribute fixes for any of the issues you found, we would very much appreciate pull requests! If you need help figuring out where to submit a change, let me know.

For the duplicate proto type message from the CLI, please see this recent post Drone CLI unwanted output

I’ll be looking into the other issues as I find time.

Thanks again