Drone depends_on more serial than it needs to be

Drone Support
1

basic .drone.yml:

kind: pipeline
type: docker
name: parallel

steps:
      - ....

---

kind: pipeline
type: docker
name: serial

steps:
      - ....

depends_on:
      - parallel

node:
      runner: single

We have two runners: an unlabeled one with DRONE_RUNNER_CAPACITY=10 which is meant to execute the “parallel” pipeline, and one labeled “runner: single” which has DRONE_RUNNER_CAPACITY=1. The idea is the “parallel” pipeline can run up to 10 different builds in parallel, while “serial” is gated to one build at a time on a specific runner. In practice it seems “parallel” won’t start until “serial” gets the queue (runner with “runner: single” starts processing the build, but correctly blocks on completion of the associated “parallel” pipeline).

Is this expected behavior, or a subtle bug in pipeline dependency management?

2

sorry, I’m not sure I understand the issue. Can you please provide a sample configuration that can be used to reproduce the problem? and describe expected results vs actual results?

3

Assume we have two repos repoA and repoB with the same .drone.yml (as described above):

Expected results:

  1. repoA triggers a build, pipeline “parallel” is run on the default (non-labeled) runner
  2. repoB triggers a build, pipeline “parallel” is run in parallel with the repoA build on the default runner
  3. repoA build finishes the “parallel” pipeline and starts executing pipeline “serial” on the runner: single runner
  4. repoB build finishes the “parallel” pipeline, and queues the pipeline “serial” on the runner: single runner
  5. repoA build finishes the “serial” pipeline and repoB “serial” pipeline starts executing.

Actual results:

  1. repoA triggers a build, pipeline “parallel” is run on the default (non-labeled) runner. This actually reserves a repoA “serial” pipeline in the runner: single queue.
  2. repoB triggers a build, but the pipeline “parallel” is pending, presumably because runner: single is not available.

Would it be helpful to share the corresponding docker-compose.yml which specifies the 2 runners?

4

I do appreciate the additional detail, however, I think I am still missing something. I recommend posting a simplified yaml file that demonstrates the problem and your docker-compose file and logs (the more information the better).

This actually reserves a repoA ‘serial’ pipeline in the ‘runner: single’ queue

I am not sure I understand this phrasing. Drone does not have a reservation system. Drone has a FIFO queue of pending pipelines. When all dependencies for a pipeline are satisfied it can be pulled from the queue by a runner.

repoB triggers a build, but the pipeline “parallel” is pending, presumably because runner: single is not available.

Whether or not a runner for serial is available has no bearing on whether or not parallel pipelines are pulled from the queue by available runners and executed. I am not able to reproduce the behavior being described in this thread, as I understand it.

Here is a real world example of my tests:

Note that build 99 and 100 were able to execute the first pipeline (unlabeled) despite there being no available runner to process the second pipeline (labeled).

5

Thanks, your setup is working as expected then.

How did you trigger two builds for the same commit?

6

How did you trigger two builds for the same commit?

I re-sent the webhook from the github webhook settings screen.

7

After duplicating your example, I’m getting similar behavior, so I’ll take another look at my setup. Thank you.