# Debugging concurrent steps not executing concurrently

**URL:** <https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661>\
**Category:** Drone Support\
**Created:** [April 8, 2019, 9:33am UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661 "2019-04-08T09:33:22Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![rikonor](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/rikonor/32/5662_2.png) [@rikonor](https://drone.discourse.group/u/rikonor)\
**Post date:** [April 8, 2019, 9:33am UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661/1 "2019-04-08T09:33:22Z")

</div>

Hi,

We’re running into a bit of a confusing situation with our Drone builds not running concurrently for some of the steps (the ones that we’d actually care about running concurrently).

We have a pipeline that looks similar to this example:

```
workspace:
  base: /go
  path: src/<path>

pipeline:
  setup:
    image: <image>
    secrets:
      - GITHUB_SSH_PRIVATE_KEY
    commands:
      - echo "--> Setup..."

  test:
    group: pre-build
    image: <image>
    commands:
      - go test ./... -v

  vet:
    group: pre-build
    image: <image>
    commands:
      - go vet ./...

  service-1-binary:
    group: binaries
    image: <image>
    commands:
      - go build -o cmd/service-1/service-1 ./cmd/service-1

  service-1-image:
    group: docker-images
    image: plugins/docker
    registry: <registry>
    repo: <registry>/service-1
    tags:
      - ${DRONE_BUILD_NUMBER}
      - ${DRONE_COMMIT_SHA:0:8}
    dockerfile: cmd/service-1/Dockerfile
    context: cmd/service-1
    build_args:
      - BASE_IMAGE_REPO=<image>
      - BASE_IMAGE_TAG=<tag>
    custom_dns: <ip1>, <ip2>
    secrets:
      - DOCKER_USERNAME
      - DOCKER_PASSWORD
    debug: true

  service-2-binary:
    group: binaries
    image: <image>
    commands:
      - go build -o cmd/service-2/service-2 ./cmd/service-2

  service-2-image:
    group: docker-images
    image: plugins/docker
    registry: <registry>
    repo: <registry>/service-2
    tags:
      - ${DRONE_BUILD_NUMBER}
      - ${DRONE_COMMIT_SHA:0:8}
    dockerfile: cmd/service-2/Dockerfile
    context: cmd/service-2
    build_args:
      - BASE_IMAGE_REPO=<image>
      - BASE_IMAGE_TAG=<tag>
    custom_dns: <ip1>, <ip2>
    secrets:
      - DOCKER_USERNAME
      - DOCKER_PASSWORD
    debug: true

```

In the above, only the `pre-build` group run concurrently, but the `binaries` and `docker-images` groups do not run concurrently.

Ideally, we’d want the `binaries` group to run first, and then the `docker-images` group to run. Any idea why this is not happening? Does it have something to do with the ordering of the steps? Do you have to co-locate the groups, i.e `binaries` have to all be right after the other? Same with `docker-images`?

Thanks!  
Or

---

<div class="post-metadata">

**Author:** ![devster](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/devster/32/4770_2.png) [@devster](https://drone.discourse.group/u/devster)\
**Post date:** [April 8, 2019, 10:01am UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661/2 "2019-04-08T10:01:33Z")

</div>

Does this page help? [https://docs.drone.io/user-guide/pipeline/multi-machine/#dependencies](https://docs.drone.io/user-guide/pipeline/multi-machine/#dependencies)  
I believe with `1` you need to declare multiple pipelines and use `depends_on` to specify execution order.

---

<div class="post-metadata">

**Author:** ![rikonor](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/rikonor/32/5662_2.png) [@rikonor](https://drone.discourse.group/u/rikonor)\
**Post date:** [April 8, 2019, 10:38am UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661/3 "2019-04-08T10:38:42Z")

</div>

Thanks @devstar! That looks really interesting. However, what’s this mention of parallel builds on that page?

> Please be warned that most projects do not require multi-machine pipelines. You should exhaust every other option, including parallel pipeline steps and more powerful machines.

I’m having a bit of trouble finding parallel pipeline steps in the 1.0.0 docs, is that something that was removed?

---

<div class="post-metadata">

**Author:** ![devster](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/devster/32/4770_2.png) [@devster](https://drone.discourse.group/u/devster)\
**Post date:** [April 8, 2019, 11:10am UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661/4 "2019-04-08T11:10:09Z")

</div>

In the section about dependencies it states:

> In the below example, the first two pipelines execute in parallel. On successful completion a third pipeline is executed that sends a Slack notification indicating the workflow is complete.

I’m assuming it means each declared pipeline will be executed concurrently only respecting the dependency specified.  
So, `steps` are sequential, `pipelines` are parallel by default it seems.

---

<div class="post-metadata">

**Author:** ![rikonor](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/rikonor/32/5662_2.png) [@rikonor](https://drone.discourse.group/u/rikonor)\
**Post date:** [April 8, 2019, 11:14am UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661/5 "2019-04-08T11:14:07Z")

</div>

> So, `steps` are sequential, `pipelines` are parallel by default it seems.

Yup, I’m not disagreeing. I’m just trying to understand if the `group` functionality that existed in 0.8.0 still exists in 1.0.0 and if so - where is it documented. I don’t have much control over our Drone deployment, therefore I’m not even sure whether there are multiple Drone agents running on multiple machines. So ideally, if I could use `group` to cause multiple steps to run concurrently, that’d be the simplest solution for us at this moment.

---

<div class="post-metadata">

**Author:** ![devster](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/devster/32/4770_2.png) [@devster](https://drone.discourse.group/u/devster)\
**Post date:** [April 8, 2019, 11:55am UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661/6 "2019-04-08T11:55:06Z")

</div>

I don’t think you can replicate the behaviour using `groups` in `1.0.0`, someone more knowledgeable than me might correct me in case I’m wrong. However, you should be able to achieve the same results rewriting the `.drone.yml` for the new version, without changing the deployment.

```auto
---
kind: pipeline
name: test

steps:
- name: test
  image: golang
  commands:
  - go build
  - go test ./... -v

---
kind: pipeline
name: vet

steps:
- name: vet
  image: golang
  commands:
  - go vet ./...

---
kind: pipeline
name: service-1-binary

steps:
- name: binary 1
  image: plugins/docker
  registry: <registry>
  repo: <registry>/service-1
  commands:
    - go build -o cmd/service-1/service-1 ./cmd/service-1

depends_on:
- test
- vet

---
kind: pipeline
name: service-1-image

steps:
- name: image 1
  image: plugins/docker
  registry: <registry>
  repo: <registry>/service-1
  tags:
  - ${DRONE_BUILD_NUMBER}
  - ${DRONE_COMMIT_SHA:0:8}
  dockerfile: cmd/service-1/Dockerfile
  context: cmd/service-1
  build_args:
  - BASE_IMAGE_REPO=<image>
  - BASE_IMAGE_TAG=<tag>
  custom_dns: <ip1>, <ip2>
  secrets:
  - DOCKER_USERNAME
  - DOCKER_PASSWORD

depends_on:
- service-1-binary

```

I’m not sure how `workspace` was translated, I believe you need to share volumes if you want persistance between pipelines.

---

<div class="post-metadata">

**Author:** ![rikonor](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/rikonor/32/5662_2.png) [@rikonor](https://drone.discourse.group/u/rikonor)\
**Post date:** [April 8, 2019, 2:08pm UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661/7 "2019-04-08T14:08:49Z")

</div>

Sounds good. Thanks for the help @devstar. I’m going to hold-off for a bit and see if anyone else will comment on this to confirm it’s not possible with 1.0.0 and that the new format needs to be used (I mean the 1.0.0 docs do mention parallel pipeline steps, so I’d be curious to know if that was accidental or whether it really is part of 1.0.0).

---

<div class="post-metadata">

**Author:** ![bradrydzewski](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/bradrydzewski/32/3513_2.png) [@bradrydzewski](https://drone.discourse.group/u/bradrydzewski)\
**Post date:** [April 8, 2019, 2:13pm UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661/8 "2019-04-08T14:13:19Z")

</div>

1.0 supports concurrent steps. See [How to setup parallel Pipeline steps (1.0+)](http://discuss.harness.io/t/how-to-setup-parallel-pipeline-steps-1-0/3251)

---

<div class="post-metadata">

**Author:** ![rikonor](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/rikonor/32/5662_2.png) [@rikonor](https://drone.discourse.group/u/rikonor)\
**Post date:** [April 9, 2019, 10:02am UTC](https://drone.discourse.group/t/debugging-concurrent-steps-not-executing-concurrently/9661/9 "2019-04-09T10:02:49Z")

</div>

Thanks @bradrydzewski, I was able to migrate our Drone file to the 1.0.0 format and get the relevant steps to run concurrently (pretty happy - it cut our build from 10 minutes to about 3 minutes).

I’m sure you already know this - but it’d have been really helpful if this was already in the docs.

Thanks again and congrats on the 1.0.0 release!