Exit 78 does not work for all steps

Drone Support
1

I’m on a kubernetes pipeline deployed via helm, using latest versions and images.

I have a step that might perform an exit 78 as the first one of a series.

On pipelines where there’s just an additional step after the potential exit 78 it’s fine, the job will be skipped.

If the pipeline has more steps after the one that should be skipped they will get executed and will not show up in the UI properly.


in the examples above check-and-tag is the method that might return exit 78.
In the rails build screenshot as you can see restore-cache is skipped, but the rest is going on in the background.

Said step source is:

  - name: check-and-tag
    image: alpine/git
    commands:
    - if (git diff --exit-code $DRONE_COMMIT_BEFORE $DRONE_COMMIT_AFTER -- rails); then exit 78; fi
    - export CUSTOM_BRANCH_NAME=$(basename "${DRONE_SOURCE_BRANCH}" | tr '[:upper:]' '[:lower:]' | sed 's/_/-/g')
    - echo -n "$CUSTOM_BRANCH_NAME-$SHORT_SHA" > .tags
    environment:
      SHORT_SHA: "${DRONE_COMMIT_SHA:0:8}"
2

I tested with the below yaml and was unable to reproduce any issues:

Perhaps you can also provide a simple yaml that can be used to reproduce, or let me know how I need to adjust the above yaml to reproduce the issue.

3

I’m not sure how to reproduce, something that comes to mind:

  • I have some services in the rails pipeline (the one with more steps)
  • I have more pipelines in the same drone.yml

I do not use failure: ignore or other commands to parallelize pipeline steps.

In this example other-step1, other-step2, other-step3 etc will get executed anyway if rails check-and-tag exits 78, although it will not show in the UI.

---
kind: pipeline
type: kubernetes
name: artchecker

steps:
  - name: check-and-tag
    image: alpine/git
    commands:
    - if (git diff --exit-code $DRONE_COMMIT_BEFORE $DRONE_COMMIT_AFTER -- artchecker); then exit 78; fi
    - export CUSTOM_BRANCH_NAME=$(basename "${DRONE_SOURCE_BRANCH}" | tr '[:upper:]' '[:lower:]' | sed 's/_/-/g')
    - echo -n "$CUSTOM_BRANCH_NAME-$SHORT_SHA" > .tags
    environment:
      SHORT_SHA: "${DRONE_COMMIT_SHA:0:8}"
  - name: build-and-push
    image: plugins/docker
    settings:
      context: artchecker
      use_cache: true
      dockerfile: artchecker/Dockerfile-pipelines

---
kind: pipeline
type: kubernetes
name: rails

services:
  - name: redis
    image: redis:alpine
    ports:
    - 6379

  - name: fakes3
    image: adobe/s3mock:2.1.18
    ports:
    - 9090
    environment:
      initialBuckets:uploads-test

  - name: pg
    image: bitnami/postgresql:11
    environment:
      POSTGRESQL_DATABASE: myapp_test
      POSTGRESQL_USERNAME: test_user
      POSTGRESQL_PASSWORD: test_user_password
    ports:
    - 5432

steps:
  - name: check-and-tag
    image: alpine/git
    commands:
    - if (git diff --exit-code $DRONE_COMMIT_BEFORE $DRONE_COMMIT_AFTER -- rails); then exit 78; fi
    - export CUSTOM_BRANCH_NAME=$(basename "${DRONE_SOURCE_BRANCH}" | tr '[:upper:]' '[:lower:]' | sed 's/_/-/g')
    - echo -n "$CUSTOM_BRANCH_NAME-$SHORT_SHA" > .tags
    environment:
      SHORT_SHA: "${DRONE_COMMIT_SHA:0:8}"
  - name: restore-cache
  - name: other-step1
  - name: other-step2
  - name: other-step3
4

Did you install using our official helm chart at drone/charts ? I ask because some of the syntax in your example is from an experimental, deprecated kubernetes runtime. If you are using our official helm chart, please note that this chart does not install the :latest runner image, so you may be missing some newer features or bug fixes.

5

I deploy via Flux / Helm Operator, this is my releases files (relevant bits):

apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
  name: drone
  namespace: drone
  annotations:
    fluxcd.io/ignore: "false"
spec:
  releaseName: drone
  chart:
    git: https://github.com/drone/charts.git
    ref: master
    path: charts/drone
  values:
    image:
      tag: 1.9.1
   ....

apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
  name: drone-runner-kube
  namespace: drone
  annotations:
    fluxcd.io/ignore: "false"
spec:
  releaseName: drone-runner-kube
  chart:
    git: https://github.com/drone/charts.git
    ref: master
    path: charts/drone-runner-kube
  values:
    image:
      tag: 1.0.0-beta.4
    env:
      DRONE_NAMESPACE_DEFAULT: drone-builds
      DRONE_RPC_SECRET: edited
    rbac:
      buildNamespaces:
        - drone-builds
6

Please test with drone/drone-runner-kube:latest to ensure you are testing with the latest version of the software, and that we are not triaging something that has already been fixed. As this is beta software, the codebase evolves quickly, and it is very possible it was an issue previously, and has since been resolved.

7

unfortunately can’t test right away since Flux does not support :latest kind of images https://docs.fluxcd.io/en/1.18.0/references/automated-image-update.html#automated-deployment-of-new-container-images

I’m gonna remove the HelmRelease and attempt a manual deploy via Helm, I’ll let you know, thanks.

8

unless you want to push out a beta5 :sweat_smile:

9

similar Using exit code 78 with Kubernetes runner doesn't stop execution

10

yes, I believe this was fixed in master since beta.4 was last released.

11

I just tried without services and it’s working - I’ll test latest then, thanks

12

Let me know how you get on, as it didn’t work for me on beta.4. If it’s fixed in latest I can update and try again. I’m the author of the linked topic (Using exit code 78 with Kubernetes runner doesn't stop execution) and still keen to find a solution.

13

For reference, the underlying code to handle exit 78 comes from this shared library and is used by all runners (docker, kubernetes, etc). You can further explore the code here:

14

I posted an update in my other thread - this issue with exit 78 not working when the pipeline contains a services: declaration seems to be common to both the Docker and Kubernetes runners.

15

drone runner kube beta 5 resolved the issue, thanks!