Inconsistencies with node image and .tags

Drone Support
1

I am using the node image to create the .tags file for use in the build step with the plugins/docker image. The build works each time, but I would say about 50% of the time the .tags file is not created. I can run the same build and one time it works and the next time it does not. There are no errors in the drone or runner container logs.

Wondering if anyone else has seen this and what your workaround you came up with. Not sure if this is an issue with the node, drone, or runner container or something else.

Relevant part of .drone.yml

- name: tag
  image: node
  volumes:
  - name: cache
    path: /drone/src/
  commands:
    - echo -n "$(date +'%Y-%m-%d_%H-%M')_${DRONE_COMMIT}, latest" > .tags
    - pwd
    - ls -al
    - more .tags

Output for instance where it does not work

latest: Pulling from library/node
Digest: sha256:3f8047ded7bb8e217a879e2d7aabe23d40ed7f975939a384a0f111cc041ea2ed
Status: Image is up to date for node:latest
+ echo -n "$(date +'%Y-%m-%d_%H-%M')_8b2fc1cb2094c8c9cd481676cc58e1655e4ddab3, latest" > .tags
+ pwd
/drone/src
+ ls -al
total 796
drwxr-xr-x 11 root root   4096 Apr 14 15:12 .
drwxr-xr-x  3 root root   4096 Apr 14 15:12 ..
-rw-r--r--  1 root root    429 Apr 14 15:12 .browserslistrc
-rw-r--r--  1 root root    245 Apr 14 15:12 .editorconfig
-rw-r--r--  1 root root    379 Apr 14 15:12 .eslintrc.json
drwxr-xr-x  8 root root   4096 Apr 14 15:12 .git
drwxr-xr-x  4 root root   4096 Apr 14 15:12 .github
-rw-r--r--  1 root root   1285 Apr 14 15:12 .gitignore
-rw-r--r--  1 root root     65 Apr 14 15:12 .tags
drwxr-xr-x  2 root root   4096 Apr 14 15:12 .vscode
-rw-r--r--  1 root root   1067 Apr 14 15:12 Dockerfile
-rw-r--r--  1 root root   1056 Apr 14 15:12 LICENSE.md
-rw-r--r--  1 root root     31 Apr 14 15:12 Procfile
-rw-r--r--  1 root root  60631 Apr 14 15:12 Public API v1.yaml
-rw-r--r--  1 root root   9001 Apr 14 15:12 README.md
-rw-r--r--  1 root root   5807 Apr 14 15:12 angular.json
-rw-r--r--  1 root root    341 Apr 14 15:12 app.json
-rw-r--r--  1 root root   1208 Apr 14 15:12 armhf.Dockerfile
drwxr-xr-x  8 root root   4096 Apr 14 15:12 backend
drwxr-xr-x  3 root root   4096 Apr 14 15:12 chart
drwxr-xr-x  4 root root   4096 Apr 14 15:12 chrome-extension
-rw-r--r--  1 root root   1704 Apr 14 15:12 chrome-extension.pem
-rw-r--r--  1 root root    796 Apr 14 15:12 docker-compose.yml
drwxr-xr-x  2 root root   4096 Apr 14 15:12 e2e
-rw-r--r--  1 root root    963 Apr 14 15:12 karma.conf.js
-rw-r--r--  1 root root    836 Apr 14 15:12 main.js
-rw-r--r--  1 root root 599023 Apr 14 15:12 package-lock.json
-rw-r--r--  1 root root   2588 Apr 14 15:12 package.json
-rw-r--r--  1 root root    722 Apr 14 15:12 protractor.conf.js
drwxr-xr-x  2 root root   4096 Apr 14 15:12 releases
drwxr-xr-x  6 root root   4096 Apr 14 15:12 src
-rw-r--r--  1 root root    577 Apr 14 15:12 tsconfig.json
-rw-r--r--  1 root root   2943 Apr 14 15:12 tslint.json
+ more .tags
::::::::::::::
.tags
::::::::::::::
Exit Code 0

Output for instance where it does work

latest: Pulling from library/node
Digest: sha256:3f8047ded7bb8e217a879e2d7aabe23d40ed7f975939a384a0f111cc041ea2ed
Status: Image is up to date for node:latest
+ echo -n "$(date +'%Y-%m-%d_%H-%M')_10c52801b81f4feef2bf2df594802f923ce42035, latest" > .tags
+ pwd
/drone/src
+ ls
Dockerfile
LICENSE.md
Procfile
Public API v1.yaml
README.md
angular.json
app.json
armhf.Dockerfile
backend
chart
chrome-extension
chrome-extension.pem
docker-compose.yml
e2e
karma.conf.js
main.js
package-lock.json
package.json
protractor.conf.js
releases
src
tsconfig.json
tslint.json
+ more .tags
::::::::::::::
.tags
::::::::::::::
2022-04-14_06-30_10c52801b81f4feef2bf2df594802f923ce42035, latest
2

Hello @samcro1967

This is definitely strange behavior. Is there any chance that when the .tags file isn’t written, it is a different type of event? Perhaps a promotion event, or something else, where the environment is slightly different?

My only other thought is maybe the steps are going so quickly that the file hasn’t actually been written by the time the next step runs? Maybe try running the sync command immediately after you create the .tags file?

Linux sync command help and examples.

3

I added sync just after the echo command. That had no effect. The whole task only takes 2 seconds. I added a sleep 5 before and after the sync and that had no effect either. I have this same step in 11 repos all having the same symptoms.

I am guessing this is an issue with the node container. I will try a different container and see if that has any impact when I get some time.

4

I switched to an ubuntu container and experience the same results. I am able to run the echo command without piping to the .tags file and it displays to the console as expected. I also added touch .tags and the file is created before the echo.

I switched to “tags: ${DRONE_COMMIT} , latest” in the build step and removed the tag step. That works consistently. Would prefer a date as the tag, but better than nothing.

5 
  - name: cache
    path: /drone/src/

The above snippet in your yaml is the problem and should be removed. Drone automatically creates a volume at /drone/src and mounts into every pipeline step. You are creating a new volume at the same path, and it is clashing with the Drone volume which is leading to unepected behavior.

You can learn more about the workspace volume in the Drone docs:

Drone automatically creates a temporary volume, known as your workspace, where it clones your repository. The workspace is the current working directory for each step in your pipeline. The default workspace path is /drone/src .

6

My bad. Removing it has no impact. It was not in there to begin. I added it trying different things to troubleshoot and get it to work consistently. Below is what I believe should work with the tag step uncommented and the tag setting in the build step commented out.

kind: pipeline
type: docker
name: mailrise

settings:
  insecure: true

clone:
  disable: true

trigger:
  branch:
  - master

steps:
- name: clone
  image: local.registry:5000/local/alpine_git
  commands:
  - git clone https://github.com/YoRyan/mailrise.git /drone/src/
  - ls /drone/src/
#- name: tag
#  image: node
#  commands:
#    - echo -n "$(date +'%Y-%m-%d_%H-%M')_${DRONE_COMMIT}, latest" > .tags
#    - pwd
#    - ls -al
#    - more .tags
- name: build
  privileged: true
  image: plugins/docker
  settings:
    repo: local.registry:5000/local/mailrise
    registry: local.registry:5000
    insecure: true
    tags: ${DRONE_COMMIT} , latest # comment out if using tag step
- name: notify
  pull: if-not-exists
  image: drillster/drone-email
  settings:
    host: 192.168.1.104
    port: 8025
    skip_verify: true
    from: alerts@local.net
    recipients: 
      - alerts@local.net
    recipients_only: true
    subject: >
      [{{ build.status }}]
      {{ repo.owner }}/{{ repo.name }}
      ({{ build.branch }} - {{ truncate build.commit 8 }})      
    body:
      http://local.logs:8069/drone/template.html
  when:
    status:
      include:
        - success
        - changed
        - failure
7

The problem I see with the yaml you provided is that you define tags in the yaml. The values in the yaml take precedence over values defined in the file. The file is therefore ignored.

I also ran a quick test that you can use as a reference, with screenshots provided below. Note that we write a tag value to the file and the file is used by the subsequent step to tag the image. I am unable to reproduce any issues and have not received any other reports of issues with regards to this feature (which has been around for a few years now).

Screen Shot 2022-04-25 at 3.38.21 PM
Screen Shot 2022-04-25 at 3.38.21 PM2470×1058 196 KB

Screen Shot 2022-04-25 at 3.37.56 PM
Screen Shot 2022-04-25 at 3.37.56 PM3040×1380 484 KB

You can audit the yaml and the results here:

8

I am only specifying the tags in the build step when the tag step is commented out. Never doing both at the same time. If I test the tags step, I comment out the tags in the build step.

The issue I am having is that the file never gets created in the tags step, not that is is created and is not being read. If I add an ls to the tag step, the file is not listed there, but the files from the repo I cloned are.

9

I have been able to get the file to be created reliably every time by putting it in a while loop to check to see if it exists and if not create it. I am guessing it is some type of timing issue. The more of the .tags file works about 50% of the time I would say. Sometimes it shows nothing and other times it shows the contents of the file. When it does not show the contents of the file, the image is properly tagged with the contents of the file. Guessing it moves to the next step before it successfully captures the output of the command to the log. I am now using the alpine image, but I would guess this will also work in the node and ubuntu images as well.

- name: tag
  image: alpine
  commands:
    - while [ ! -f .tags ]; do echo -n "$(date +'%Y-%m-%d_%H-%M'), latest" > .tags; done;
    - ls -al /drone/src/
    - more .tags