We face a weird issue recently, some pipelines missing node_moudles that had been installed in previous steps.
I’ve tried to add a temp folder for each step but it doesn’t work.
by the way, after restarting the pipeline this step pass with no issue. thoughts?
/drone/src is a volume that persists between steps in the same pipeline which means the node_modules folder is available to subsequent steps [1]. I created a simple test and was unable to reproduce the issue you are describing [2][3]. I can confirm there have been no changes or regressions that would prevent this from working as expected.
I have attached a screenshot below that demonstrates my successful test. I have also included links below so that you can see our sample code and configuration, and compare with your own.
The only root cause I am aware of for the issue you described would be yaml misconfiguration. We are happy to help advise, but would kindly request a sample yaml and repository that we can use to reproduce the problem (similar to the sample repository and yaml we provided at the links below).
[1] https://docs.drone.io/pipeline/docker/syntax/workspace/
[2] https://github.com/drone/hello-world/blob/dffe7eb820dc98f6682eb915dc2cd9ed09868a77/.drone.yml
[3] https://cloud.drone.io/drone/hello-world/293/1/3
It’s working for 90% of the pipelines, but some of them missing this shared folder I guess. how can I verify this folder is shared between each step?
I am not aware of any instance in which the shared folder would be missing (unless you somehow added a yaml configuration that accidentally conflicted with the shared folder). The shared folder contains the source code cloned from your git repository. If the shared folder was missing, your source code would be missing too.
If the shared folder were missing, and your source code was missing, the npm install would fail due to missing package.json file. This does not seem to be the case based on the error message in your screenshot.
yes, that makes sense. I’ll dig into the yaml configuration. can I share it here just to double-check?
yes, you can share a copy of your yaml in this forum.
this is the problematic pipeline, please let me know if you spot any issue.
kind: pipeline
type: kubernetes
name: regression
node_selector:
eks.amazonaws.com/nodegroup: automation
resources:
requests:
cpu: 7500
memory: 30Gi
trigger:
event:
- pull_request
steps:
- name: Install dev dependencies
image: node:14
commands:
- npm ci
environment:
NODE_ENV: development
GIT_PAT:
from_secret: git_pat
- name: Cancel running regressions
image: node:14
commands:
- node scripts/drone/cancel_running_regression.js
environment:
DRONE_TOKEN:
from_secret: DRONE_TOKEN
depends_on:
- Install dev dependencies
- name: Run lint
image: node:14
commands:
- npm run lint
environment:
NODE_ENV: test
depends_on:
- Cancel running regressions
- name: Run regression test
image: node:14
failure: ignore
commands:
- echo "REGRESSION_STATUS=failure" > /tmp/.env
- npm run test
- rm -rf /tmp/.env
- echo "REGRESSION_STATUS=success" > /tmp/.env
environment:
NODE_ENV: test
TEST_DATABASE_URL: postgres://xxxx:yyyy@database:5432/zzzz
volumes:
- name: env
path: /tmp
depends_on:
- Run lint
services:
- name: database
image: postgres:12-alpine
commands:
- |
docker-entrypoint.sh -c 'min_wal_size=2GB' -c 'min_wal_size=4GB' -c 'shared_buffers=4GB' -c 'effective_cache_size=4GB' -c 'wal_buffers=16MB' -c 'maintenance_work_mem=1GB' -c 'fsync=off' -c 'synchronous_commit=off' -c 'full_page_writes=off' -c 'max_connections=150' -c 'client_min_messages=warning'
environment:
POSTGRES_USER: xxxx
POSTGRES_DB: yyyy
POSTGRES_PASSWORD: zzzz
volumes:
- name: env
temp: {}
I’ve just noticed that some pipelines shared the same name, is that might be the issue?
any update here ? @bradrydzewski
@Amit_Baranes what happens if you change npm ci to npm i?
In my head it is a fresh docker container and install each time the pipeline is run
based on the docs npm-ci | npm Docs - there needs to be a package-lock.json, but I don’t think there would be a package-lock.json because it is a fresh install
apart from that is there any way there could be a jest version mismatch - this feels like more of a node dependency mismatch issue than a drone issue
there is a package-lock.json file since we run npm ci for all our pipelines. the thing is it doesn’t happen all the time. it might happen once \ twice a day or 15 times a day. also, it’s not always jest module but others as well.
as for now, I have 2 guesses -
any update here? @bradrydzewski