Drone cloud builds stuck in pending

Drone Support
1

Hi,

I’m using Drone Cloud with a Github repo:

Over the past few days, all of the builds I’ve triggered have been stuck in a pending state:

image
image1692×1528 151 KB

My last successful build was a couple months ago and I’ve made no change to .drone.yml, my Drone Cloud settings, or my Github Webhook integration. I’ve since tried removing all but the simplest steps from .drone.yml but those builds won’t exit pending either:

---
kind: pipeline
type: docker
name: default

workspace:
  base: /drone/trshcmpctr

steps:
- name: install
  image: node:16.15.1
  commands:
    - node common/scripts/install-run-rush.js install --purge

I wonder if this is a case of extra long wait times, similar to Long pending times on cloud.drone.io - #6 by bradrydzewski but at this point I’ve had builds pending for >24 hours, e.g. Drone CI
Unless there’s a particularly large queue right now, that seems less likely.

Can anyone tell me if there is such an extra-large queue at the moment? Is there anywhere that I can see the state of the Cloud Drone queue?

If that’s not the case, is there anything that might have changed underneath me since the end of September that might be locking up my pipeline like this?

2

Seems like the same issue faced at Drone Cloud ARM64 builds are hanging (not starting) since around November 8th

1 Like
3

Unfortunately cloud.drone.io has a fixed amount of resources, so if users queue a large amount of pipelines, they will block pipelines for other users.

We are trying to create more content around running your own Drone server and runners, such as Drone CI Quickstart - YouTube and Run your own Drone CI - DEV Community 👩‍💻👨‍💻.

Harness CI also offers a free tier that you can check out here.

1 Like
4

@jimsheldon @llivne Thanks for your replies! Is that definitely what’s happening? I just have no way that I can see to tell for myself.

I’d be interested in exploring a free tier of a similar service if it meant I didn’t have to contend with a public queue. What is the difference between Cloud Drone and Harness CI?

5

@jimsheldon What is considered a large amount of pipelines?

The same is true when I’ve made a single execution, is there some sort of timeout put in place before it will start working again for a single execution?

Also why the sudden change? is there a new limit you’ve put in place? can you share what said limit is?

6

@llivne @shanedg its not an issue of you as an individual user queuing up multiple builds.

The problem is that resources are shared by all users in cloud.drone.io

cloud.drone.io is used by alot of people and sadly is a target for crypto miners who regularly block up these queues, we try to stay on top of removing users / cancelling builds but it is a hugely time consuming task - personally I would recommend standing up your own server - or trying Harness CI free tier.

7

@d1wilko - builds are stuck again. Any chance you could do one round of clean-up? I’d love to run a few builds for my OSS project GitHub - oliver006/redis_exporter: Prometheus Exporter for Redis Metrics. Supports Redis 2.x, 3.x, 4.x, 5.x, 6.x, and 7.x and don’t want to migrate off of drone.io just yet.

8

Hi @oliver I think the issue here is the ARM64 builds - we have been having alot of issues with cloud drone’s donated ARM64 machines so I can only recommend standing up your own drone server or moving to Harness CI - Continuous Integration Platform: Self-Service & On-Demand

apologies :frowning:

9

Thanks for the response @d1wilko - much appreciated!

I don’t think I run arm64 builds so not sure how that would affect me.
In the meantime, I can stand up my own drone server but frankly, I’d rather not.