It looks like there is no way to skip queued build.
$ drone build stop go-reform/reform 6
client error 400: Cannot cancel a non-running build
Additionally, it would be cool to have something like AppVeyor’s rolling builds.
2 Likes
Also it looks like it is not possible to cancel build jobs with CLI:
$ drone build stop go-reform/reform 6
Stopping build go-reform/reform#6.1
$ drone build stop go-reform/reform 6
client error 400: Cannot cancel a non-running build
$ drone build stop go-reform/reform 6.2
strconv.ParseInt: parsing "6.2": invalid syntax
It is not possible to cancel pending builds at this time (no way to dequeue a queued item)
This feature would be amazing to have in Drone. We’re trying to optimize our CI infrastructure to maximize performance and thus minimize each job’s duration. If somebody pushes several times to their PR - we would like to cancel pending builds for that PR and only run the latest job (matrix build ==> set of jobs).
1 Like
This is a really important feature. Is it any chance we can get it soon? Cancelling erroneous pending tasks not only helps when optimizing CI execution, but when debugging Drone misbehaviours.
1 Like
@mikeifomin if you have jobs stuck in pending, a Drone administrator can use the drone kill command to force cancel them.
1 Like
Is It ok that if a drone-server has an available and free drone-worker it doesn’t trying to seek for pending jobs and then run?
If your just have not time to implement it, I can make a PR.
Is It ok that if a drone-server has an available and free drone-worker it doesn’t trying to seek for pending jobs and then run?
No, that would not be normal behavior.
Agents connect to the server and pull jobs from a fifo queue. Jobs are always distributed immediately to available agents. So if jobs are sitting in a pending state, it would typically signal that your agents are not connecting, or are being force disconnected, from the server. The most common root cause we see for network connectivity issues is when someone puts a load balancer or reverse proxy between the server and agents that is killing connections, since load balancers and grpc do not play nice together.
You can enable DRONE_DEBUG=true to troubleshoot agent connectivity issues.
Even I have some network issues I’ve expected that an agent will handle pending jobs as soon as connection established.
But in my case some jobs are still in the pending state. New builds are going well, but the old ones all remain stuck.
The screenshot from my first message shows this case.
As I understand there should not be pending jobs if agent connected.
It’s not the first time I’ve faced with.
This could be a bug.
The only known way for a job to get stuck in pending is for the agent to pull the job and then lose connectivity, via restart or a fatal network error which prevents it from reconnecting, before it has the chance to update the status to “started”.
If there is another scenario that can cause builds to be stuck in pending, I am not currently aware of it. I would certainly encourage you to jump into the code and send a patch if you think you might have uncovered a bug. If you need help triaging, we also have enterprise support options available.
1 Like
Thanks for the detailed explanation, I will continue to dig.
If a clone have a condition and the current push don’t go in this condition the build is stuck in pending :
Ex :
I pushed on master and now one of build is stuck in pending. I try to kill it from the cli but got a client error 400: Cannot force cancel a non-running build
It would be lovely if this is fixed. I cannot kill pending builds, and I have many in my servers…
We also have several very old Pending builds, cannot do anything with them right now. Any advice? I would just like to clean them up. They are not getting picked up by agents while new builds are.
This will be fixed in version 0.9 but in the mean time, sql is your friend:
UPDATE builds
SET build_status = 'error'
WHERE build_status = 'pending'
AND build_created < 1519862400
Note that the build_created date is a unix timestamp. You can generate one here: https://www.unixtimestamp.com/index.php
1 Like
You also might want to update individual steps to failed status for sanity sake.
Thanks, this worked. For future reference, I ran sqlite3 /var/lib/drone/drone.sqlite and copy-pasted that command + an ending ‘;’.
1 Like