Similar to the DRONE_POOL_MIN_AGE setting which sets a minimum duration since an agent was created, it would be great to be able to instead define a minimum time that an agent must be idle before it gets terminated. The problem I’m looking to solve is that once an agent passes MIN_AGE, and finishes whatever job it’s running, it terminates apparently almost instantly. If another job fires a couple minutes later, it needs to wait for a new instance to spin up. While this is only a couple of minutes, it’s annoying to be iterating on a job and every 30 minutes or so you’re waiting for a new agent. Instead, we’d like to be able to set a MIN_IDLE_AGE to 15 minutes and if a job hasn’t run on that agent for 15 minutes, then it’s able to get terminated. So if jobs keep firing every 5 minutes, the agent will not terminate. Hm… if you have jobs scheduled to run or are active enough, then that might present a situation where an agent will never terminate. So I guess to go with this there should either be a MAX_AGE to hit an absolute max lifetime or even a MAX_JOBS to force a termination after it’s run a certain number of jobs.
We are working on a new VM runner that creates a new virtual machine for each pipeline execution [1]. It has built-in autoscaling with pooling and warn instances to ensure pipeline execution feels immediate. In practice this seems to be working much better than the autoscaler, and I suspect over time, the VM runner will supersede the autoscaler for this reason.
In addition to easier autoscaling, the main benefit to the VM runner is that it supports executing pipeline in containers (similar to the Docker runner) but also provides the option to execute pipelines directly on the host machine outside of a container. This is especially useful for workloads that are not well suited for container runtimes, including workloads that need to work directly with Docker (but often struggle due to issues with docker in docker, or security related to exposing the host machine docker socket).
The VM runner will launch for AWS end of year.