[solved] Failed to pull image "drone/controller:1" due to pull rate limit

Drone Support
1

I’m using droneci in our on-premise infra using Kubernetes runner and I’ve been using a registry mirror in my pipelines using the ECR plugin’s mirror parameter and everything is working as expected except that when when new builds is happening sometimes I get this error:

Normal   Scheduled  16m                 default-scheduler                    Successfully assigned drone/drone-job-198609-9ot6qj9punizcsmwf-mcf9n to ip-10-0-12-34.ec2.internal
  Warning  Failed     16m                 kubelet, ip-10-0-12-34.ec2.internal  Failed to pull image "drone/controller:1": rpc error: code = Unknown desc = toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
  Warning  Failed     15m (x6 over 16m)   kubelet, ip-10-0-12-34.ec2.internal  Error: ImagePullBackOff

My guess is that the new pods created by drone in order to execute the build steps is being pulled from dockerhub every-single time a new runner pod is created and not utilizing the already downloaded images that’s supposedly cached on the k8s nodes itself? Note that There’s no scaling happened so no new nodes added. Is this the expected behavior? shouldn’t this pods get created using the the same images that old runners used?

2

FWIW, docker has made new pull rate limits. I’ve noticed the same many-a-times.

3

please note the drone/controller:1 image was experimental and is now deprecated. If you are using the deprecated configuration, you should consider upgrading your installation to use the official helm charts (at github.com/drone/charts) to upgrade to the kubernetes runner. Please see docs.drone.io to learn more about the kubernetes runner and kubernetes pipelines. Upgrading to the kubernetes runner will require changes to your yaml (see docs.drone.io for more details).