Kubernetes Runner Not Honoring Resource Requests

Drone Support
1

We have a build step like so

- name: "PythonUnitTests"
  image: "{IMAGE}"
  pull: "if-not-exists"
  commands:
    - "/bin/run-step.sh"
  when:
    event:
      - "push"
      - "pull_request"
  resources:
    limits:
      cpu: 3000
      memory: "5G"
    requests:
      cpu: 3000
      memory: "5G"

But, when the pod is created and the step is active the resources in the YAML are not as requested

resources:
  limits:
    cpu: "3"
    memory: 5Gi
  requests:
    cpu: 1m
    memory: 4Mi

So, as you can see - The limits are being honored but not the requests.

2

can you please provide more details (version of drone and drone runner, etc)

3

@bradrydzewski Sorry about that. We are on the following versions:

Kubernetes - 1.15
Drone Server - 1.9.2
Drone Kube Runner - latest (not sure the version, just latest tag)))

Please let me know if I missing anything else helpful

4

I recommend using drone/drone-runner-kube:1.0.0-beta.6 which is the latest tagged release. When you use the :latest version of the image you are using an unstable version that may not match the documentation. In this particular instance, the unstable version moves the resource request to the pipeline-level, instead of the step-level.

I think the new syntax looks something like this:

resources:
  requests:
    cpu: 3000
    memory: "5G"

steps:
- name: test
  image: alpine
  commands: ...
  resources:
    limits:
      cpu: 3000
      memory: "5G"
5

@bradrydzewski I see, thank you much for that info.

Curious on the change to have requests at the pipeline level. Is this going to be an option, so we can still define at the step level? Or a change?

I, personally, like the flexibility to define this at the step/container level - we do a good amount of dynamic drone configuration based on repos. We do this via the yaml configuration endpoint. While, not impossible to have this moved to a pipeline level I just like the flexibility.

6

Resource limits are still defined per-step, however, resource requests are defined per-pipeline for reasons described in this thread: [SOLVED] How to Create Linux and Windows Server Project.

I recommend reading the full thread to learn more about this history of this change, however, I think the below comment summarizes the problem we were looking to solve:

Also, even if everyone were to add per-container cpu memory/limits we’d still have a bit of “waste” because Kubernetes assumes that a pod’s sum-of cpu/memory settings are required for a node to be able to schedule the pod, while in fact most pipelines are serial so the real capacity needed at any time is simply the most “expensive” container.

7

@bradrydzewski After changing the version and having things work as expected. I fully understand the problem and the pipeline level is 100% the right way to go about this!

Thanks for the assist here, might just go back to latest and start using the pipeline level config :wink:

8

Great, glad to hear you got it working. If you decide to try out :latest with pipeline-level resource requests, we would love for you to share your feedback in this thread. Cheers!