# \[K8s runner\] resource requests failing to work **URL:** \ **Category:** Drone Support\ **Created:** [April 21, 2021, 7:04am UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997 "2021-04-21T07:04:45Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![gzatko](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/gzatko/32/3304_2.png) [@gzatko](https://drone.discourse.group/u/gzatko)\ **Post date:** [April 21, 2021, 7:04am UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/1 "2021-04-21T07:04:45Z")
I can confirm that resource limits work as [described](https://docs.drone.io/pipeline/kubernetes/syntax/steps/#resources). However requests seem not to. An example: 1. the runner deployment has following env variables set to initialize containers with default values: ```auto DRONE_RESOURCE_LIMIT_CPU = 250 DRONE_RESOURCE_LIMIT_MEMORY = "256MiB" DRONE_RESOURCE_REQUEST_CPU = 50 DRONE_RESOURCE_REQUEST_MEMORY = "64MiB" ``` 1. I have a step with overriden requests and limits: ```auto resources: limits: cpu: 4000 memory: 6GiB requests: cpu: 2000 memory: 4GiB ``` 1. in a pipeline with 2 steps (1 default settings, 1 overriden) we can check that limits are both ok but requests both wrong: ```auto kubectl -n tools-drone get pods drone-crc8zqp3p03tgy3t910x -o=jsonpath='{.spec.containers[0].resources}' {"limits":{"cpu":"4","memory":"6Gi"},"requests":{"cpu":"48m","memory":"56Mi"}} ``` ```auto kubectl -n tools-drone get pods drone-crc8zqp3p03tgy3t910x -o=jsonpath='{.spec.containers[1].resources}' {"limits":{"cpu":"250m","memory":"256Mi"},"requests":{"cpu":"1m","memory":"4Mi"}} ``` Is this really a bug or just my misconfiguration? In the bug case should I file a ticket somewhere else? ---
**Author:** ![Kash](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/kash/32/4921_2.png) [@Kash](https://drone.discourse.group/u/Kash)\ **Post date:** [April 21, 2021, 10:05am UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/2 "2021-04-21T10:05:08Z")
Hi @gzatko, from what I understand this is now expected behaviour from the recent runner changes (although not desired in our case). We hit the same issue and my colleague documented this with a few different scenarios and settings: [[drone-runner-kube] Not correctly setting resource requests for Steps - #5 by onur](http://discuss.harness.io/t/drone-runner-kube-not-correctly-setting-resource-requests-for-steps/8936/5) ---
**Author:** ![gzatko](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/gzatko/32/3304_2.png) [@gzatko](https://drone.discourse.group/u/gzatko)\ **Post date:** [April 21, 2021, 1:09pm UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/3 "2021-04-21T13:09:07Z")
Thanks for the thread - I overlooked it somehow in search. For me it’s important that Docker image 1.0.0-beta.6 enabled to set request per pod. ---
**Author:** ![jim5252](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/jim5252/32/6405_2.png) [@jim5252](https://drone.discourse.group/u/jim5252)\ **Post date:** [September 17, 2021, 3:45pm UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/4 "2021-09-17T15:45:33Z")
Hi, We have recently updated our runner to version [1.0.0-beta.11](https://hub.docker.com/layers/drone/drone-runner-kube/1.0.0-beta.11/images/sha256-c9ffcfa9e63bca87f23da7ed09123c2fc539f8a0db562b70938e679578ea414b?context=explore) We still seem to be experiencing the same issue as stated before. There appears to be no way to set the resource requests in the drone file. When working with a drone.yaml example at: [Resources | Drone](https://docs.drone.io/runner/kubernetes/configuration/resources/) When I check the pods each container will have an equal share of ```auto requests: cpu: 100m memory: 100Mi ``` This allocation will be divided by: `no. of steps +1 (assuming for the clone step)` When using a .drone.star file we are able to set the request if the steps request is less than the default values that appear to be above. However if we attempt to set the value to anything higher in say a 1 step drone file, it will accept the limit but the the maximum request allowance is blocked. Here is the drone.star: ```auto def main(ctx): return { "name": "build", "kind": "pipeline", "type": "kubernetes", "steps": [ { # Step 2, requests are lower than a MIN request amount "name": "stage-1", "image": "alpine", "resources": { "requests": { "cpu": 1000, "memory": "2000Mib" }, "limits": { "cpu": 1000, "memory": "2000Mib" } }, "commands": [ "sleep 60" ] } ] } ``` Here is the resources in the stage 1 container: ```auto resources: limits: cpu: "1" memory: 2000Mi requests: cpu: 50m memory: 48Mi ``` Is this still a known problem or can I be advised if there is anything we can change in the configuration to allocate a higher request in our drone steps? ---
**Author:** ![bradrydzewski](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/bradrydzewski/32/3513_2.png) [@bradrydzewski](https://drone.discourse.group/u/bradrydzewski)\ **Post date:** [September 18, 2021, 1:32pm UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/5 "2021-09-18T13:32:23Z")
Resource requests are defined at pipeline level, not at the step level. See [http://discuss.harness.io/t/kubernetes-runner-latest-ignoring-step-resource-requests/9392/2](http://discuss.harness.io/t/kubernetes-runner-latest-ignoring-step-resource-requests/9392/2) and [http://discuss.harness.io/t/strategy-for-pod-cpu-settings-on-kubernetes-runner/7813/4](http://discuss.harness.io/t/strategy-for-pod-cpu-settings-on-kubernetes-runner/7813/4) ---
**Author:** ![jim5252](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/jim5252/32/6405_2.png) [@jim5252](https://drone.discourse.group/u/jim5252)\ **Post date:** [September 20, 2021, 10:27am UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/6 "2021-09-20T10:27:47Z")
Even setting at pipeline does not make a difference: ```auto kind: pipeline type: kubernetes resources: request: cpu: 2000 memory: 2000MiB steps: - name: en image: alpine resources: limit: cpu: 1000 memory: 1000MiB commands: - echo hello - name: es image: alpine commands: - echo hola - name: fr image: alpine commands: - echo bonjour ``` container values: ```auto -- resources: requests: cpu: 25m memory: 28Mi securityContext: -- resources: requests: cpu: 25m memory: 24Mi securityContext: -- resources: requests: cpu: 25m memory: 24Mi securityContext: -- resources: requests: cpu: 25m memory: 24Mi securityContext: ``` The values should be 50 and the limit should have been set in the `en` container according to [Strategy for pod/cpu settings on kubernetes runner - #3 by trondhindenes](http://discuss.harness.io/t/strategy-for-pod-cpu-settings-on-kubernetes-runner/7813/3) ---
**Author:** ![jim5252](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/jim5252/32/6405_2.png) [@jim5252](https://drone.discourse.group/u/jim5252)\ **Post date:** [September 21, 2021, 8:39am UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/7 "2021-09-21T08:39:34Z")
Hi @bradrydzewski , do you have any further input from my reply? ---
**Author:** ![bradrydzewski](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/bradrydzewski/32/3513_2.png) [@bradrydzewski](https://drone.discourse.group/u/bradrydzewski)\ **Post date:** [September 21, 2021, 4:25pm UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/8 "2021-09-21T16:25:47Z")
I think you need to use `requests` and `limits` (plural instead of singular) I tested with the following yaml ```auto kind: pipeline type: kubernetes resources: requests: cpu: 2000 memory: 2000MiB steps: - name: en image: alpine commands: - echo hello resources: limits: cpu: 1000 memory: 1000MiB - name: es image: alpine commands: - echo hola ``` I then used the below command to generate a kubernetes spec file. _Note that this is not meant to be a usable spec, it just shows us how data from the yaml might be mapped to Kubernetes to help debug_. ```nohighlight drone-runner-kube compile --spec ``` We see that each step is assigned 668Mi of memory which is 2000MiB divided by 3 steps (2 steps defined in the yaml, as well as the implicit clone step) ```auto resources: requests: cpu: 667m memory: 668Mi ``` ---
**Author:** ![jim5252](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/jim5252/32/6405_2.png) [@jim5252](https://drone.discourse.group/u/jim5252)\ **Post date:** [September 23, 2021, 10:20am UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/9 "2021-09-23T10:20:38Z")
That works more as expected @bradrydzewski , thank you. ```auto -- resources: requests: cpu: 500m memory: 500Mi -- resources: limits: cpu: "1" memory: 1000Mi requests: cpu: 500m memory: 500Mi -- resources: requests: cpu: 500m memory: 500Mi -- resources: requests: cpu: 500m memory: 500Mi ``` ---
**Author:** ![beniamin](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/beniamin/32/3078_2.png) [@beniamin](https://drone.discourse.group/u/beniamin)\ **Post date:** [October 16, 2021, 5:30pm UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/10 "2021-10-16T17:30:27Z")
I encountered the issue described above with `limits`. I have set default resources limits using environment variables. It seems that values set in each step it’s ignored. I was trying to set limits at the stage level, but I receive the following error: `msg="cannot accept configuration" build.id=1152 build.number=404 error="linter: cpu limit cannot be applied at stage level"`. I am running 1.0.0-beta12 with following environment variables: ```auto DRONE_RESOURCE_LIMIT_CPU: 500 DRONE_RESOURCE_LIMIT_MEMORY: 512MiB ``` My pipeline: ```auto kind: pipeline name: publish type: kubernetes resources: limits: cpu: 2000 memory: "1Gi" steps: - name: install image: golang:1.16.2-buster resources: limits: cpu: 2000 memory: "1Gi" commands: - go mod vendor depends_on: - clone ``` Removing the resources from the stage level, the pipeline manage to run but the containers have the values from below: ```bash beniamin@macbook ~ % kubectl -n content-ci get pods drone-b8z6z7oo5qlqam9w4gc4 -o=jsonpath='{.spec.containers[0].resources}' {"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"20m","memory":"28Mi"}}% beniamin.calota@Beniamins-MacBook ~ % kubectl -n content-ci get pods drone-b8z6z7oo5qlqam9w4gc4 -o=jsonpath='{.spec.containers[1].resources}' {"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"20m","memory":"28Mi"}}% ``` Is there anything I can do, besides setting a very high value from environment and overriding with whatever I need from the step? I have to mention that the cluster administrator does not allow to run containers without limits set, so the the remove of the environment variable it’s not an option for me. ---
**Author:** ![bradrydzewski](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/bradrydzewski/32/3513_2.png) [@bradrydzewski](https://drone.discourse.group/u/bradrydzewski)\ **Post date:** [October 16, 2021, 5:53pm UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/11 "2021-10-16T17:53:16Z")
You need to define resource requests at the pipeline level and resource limits at the step level. This (along with using plural vs singular spelling) solved the issue for the previous individual that created this thread. You can see an example in [this comment](http://discuss.harness.io/t/k8s-runner-resource-requests-failing-to-work/9037/8). I also think the below section from the [documentation](https://docs.drone.io/runner/kubernetes/configuration/resources/) may be relevant to your issue. The global resource limits defined by environment variable represent the maximum value and cannot be overridden in the yaml. > Note that unlike resource requests, a resource limit is set for each container individually. In addition […] limits defined in the yaml […] can’t be higher than the value set by environment variable or a policy. Further reading about resource requests and limits: [https://docs.drone.io/runner/kubernetes/configuration/resources/](https://docs.drone.io/runner/kubernetes/configuration/resources/) ---
**Author:** ![beniamin](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/beniamin/32/3078_2.png) [@beniamin](https://drone.discourse.group/u/beniamin)\ **Post date:** [October 16, 2021, 6:11pm UTC](https://drone.discourse.group/t/k8s-runner-resource-requests-failing-to-work/9997/12 "2021-10-16T18:11:07Z")
Thanks for the quick response. Somehow I missed that part of documentation. Do you plan to add something like `DRONE_RESOURCE_MIN_REQUEST_*`, but for limits?