# Drone Kubernetes Job Failure "cannot get stage details" **URL:** \ **Category:** Drone Support\ **Created:** [January 21, 2019, 5:11am UTC](https://drone.discourse.group/t/drone-kubernetes-job-failure-cannot-get-stage-details/4712 "2019-01-21T05:11:24Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![NickCarton](https://avatars.discourse-cdn.com/v4/letter/n/838e76/32.png) [@NickCarton](https://drone.discourse.group/u/NickCarton)\ **Post date:** [January 21, 2019, 5:11am UTC](https://drone.discourse.group/t/drone-kubernetes-job-failure-cannot-get-stage-details/4712/1 "2019-01-21T05:11:24Z")
Hi, I’m trying to set up a new production environment, using Kubernetes. Drone seems like the product of choice when setting up a new CI/CD pipeline. I’ve set up drone with kubernetes and although I know this is an alpha/beta feature I was wondering if I could get some support here. I’ve set up Drone and have a project set up to build in drone. But every time I try to start a new pipeline, the kubernetes job that gets triggered errors like so. **Error:** `{"arch":"amd64","level":"warning","machine":"ip-10-0-11-202.ap-southeast-2.compute.internal","msg":"runner: cannot get stage details","os":"linux","stage-id":1,"time":"2019-01-21T04:21:01Z"}` `{"error":"Forbidden","level":"warning","msg":"program terminated","time":"2019-01-21T04:21:01Z"}` I followed the basic set up on the docs website. Does anyone know what I’m missing? Thanks, Nick ---
**Author:** ![bradrydzewski](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/bradrydzewski/32/3513_2.png) [@bradrydzewski](https://drone.discourse.group/u/bradrydzewski)\ **Post date:** [January 21, 2019, 7:17am UTC](https://drone.discourse.group/t/drone-kubernetes-job-failure-cannot-get-stage-details/4712/2 "2019-01-21T07:17:26Z")
Hi Nick, Drone creates a Kubernetes job for each build, which consists of a single-user controller container that orchestrates the build pipeline. One of the first thing that the controller does is contact the main Drone server to try and fetch addition details required for execution. This error message may indicate the controller is unable to communicate with the server. This is something we should be able to easily verify. If the controller is successfully communicating with the server we would see this message printed in the logs: ```auto { "level": "debug", "msg": "manager: fetching stage details", "step-id": 1, "time": "2019-01-17T17:19:35-08:00" } ``` Can you please check the server logs and let me know if you see anything? ---
**Author:** ![NickCarton](https://avatars.discourse-cdn.com/v4/letter/n/838e76/32.png) [@NickCarton](https://drone.discourse.group/u/NickCarton)\ **Post date:** [January 21, 2019, 7:42am UTC](https://drone.discourse.group/t/drone-kubernetes-job-failure-cannot-get-stage-details/4712/3 "2019-01-21T07:42:38Z")
Here are my server logs. `{"build-id":3,"level":"debug","msg":"kubernetes: creating job","repo-id":165,"stage-id":3,"stage-name":"build-hello-hapi","stage-number":1,"time":"2019-01-21T07:39:21Z"}` `{"build-id":3,"level":"debug","msg":"kubernetes: successfully created job","repo-id":165,"stage-id":3,"stage-name":"build-hello-hapi","stage-number":1,"time":"2019-01-21T07:39:21Z"}` `{"fields.time":"2019-01-21T07:39:21Z","latency":1312831421,"level":"debug","method":"POST","msg":"","remote":"10.45.192.1:47386","request":"/hook","request-id":"1G4EEiCvS5xPBed27ZILVCZDsHt","time":"2019-01-21T07:39:21Z"}` I dont see anything that says fetching state details. I’m wondering if it has something to do with my DRONE\_RPC\_HOST? should it be set to the name of the drone service in kubernetes? I have it set to the same DNS as my drone website eg. “[drone.mywebsite.com](http://drone.mywebsite.com)” ---
**Author:** ![NickCarton](https://avatars.discourse-cdn.com/v4/letter/n/838e76/32.png) [@NickCarton](https://drone.discourse.group/u/NickCarton)\ **Post date:** [January 21, 2019, 7:55am UTC](https://drone.discourse.group/t/drone-kubernetes-job-failure-cannot-get-stage-details/4712/4 "2019-01-21T07:55:08Z")
You know what? I might not be hitting my drone server because I have an IP whitelist set up on my ingress. I’ll set up an internal passthrough on dns “drone-service” and see how I go. Thanks for your help 🙂 ---
**Author:** ![bradrydzewski](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/bradrydzewski/32/3513_2.png) [@bradrydzewski](https://drone.discourse.group/u/bradrydzewski)\ **Post date:** [January 21, 2019, 8:01am UTC](https://drone.discourse.group/t/drone-kubernetes-job-failure-cannot-get-stage-details/4712/5 "2019-01-21T08:01:47Z")
Thanks for checking the server logs. It definitely sounds like a networking issue, with the controller unable to reach the server. The controller will attempt to contact the server using the address specified in the following environment variables: ```auto DRONE_SERVER_HOST=some.address.com DRONE_SERVER_PROTO=http ``` In some cases, you may want to use an internal address for communication. The following environment variables can be used to override and customize the address: ```auto DRONE_RPC_HOST=some.internal.address:80 # you can specify port, if necessary DRONE_RPC_PROTO=http ``` I recommend exploring the above configuration parameters to see if you can find an address and protocol setting that are able to successfully contact the Drone server. If that does not work you can enable trace logging which will expose detailed controller logs for the controller-to-server communications. This should only be enabled temporarily and should be disabled once troubleshooting is complete. ```auto DRONE_LOGS_TRACE=true ``` I hope this helps. If you find a solution please post back so that others may benefit 🙂 ---
**Author:** ![NickCarton](https://avatars.discourse-cdn.com/v4/letter/n/838e76/32.png) [@NickCarton](https://drone.discourse.group/u/NickCarton)\ **Post date:** [January 21, 2019, 9:40am UTC](https://drone.discourse.group/t/drone-kubernetes-job-failure-cannot-get-stage-details/4712/6 "2019-01-21T09:40:45Z")
Thanks for your very speedy responses! Your suggestion about the port number fixed my issue entirely, I should have specified my port number as 8080. Setting my RPC\_PROTO to http helped too haha Thanks for your help! ---
**Author:** ![cuijxin](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/cuijxin/32/5563_2.png) [@cuijxin](https://drone.discourse.group/u/cuijxin)\ **Post date:** [July 11, 2019, 3:14am UTC](https://drone.discourse.group/t/drone-kubernetes-job-failure-cannot-get-stage-details/4712/7 "2019-07-11T03:14:11Z")
my drone config is DRONE\_KUBERNETES\_ENABLED: true DRONE\_KUBERNETES\_NAMESPACE: default DRONE\_GITHUB\_SERVER: [https://github.comgithub.com](https://github.comgithub.com) DRONE\_GITHUB\_CLIENT\_ID: \*\*\*\*\*\*\*\*\*\*\*\* DRONE\_GITHUB\_CLIENT\_SECRET: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* DRONE\_SERVER\_HOST: my\_domain DRONE\_SERVER\_PROTO: http DRONE\_RPC\_HOST: drone-server-service DRONE\_RPC\_PROTO: http DRONE\_DATABASE\_DRIVER: sqlite3 DRONE\_DATABASE\_DATASOURCE: /drone/drone.sqlite DRONE\_USER\_CREATE: username:cuijxin,admin:true DRONE\_SECRET\_SECRET: \*\*\*\*\*\*\*\*\*\*\*\* DRONE\_SECRET\_ENDPOINT: [http://drone-secrets-service](http://drone-secrets-service) DRONE\_LOGS\_TRACE: true and drone server’s log drone server 日志：“{“build-id”:1,“level”:“debug”,“msg”:“kubernetes: creating job”,“repo-id”:8,“stage-id”:1,“stage-name”:“deploy”,“stage-number”:1,“time”:“2019-07-11T02:42:52Z”} {“build-id”:1,“level”:“debug”,“msg”:“kubernetes: successfully created job”,“repo-id”:8,“stage-id”:1,“stage-name”:“deploy”,“stage-number”:1,“time”:“2019-07-11T02:42:52Z”} {“fields.time”:“2019-07-11T02:42:52Z”,“latency”:3003858571,“level”:“debug”,“method”:“POST”,“msg”:"",“remote”:“10.32.0.6:47350”,“request”:"/hook",“request-id”:“1NqeHuyaEnnggoyy3PthcodAdqv”,“time”:“2019-07-11T02:42:52Z”}” job’s log “2019/07/11 02:42:57 [DEBUG] POST [http://drone-server-service/rpc/v1/details](http://drone-server-service/rpc/v1/details) {“arch”:“amd64”,“error”:"",“level”:“warning”,“machine”:“master”,“msg”:“runner: cannot get stage details”,“os”:“linux”,“stage-id”:1,“time”:“2019-07-11T02:43:02Z”} {“error”:"",“level”:“warning”,“msg”:“program terminated”,“time”:“2019-07-11T02:43:02Z”}”