[solved] Build never starts - kubernets install - logs looks ok, builds are created triggered by github changes, but nothing happens

Hi,
Just did a fresh install on kubernets and most things works:
UI and Auth with Github - OK
Webhooks - OK
Builds - Nope :frowning:

Builds are created but nothing happens, it shows “loading” and the empty console and that is it.

Turned logs on to debug level, but did not notice anything wrong. No errors.

What is the best way to debug drone on kubernets ? Could this be related to roles ? or other kubernets config ?

Agent logs:
time=“2021-08-22T04:48:11Z” level=info msg=“starting the server” addr=":3000"
time=“2021-08-22T04:48:11Z” level=info msg=“successfully pinged the remote server”
time=“2021-08-22T04:48:11Z” level=info msg=“polling the remote server” capacity=4 endpoint=“http://drone-service” kind=pipeline type=kubernetes

Server Logs:
DEBU[4253] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4267] fields.time=“2021-08-22T04:43:20Z” latency=“414.607µs” method=GET remote=“10.44.0.0:35920” request=/api/user request-id=1x4HNXZUnYsJB6z7TaAevA2aTie
DEBU[4269] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HNwze7GJBi5E7E7oeyozcEWY user.login=pentateu visibility=public
DEBU[4269] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HNuUsHafJQBwCzAFyRjktkWi user.login=pentateu visibility=public
DEBU[4269] fields.time=“2021-08-22T04:43:23Z” latency=“965.672µs” method=GET remote=“10.44.0.0:35920” request=/api/repos/moleculer-go/gateway request-id=1x4HNwze7GJBi5E7E7oeyozcEWY
DEBU[4269] fields.time=“2021-08-22T04:43:23Z” latency=1.993142ms method=GET remote=“10.44.0.0:35460” request="/api/repos/moleculer-go/gateway/builds?page=1&per_page=50" request-id=1x4HNuUsHafJQBwCzAFyRjktkWi
DEBU[4271] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HODuSBqfIx4SsIU0Jh1oN7fv user.login=pentateu visibility=public
DEBU[4271] fields.time=“2021-08-22T04:43:25Z” latency=1.765044ms method=GET remote=“10.44.0.0:35460” request=/api/repos/moleculer-go/gateway/builds/7 request-id=1x4HODuSBqfIx4SsIU0Jh1oN7fv
DEBU[4276] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HOhuBDnSbpaNxbHF5Hx6rI5x user.login=pentateu visibility=public
DEBU[4276] fields.time=“2021-08-22T04:43:29Z” latency=1.984434ms method=GET remote=“10.44.0.0:35460” request="/api/repos/moleculer-go/gateway/builds?page=1&per_page=50" request-id=1x4HOhuBDnSbpaNxbHF5Hx6rI5x
DEBU[4277] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4277] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4279] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HP6yvX9TmZn2n6eYYr47QUBb user.login=pentateu visibility=public
DEBU[4279] fields.time=“2021-08-22T04:43:33Z” latency=1.550588ms method=GET remote=“10.44.0.0:35460” request=/api/repos/moleculer-go/gateway/builds/branches request-id=1x4HP6yvX9TmZn2n6eYYr47QUBb
DEBU[4279] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4279] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4280] fields.time=“2021-08-22T04:43:33Z” latency=1.276735ms method=GET remote=“10.44.0.0:35460” request=/static/media/Inter-Bold.3e3cb905.ttf request-id=1x4HPBT6CtrWlVn4Jqx2UFdf89G
DEBU[4280] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HPCnoRdWVHdW84iKMnP9enhl user.login=pentateu visibility=public
DEBU[4280] fields.time=“2021-08-22T04:43:33Z” latency=2.144083ms method=GET remote=“10.44.0.0:35920” request="/api/repos/moleculer-go/gateway/builds?page=1&per_page=50" request-id=1x4HPCnoRdWVHdW84iKMnP9enhl
DEBU[4281] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HPHPgFQDd6d3UjIBQOMUqik2 user.login=pentateu visibility=public
DEBU[4281] fields.time=“2021-08-22T04:43:34Z” latency=“981.051µs” method=GET remote=“10.44.0.0:35946” request=/api/repos/moleculer-go/gateway request-id=1x4HPHPgFQDd6d3UjIBQOMUqik2
WARN[4284] trigger: cannot find yaml commit=ef2d812ce5e4d6af56fd8b0881bceb28a175f5f7 error=“Not Found” event=custom ref=refs/heads/master repo=moleculer-go/example-media-store
DEBU[4284] fields.time=“2021-08-22T04:43:37Z” latency=46.691076424s method=POST remote=“10.44.0.0:35894” request="/api/repos/moleculer-go/example-media-store/builds?branch=master" request-id=1x4HJqQk8F9tdoJjSUQuSz3FGHe
DEBU[4285] fields.time=“2021-08-22T04:43:39Z” latency=“386.443µs” method=GET remote=“10.44.0.0:35946” request=/api/user request-id=1x4HPvGolKeRrHWnVoxJOPcAbnO
DEBU[4292] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HQntw0Ha4HGt1MNnhN5kCWjU user.login=pentateu visibility=public
DEBU[4292] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HQrHiRXEzpfNdBgrYYzk4qqH user.login=pentateu visibility=public
DEBU[4292] fields.time=“2021-08-22T04:43:46Z” latency=“963.383µs” method=GET remote=“10.44.0.0:35920” request=/api/repos/moleculer-go/gateway request-id=1x4HQrHiRXEzpfNdBgrYYzk4qqH
DEBU[4292] fields.time=“2021-08-22T04:43:46Z” latency=1.967617ms method=GET remote=“10.44.0.0:35946” request="/api/repos/moleculer-go/gateway/builds?page=1&per_page=50" request-id=1x4HQntw0Ha4HGt1MNnhN5kCWjU
DEBU[4294] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HQyMBjVrDOVXWfrZV8QUiOpM user.login=pentateu visibility=public
DEBU[4294] fields.time=“2021-08-22T04:43:47Z” latency=1.901769ms method=GET remote=“10.44.0.0:35946” request=/api/repos/moleculer-go/gateway/builds/7 request-id=1x4HQyMBjVrDOVXWfrZV8QUiOpM
DEBU[4297] api: read access granted name=gateway namespace=moleculer-go request-id=1x4HRHqbuSRMo0a1I0wb9R5nv3N user.login=pentateu visibility=public
DEBU[4297] fields.time=“2021-08-22T04:43:50Z” latency=1.985392ms method=GET remote=“10.44.0.0:35946” request="/api/repos/moleculer-go/gateway/builds?page=1&per_page=50" request-id=1x4HRHqbuSRMo0a1I0wb9R5nv3N
DEBU[4299] fields.time=“2021-08-22T04:43:53Z” latency=“461.939µs” method=GET remote=“10.44.0.0:35946” request=/api/user request-id=1x4HRcybNKoz4OGrQ4LUcPjbKZj
DEBU[4302] api: read access granted name=moleculer namespace=moleculer-go request-id=1x4HRw479WhIJvi0VHV7w2VqYF3 user.login=pentateu visibility=public
DEBU[4302] fields.time=“2021-08-22T04:43:55Z” latency=“904.14µs” method=GET remote=“10.44.0.0:35920” request=/api/repos/moleculer-go/moleculer request-id=1x4HRw479WhIJvi0VHV7w2VqYF3
DEBU[4302] api: read access granted name=moleculer namespace=moleculer-go request-id=1x4HRwNRRTvg85FQ3qLLzpq3y2D user.login=pentateu visibility=public
DEBU[4302] fields.time=“2021-08-22T04:43:55Z” latency=1.700323ms method=GET remote=“10.44.0.0:35946” request="/api/repos/moleculer-go/moleculer/builds?page=1&per_page=50" request-id=1x4HRwNRRTvg85FQ3qLLzpq3y2D
DEBU[4307] api: read access granted name=moleculer namespace=moleculer-go request-id=1x4HSZKesop8GlixtG8aboelbbC user.login=pentateu visibility=public
DEBU[4307] api: access granted admin=true name=moleculer namespace=moleculer-go read=true request-id=1x4HSZKesop8GlixtG8aboelbbC user.login=pentateu visibility=public write=true
DEBU[4307] trigger: received commit=0b2a111f192464013815a1a9f4cd955e75c568a4 event=custom ref=refs/heads/develop repo=moleculer-go/moleculer
DEBU[4308] fields.time=“2021-08-22T04:44:01Z” latency=1.023258302s method=POST remote=“10.44.0.0:35946” request="/api/repos/moleculer-go/moleculer/builds?branch=develop" request-id=1x4HSZKesop8GlixtG8aboelbbC
DEBU[4309] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4309] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4312] fields.time=“2021-08-22T04:44:05Z” latency=“424.97µs” method=GET remote=“10.44.0.0:35946” request=/api/user request-id=1x4HTBiIsusCQVVPh8EpBb9wigH
DEBU[4313] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4313] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4315] fields.time=“2021-08-22T04:44:08Z” latency=“478.349µs” method=GET remote=“10.44.0.0:35946” request=/ request-id=1x4HTaBsQ6YWJlZRPhxoX8qxLUg
DEBU[4315] events: stream error request-id=1x4H0BW3q4ElBV3MQ3zt5OMhJSm user.login=pentateu
DEBU[4315] events: stream closed request-id=1x4H0BW3q4ElBV3MQ3zt5OMhJSm user.login=pentateu
DEBU[4315] fields.time=“2021-08-22T04:44:08Z” latency=3m54.328035114s method=GET remote=“10.44.0.0:35816” request=/api/stream request-id=1x4H0BW3q4ElBV3MQ3zt5OMhJSm
DEBU[4315] fields.time=“2021-08-22T04:44:08Z” latency=“382.543µs” method=GET remote=“10.44.0.0:35946” request=/api/user request-id=1x4HTbr32MsLrbelPS8jzmW3AJg
DEBU[4315] events: stream opened request-id=1x4HTaFp5bkxRrhsoUxIik0LVHe user.login=pentateu
DEBU[4315] fields.time=“2021-08-22T04:44:09Z” latency=“99.876µs” method=GET remote=“10.44.0.0:35970” request=/favicon.png request-id=1x4HThs5pzU75PyBR8MFaTPbcFM
DEBU[4316] fields.time=“2021-08-22T04:44:09Z” latency=“403.078µs” method=GET remote=“10.44.0.0:35946” request=/api/user request-id=1x4HThUpM0sy3CVtw1S2W2QR4d4
DEBU[4316] fields.time=“2021-08-22T04:44:09Z” latency=11.105982ms method=GET remote=“10.44.0.0:35970” request="/api/user/repos?latest=true" request-id=1x4HTfHddW5UqMfhdrxIOvsXbJO
DEBU[4318] api: read access granted name=moleculer namespace=moleculer-go request-id=1x4HTxcneG3HFgwnyjok6fXk0Vw user.login=pentateu visibility=public
DEBU[4318] fields.time=“2021-08-22T04:44:11Z” latency=1.014609ms method=GET remote=“10.44.0.0:35970” request=/api/repos/moleculer-go/moleculer request-id=1x4HTxcneG3HFgwnyjok6fXk0Vw
DEBU[4319] api: read access granted name=moleculer namespace=moleculer-go request-id=1x4HU6hoo4bq0WSyuYbh9Bfkf3r user.login=pentateu visibility=public
DEBU[4319] fields.time=“2021-08-22T04:44:12Z” latency=1.64608ms method=GET remote=“10.44.0.0:35970” request="/api/repos/moleculer-go/moleculer/builds?page=1&per_page=50" request-id=1x4HU6hoo4bq0WSyuYbh9Bfkf3r
DEBU[4320] api: read access granted name=moleculer namespace=moleculer-go request-id=1x4HUKLXDyrKaxUnqMsQKyMX60x user.login=pentateu visibility=public
DEBU[4320] fields.time=“2021-08-22T04:44:14Z” latency=1.82508ms method=GET remote=“10.44.0.0:35970” request=/api/repos/moleculer-go/moleculer/builds/4 request-id=1x4HUKLXDyrKaxUnqMsQKyMX60x
DEBU[4337] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4337] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4339] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4339] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4369] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4369] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4373] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4373] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4397] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4397] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4397] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4400] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4427] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4429] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4430] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4432] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4457] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4457] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4457] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4457] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4487] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4487] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4490] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4490] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4517] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4517] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4520] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4520] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4550] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4550] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4553] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4553] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4558] manager: request queue item arch= kernel= kind=pipeline os= type=kubernetes variant=
DEBU[4577] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4577] manager: context canceled arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4577] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4577] manager: request queue item arch=arm64 kernel= kind=pipeline os=linux type=docker variant=v8
DEBU[4588] manager: context canceled arch= kernel= kind=pipeline os= type=kubernetes variant=
DEBU[4598] manager: request queue item arch= kernel= kind=pipeline os= type=kubernetes variant=

Found the issue, it was missing the “type: kubernetes” in my “.drone-io.yml” build file.

I did not expect that I had to be specific as the type of backend being used. Now I know :slight_smile: Works fine now.

I am glad to see you were able to resolve.

For anyone coming across this thread in the future, if you are experiencing this issue, please see both of the below guides from our faq. These guides discuss all known root causes for this behavior.

http://discuss.harness.io/t/builds-are-stuck-in-pending-status/4437