[SOLVED] Agent not picking up builds

NyxCode · March 29, 2018, 3:03pm

Hey, sorry for my 3rd post here in such a short time.
The drone-server and one drone-agent is running on the same machine. Unfortunatly, my agent does not pick up builds which means that every build is stuck in the “pending” state. I have set the log level to debug, but neither the server nor the agent show any errors.

# docker container logs drone_drone-agent_1 
2018-03-29T14:50:08Z |DEBU| request next execution

# docker container logs drone_drone-server_1
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:	export GIN_MODE=release
 - using code:	gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET    /logout                   --> github.com/drone/drone/server.GetLogout (12 handlers)
[GIN-debug] GET    /login                    --> github.com/drone/drone/server.HandleLogin (12 handlers)
[GIN-debug] GET    /api/user                 --> github.com/drone/drone/server.GetSelf (13 handlers)
[GIN-debug] GET    /api/user/feed            --> github.com/drone/drone/server.GetFeed (13 handlers)
[GIN-debug] GET    /api/user/repos           --> github.com/drone/drone/server.GetRepos (13 handlers)
[GIN-debug] POST   /api/user/token           --> github.com/drone/drone/server.PostToken (13 handlers)
[GIN-debug] DELETE /api/user/token           --> github.com/drone/drone/server.DeleteToken (13 handlers)
[GIN-debug] GET    /api/users                --> github.com/drone/drone/server.GetUsers (13 handlers)
[GIN-debug] POST   /api/users                --> github.com/drone/drone/server.PostUser (13 handlers)
[GIN-debug] GET    /api/users/:login         --> github.com/drone/drone/server.GetUser (13 handlers)
[GIN-debug] PATCH  /api/users/:login         --> github.com/drone/drone/server.PatchUser (13 handlers)
[GIN-debug] DELETE /api/users/:login         --> github.com/drone/drone/server.DeleteUser (13 handlers)
[GIN-debug] POST   /api/repos/:owner/:name   --> github.com/drone/drone/server.PostRepo (16 handlers)
[GIN-debug] GET    /api/repos/:owner/:name   --> github.com/drone/drone/server.GetRepo (15 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/builds --> github.com/drone/drone/server.GetBuilds (15 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/builds/:number --> github.com/drone/drone/server.GetBuild (15 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/logs/:number/:pid --> github.com/drone/drone/server.GetProcLogs (15 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/logs/:number/:pid/:proc --> github.com/drone/drone/server.GetBuildLogs (15 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/files/:number --> github.com/drone/drone/server.FileList (15 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/files/:number/:proc/*file --> github.com/drone/drone/server.FileGet (15 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/secrets --> github.com/drone/drone/server.GetSecretList (16 handlers)
[GIN-debug] POST   /api/repos/:owner/:name/secrets --> github.com/drone/drone/server.PostSecret (16 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/secrets/:secret --> github.com/drone/drone/server.GetSecret (16 handlers)
[GIN-debug] PATCH  /api/repos/:owner/:name/secrets/:secret --> github.com/drone/drone/server.PatchSecret (16 handlers)
[GIN-debug] DELETE /api/repos/:owner/:name/secrets/:secret --> github.com/drone/drone/server.DeleteSecret (16 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/registry --> github.com/drone/drone/server.GetRegistryList (16 handlers)
[GIN-debug] POST   /api/repos/:owner/:name/registry --> github.com/drone/drone/server.PostRegistry (16 handlers)
[GIN-debug] GET    /api/repos/:owner/:name/registry/:registry --> github.com/drone/drone/server.GetRegistry (16 handlers)
[GIN-debug] PATCH  /api/repos/:owner/:name/registry/:registry --> github.com/drone/drone/server.PatchRegistry (16 handlers)
[GIN-debug] DELETE /api/repos/:owner/:name/registry/:registry --> github.com/drone/drone/server.DeleteRegistry (16 handlers)
[GIN-debug] PATCH  /api/repos/:owner/:name   --> github.com/drone/drone/server.PatchRepo (16 handlers)
[GIN-debug] DELETE /api/repos/:owner/:name   --> github.com/drone/drone/server.DeleteRepo (16 handlers)
[GIN-debug] POST   /api/repos/:owner/:name/chown --> github.com/drone/drone/server.ChownRepo (16 handlers)
[GIN-debug] POST   /api/repos/:owner/:name/repair --> github.com/drone/drone/server.RepairRepo (16 handlers)
[GIN-debug] POST   /api/repos/:owner/:name/move --> github.com/drone/drone/server.MoveRepo (16 handlers)
[GIN-debug] POST   /api/repos/:owner/:name/builds/:number --> github.com/drone/drone/server.PostBuild (16 handlers)
[GIN-debug] DELETE /api/repos/:owner/:name/builds/:number --> github.com/drone/drone/server.ZombieKill (16 handlers)
[GIN-debug] POST   /api/repos/:owner/:name/builds/:number/approve --> github.com/drone/drone/server.PostApproval (16 handlers)
[GIN-debug] POST   /api/repos/:owner/:name/builds/:number/decline --> github.com/drone/drone/server.PostDecline (16 handlers)
[GIN-debug] DELETE /api/repos/:owner/:name/builds/:number/:job --> github.com/drone/drone/server.DeleteBuild (16 handlers)
[GIN-debug] DELETE /api/repos/:owner/:name/logs/:number --> github.com/drone/drone/server.DeleteBuildLogs (16 handlers)
[GIN-debug] GET    /api/badges/:owner/:name/status.svg --> github.com/drone/drone/server.GetBadge (12 handlers)
[GIN-debug] GET    /api/badges/:owner/:name/cc.xml --> github.com/drone/drone/server.GetCC (12 handlers)
[GIN-debug] POST   /hook                     --> github.com/drone/drone/server.PostHook (12 handlers)
[GIN-debug] POST   /api/hook                 --> github.com/drone/drone/server.PostHook (12 handlers)
[GIN-debug] GET    /stream/events            --> github.com/drone/drone/server.EventStreamSSE (12 handlers)
[GIN-debug] GET    /stream/logs/:owner/:name/:build/:number --> github.com/drone/drone/server.LogStreamSSE (15 handlers)
[GIN-debug] GET    /api/info/queue           --> github.com/drone/drone/server.GetQueueInfo (13 handlers)
[GIN-debug] GET    /authorize                --> github.com/drone/drone/server.HandleAuth (12 handlers)
[GIN-debug] POST   /authorize                --> github.com/drone/drone/server.HandleAuth (12 handlers)
[GIN-debug] POST   /authorize/token          --> github.com/drone/drone/server.GetLoginToken (12 handlers)
[GIN-debug] GET    /api/builds               --> github.com/drone/drone/server.GetBuildQueue (13 handlers)
[GIN-debug] GET    /api/debug/pprof/         --> github.com/drone/drone/server/debug.IndexHandler.func1 (13 handlers)
[GIN-debug] GET    /api/debug/pprof/heap     --> github.com/drone/drone/server/debug.HeapHandler.func1 (13 handlers)
[GIN-debug] GET    /api/debug/pprof/goroutine --> github.com/drone/drone/server/debug.GoroutineHandler.func1 (13 handlers)
[GIN-debug] GET    /api/debug/pprof/block    --> github.com/drone/drone/server/debug.BlockHandler.func1 (13 handlers)
[GIN-debug] GET    /api/debug/pprof/threadcreate --> github.com/drone/drone/server/debug.ThreadCreateHandler.func1 (13 handlers)
[GIN-debug] GET    /api/debug/pprof/cmdline  --> github.com/drone/drone/server/debug.CmdlineHandler.func1 (13 handlers)
[GIN-debug] GET    /api/debug/pprof/profile  --> github.com/drone/drone/server/debug.ProfileHandler.func1 (13 handlers)
[GIN-debug] GET    /api/debug/pprof/symbol   --> github.com/drone/drone/server/debug.SymbolHandler.func1 (13 handlers)
[GIN-debug] POST   /api/debug/pprof/symbol   --> github.com/drone/drone/server/debug.SymbolHandler.func1 (13 handlers)
[GIN-debug] GET    /api/debug/pprof/trace    --> github.com/drone/drone/server/debug.TraceHandler.func1 (13 handlers)
[GIN-debug] GET    /metrics                  --> github.com/drone/drone/server/metrics.PromHandler.func1 (13 handlers)
[GIN-debug] GET    /version                  --> github.com/drone/drone/server.Version (12 handlers)
[GIN-debug] GET    /healthz                  --> github.com/drone/drone/server.Health (12 handlers)
time="2018-03-29T14:50:08Z" level=debug msg="agent connected: 2d6d9bce13d5: polling" 
time="2018-03-29T14:50:27Z" level=info ip=5.10.59.61 latency=7.143587ms method=GET path="/nyxcode/jumpy/3" status=200 time="2018-03-29T14:50:27Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:50:27Z" level=info ip=5.10.59.61 latency=4.671264ms method=GET path="/api/user/feed" status=200 time="2018-03-29T14:50:27Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:50:27Z" level=debug msg="nyxcode granted &{UserID:1 RepoID:1 Repo: Pull:true Push:true Admin:true Synced:1522334139} permission to nyxcode/jumpy" 
time="2018-03-29T14:50:27Z" level=info ip=5.10.59.61 latency=7.57947ms method=GET path="/api/repos/nyxcode/jumpy" status=200 time="2018-03-29T14:50:27Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:50:27Z" level=info ip=5.10.59.61 latency=7.161088ms method=GET path="/favicon.png" status=200 time="2018-03-29T14:50:27Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:50:27Z" level=debug msg="nyxcode granted &{UserID:1 RepoID:1 Repo: Pull:true Push:true Admin:true Synced:1522334139} permission to nyxcode/jumpy" 
time="2018-03-29T14:50:27Z" level=info ip=5.10.59.61 latency=14.256654ms method=GET path="/api/repos/nyxcode/jumpy/builds/3" status=200 time="2018-03-29T14:50:27Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:50:27Z" level=debug msg="user feed: connection opened" 
time="2018-03-29T14:50:29Z" level=debug msg="nyxcode granted &{UserID:1 RepoID:1 Repo: Pull:true Push:true Admin:true Synced:1522334139} permission to nyxcode/jumpy" 
time="2018-03-29T14:50:29Z" level=info ip=5.10.59.61 latency=8.217433ms method=GET path="/api/repos/nyxcode/jumpy/builds" status=200 time="2018-03-29T14:50:29Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:50:47Z" level=info ip=5.10.59.61 latency=657.56473ms method=POST path="/hook" status=200 time="2018-03-29T14:50:47Z" user-agent=GiteaServer 
time="2018-03-29T14:52:16Z" level=info ip=5.10.59.61 latency=2.349672ms method=GET path="/" status=200 time="2018-03-29T14:52:16Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:52:17Z" level=info ip=5.10.59.61 latency=3.605914ms method=GET path="/api/user/feed" status=200 time="2018-03-29T14:52:17Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:52:17Z" level=debug msg="user feed: connection opened" 
time="2018-03-29T14:52:17Z" level=debug msg="nyxcode granted &{UserID:1 RepoID:1 Repo: Pull:true Push:true Admin:true Synced:1522334139} permission to nyxcode/jumpy" 
time="2018-03-29T14:52:17Z" level=info ip=5.10.59.61 latency=5.550692ms method=GET path="/api/repos/nyxcode/jumpy" status=200 time="2018-03-29T14:52:17Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:52:17Z" level=debug msg="nyxcode granted &{UserID:1 RepoID:1 Repo: Pull:true Push:true Admin:true Synced:1522334139} permission to nyxcode/jumpy" 
time="2018-03-29T14:52:17Z" level=info ip=5.10.59.61 latency=8.57062ms method=GET path="/api/repos/nyxcode/jumpy/builds/4" status=200 time="2018-03-29T14:52:17Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:52:18Z" level=debug msg="nyxcode granted &{UserID:1 RepoID:1 Repo: Pull:true Push:true Admin:true Synced:1522334139} permission to nyxcode/jumpy" 
time="2018-03-29T14:52:18Z" level=info ip=5.10.59.61 latency=9.415979ms method=GET path="/api/repos/nyxcode/jumpy/builds" status=200 time="2018-03-29T14:52:18Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 
time="2018-03-29T14:52:47Z" level=debug msg="user feed: connection closed" 
time="2018-03-29T14:52:47Z" level=info ip=5.10.59.61 latency=2m20.424502296s method=GET path="/stream/events" status=200 time="2018-03-29T14:52:47Z" user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"

My docker-compose.yml file (almost identical to the one i found in the documentation)

version: '2'

services:
  drone-server:
    # Drone v0.8.4 compiled for arm
    image: nyxcode/drone
   
    ports:
      - 8000:8000
      - 9000:9000
    volumes:
      - /var/lib/drone:/var/lib/drone/
    restart: always
    environment:
      - DRONE_OPEN=true
      - DRONE_HOST=https://build.nyxcode.com
      - DRONE_GITEA=true
      - DRONE_GITEA_URL=https://git.nyxcode.com
      - DRONE_SECRET=(...)
      - DRONE_DEBUG=true
      - DRONE_DEBUG_PRETTY=true

  drone-agent:
    image: drone/agent:linux-arm

    command: agent
    restart: always
    depends_on:
      - drone-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_SERVER=drone-server:9000
      - DRONE_SECRET=(...)
      - DRONE_DEBUG=true
      - DRONE_DEBUG_PRETTY=true

Currently, I have 4 builds pending - the oldest one was started 30 minutes ago and is stuck at “pending” since then. I am certain that the agent didn’t pick up any build because there’s no docker container apart from the server and the agent.

# docker container ls
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS                    PORTS                                                             NAMES
2d6d9bce13d5        drone/agent:linux-arm   "/bin/drone-agent ag…"   10 minutes ago      Up 10 minutes (healthy)   3000/tcp                                                          drone_drone-agent_1
cc34a7e91913        nyxcode/drone           "/bin/drone-server"      10 minutes ago      Up 10 minutes             80/tcp, 0.0.0.0:8000->8000/tcp, 443/tcp, 0.0.0.0:9000->9000/tcp   drone_drone-server_1

I hope I havn’t missed something, but I visited every forum/stack overflow post and every github issue i could find without success.
Best regards!

bradrydzewski · March 29, 2018, 3:10pm

Have you searched for existing Discourse threads on this topic?

NyxCode · March 29, 2018, 3:11pm

Yes, I have and I tried every solution I could find.

bradrydzewski · March 29, 2018, 3:14pm

Does your .drone.yml file include the platform field, as described here in the below comment, so that drone knows to route the build to an arm agent?

https://github.com/drone/drone/issues/1767#issuecomment-316429910

NyxCode · March 29, 2018, 3:25pm

That was it! I didn’t knew that the arm-agent only accepts builds only for arm - even though it makes 100% sense. Because I thought that the issue was not specific to the architecture I didn’t find your post. Thanks, again, for your awseome help!!

Topic		Replies	Views
Build pending / missing agents Drone Support	2	894	October 28, 2017
Build shows running when all steps complete Drone Support	5	510	January 25, 2018
Drone:0.5 builds hanging Drone Support	2	1013	February 20, 2017
[SOLVED] Gogs+drone webhook does not trigger builds Drone Support	9	1123	April 11, 2019
Builds stuck in pending state Drone Support	7	4625	July 3, 2019

[SOLVED] Agent not picking up builds

Related topics