# Kubernetes docker build failed

**URL:** <https://drone.discourse.group/t/kubernetes-docker-build-failed/11018>\
**Category:** Drone Support\
**Created:** [May 25, 2019, 9:56am UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018 "2019-05-25T09:56:34Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![metal3d](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/metal3d/32/4750_2.png) [@metal3d](https://drone.discourse.group/u/metal3d)\
**Post date:** [May 25, 2019, 9:56am UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/1 "2019-05-25T09:56:34Z")

</div>

Hello  
I spent a long time with 2 kubernetes clusters to try to build images with Drone.

I installed Drone with helm chart (official) and I’m using Gitea. Everything is fine until I try to build image.

My .drone.yml file is:

```
kind: pipeline
name: default

steps:
  - name: test
    image: plugins/docker
    commands:
      - docker ps

```

I also tried “docker” image and “docker:dind”. What I see is:

```
+ docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

```

My repository is “Trusted” and I set up drone with my admin account that is the user on gitea which is mine.

The problem is that, later, I want to use “source to image” to build image and push it on my private repository, so I will need that kind of commands:

```
- s2i build . my-base-image 10.x.x.x:5000/my-app-image
- docker push 10.x.x.x:5000/my-app-image

```

And it needs docker to pull base image, then push my image.

Here is the lines in drone deployment:

```
  - env:
    - name: DRONE_KUBERNETES_ENABLED
      value: "true"
    - name: DRONE_KUBERNETES_NAMESPACE
      value: drone
    - name: DRONE_KUBERNETES_SERVICE_ACCOUNT
      value: drone-drone-pipeline
    - name: DRONE_GIT_ALWAYS_AUTH
      value: "false"
    # I hide that
    - name: DRONE_SERVER_HOST
      value: drone.XXXXX
    - name: DRONE_SERVER_PROTO
      value: https
    - name: DRONE_USER_CREATE
      value: username:metal3d,machine:false,admin:true
    - name: DRONE_RPC_SECRET
      valueFrom:
        secretKeyRef:
          key: secret
          name: drone-drone
    - name: DRONE_DATABASE_DATASOURCE
      value: /var/lib/drone/drone.sqlite
    - name: DRONE_DATABASE_DRIVER
      value: sqlite3
    - name: DRONE_LOGS_DEBUG
      value: "false"
    # I hide that
    - name: DRONE_GITEA_SERVER
      value: https://XXXXXX

```

Also, note that I didn’t found any /var/lib/docker mount point in the deployment, so I really wonder how Drone can build image in Kubernetes “pods”.

Thanks

---

<div class="post-metadata">

**Author:** ![metal3d](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/metal3d/32/4750_2.png) [@metal3d](https://drone.discourse.group/u/metal3d)\
**Post date:** [May 25, 2019, 9:59am UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/2 "2019-05-25T09:59:52Z")

</div>

I forget to say that several repositories, where I’m not the owner, will need to build images with s2i. I really wonder how I will be able to make that working.

---

<div class="post-metadata">

**Author:** ![narze](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/narze/32/4982_2.png) [@narze](https://drone.discourse.group/u/narze)\
**Post date:** [May 25, 2019, 1:23pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/3 "2019-05-25T13:23:49Z")

</div>

This drone configuration works for me on official helm chart.  
Note that I have to use volumes on every pipelines to make with docker layer caching working.  
But I’m still stuck at [no space left on device](http://discuss.harness.io/t/drone-1-0-on-helm-no-space-left-on-device/4584/4) issue. I’m going to stop using k8s method for now and use [https://autoscale.drone.io/](https://autoscale.drone.io/) instead.

```
kind: pipeline
name: prepare

clone:
  disable: true

failure: ignore

steps:
  - name: ensure-cache-path
    image: busybox
    commands:
      - df -h
      - ls -la /drone/src
    volumes:
      - name: cache
        path: /cache

services:
  - name: docker
    image: docker:dind
    privileged: true
    environment:
      DOCKER_LAUNCH_DEBUG:
        true
      DOCKER_HOST:
        unix:///var/run/host.sock
    volumes:
      - name: dockersock
        path: /var/run
      - name: docker
        path: /var/lib/docker

volumes:
  - name: docker
    host:
      path: /var/cache/${DRONE_REPO}/docker
  - name: dockersock
temp: {}

---
kind: pipeline
name: build

depends_on:
  - prepare

clone:
  depth: 1

globals:
  - &docker_creds
    username:
      from_secret: docker_username
    password:
      from_secret: docker_password

steps:
  - name: ls
    image: busybox
    commands:
      - ls -la /drone/src
  - name: build-docker
    image: plugins/docker
    settings:
      tags:
        - ${DRONE_BRANCH//\//-}-${DRONE_COMMIT_SHA:0:8}
      repo: xxxx/yyyy
      use_cache: true
      <<: *docker_creds
    volumes:
      - name: docker
        path: /var/lib/docker

volumes:
  - name: cache
    host:
      path: /var/cache
  - name: docker
    host:
      path: /var/cache/${DRONE_REPO}/docker
```

---

<div class="post-metadata">

**Author:** ![metal3d](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/metal3d/32/4750_2.png) [@metal3d](https://drone.discourse.group/u/metal3d)\
**Post date:** [May 25, 2019, 2:01pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/4 "2019-05-25T14:01:36Z")

</div>

Hi, thanks a lot, I will try that.

In the same time I realized that **plugins/docker is working** , the problem is that I wanted to use s2i. So I managed a “fake s2i” like that:

```
kind: pipeline
name: default

steps:
  - name: prepare-s2i
    image: alpine
    commands:
      - |
        cat > Dockerfile << EOF
        FROM 10.233.16.154:5000/metal3d/nginx:1.15-s2i
        ADD . ./
        RUN mkdir -p /tmp/src; /usr/libexec/s2i/assemble 
        CMD ["/usr/libexec/s2i/run"]
        EOF
      - cat Dockerfile
 
  - name: build
    image: plugins/docker
    settings:
      registry: 10.233.16.154:5000
      repo: 10.233.16.154:5000/metal3d/httptest
      insecure: true

```

This is a “trick”, I only prepare a Dockerfile that makes exactly what s2i does: take an image, build sources with “assemble” and set the command to “run”.

Note that the image is now well built and pushed to the private registry (only accessible through Kubernetes cluster, I didn’t tried to use service name but it should work)

That is very simple, maybe more adapted to my workflow. But, anyway, I will take a look on your solution that can help me a lot later. One more time, thanks a lot !

---

<div class="post-metadata">

**Author:** ![bradrydzewski](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/bradrydzewski/32/3513_2.png) [@bradrydzewski](https://drone.discourse.group/u/bradrydzewski)\
**Post date:** [May 25, 2019, 3:26pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/5 "2019-05-25T15:26:58Z")

</div>

_EDIT_ looks like you figured out how to get the Docker plugin working. The below explanations may still be useful to understand _why_ the various configurations previously attempted did not work.

_NOTE_ the recommended approach to building and publishing docker images is using the `plugins/docker` plugin, as you have done in your updated example. This can be done _without_ requiring trusted mode and is therefore more safe. Other options that require trusted mode are of course viable alternatives, but trusted mode inherently introduces security concerns since it effectively grants the build unrestricted root access to your host machine.

* * *

There are two documented examples for how to build docker images using drone:  
[https://docs.drone.io/examples/service/docker\_dind/](https://docs.drone.io/examples/service/docker_dind/)  
[https://docs.drone.io/examples/service/docker/](https://docs.drone.io/examples/service/docker/)

One issue with your example is that there is no docker daemon running inside `plugins/docker` because you are overriding the default plugin entrypoint when you set `commands`. If you are going to use this plugin image, it should be used in the following manner [http://plugins.drone.io/drone-plugins/drone-docker/](http://plugins.drone.io/drone-plugins/drone-docker/)

> I also tried “docker” image and “docker:dind”

Here are some root causes for the error you described:

1. the `docker` image does not start a docker daemon
2. the `docker:dind` starts a docker daemon, but the image requires `privileged: true`
3. the `docker:dind` starts a docker daemon as the entrypoint, but when you set `commands` you are overriding the default container entrypoint, and as a result no docker daemon starts.

> My repository is “Trusted” and I set up drone with my admin account that is the user on gitea which is mine.

setting a repository as trusted is the first step. The second step is that you need to run containers in privileged mode, using `privileged: true`

---

<div class="post-metadata">

**Author:** ![metal3d](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/metal3d/32/4750_2.png) [@metal3d](https://drone.discourse.group/u/metal3d)\
**Post date:** [May 25, 2019, 4:02pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/6 "2019-05-25T16:02:30Z")

</div>

Thanks, that’s what I’ve understood later.

What I really need, actually, is to create a plugin that uses “s2i” command - this command needs a docker daemon somewhere to pull base image, and to push resulting image.

What I did is a “trick” to avoid that command usage, but s2i allows several options.

I already tried to build a “s2i” image using “FROM plugins/docker” but it seems that I’ve got many problems to override it…

My plugin Dockerfile makes:

```
FROM plugins/docker

ENV S2I_VERSION=1.1.14 \
    S2I_GITCOMMIT=874754de

RUN set -xe; \
    wget -O - https://github.com/openshift/source-to-image/releases/download/v${S2I_VERSION}/source-to-image-v${S2I_VERSION}-${S2I_GITCOMMIT}-linux-amd64.tar.gz | tar -C /usr/local/bin -zxf - ./s2i

ADD s2ibuild.sh /usr/local/bin/s2ibuild.sh

ENTRYPOINT ["/usr/local/bin/dockerd-entrypoint.sh", "/usr/local/bin/s2ibuild.sh"]

```

And the builder is:  
#!/bin/sh -e

```
set -e

s2i build /drone/src ${PLUGIN_CONTEXT-./} ${PLUGIN_IMAGE} ${PLUGIN_TARGET}
docker push ${PLUGIN_TARGET}

```

And the .drone.yml file is:

```
kind: pipeline
name: default

steps:
  - name: s2i-build
    image: metal3d/drone-plugin-s2i
    settings:
      registry: docker-docker-registry.registry:5000
      image: docker-docker-registry.registry:5000/metal3d/nginx:1.15-s2i
      target: docker-docker-registry.registry:5000/metal3d/httptest
      insecure: true

```

And of course… I’ve got the error:

`FATAL: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?`

Something is missing in my image, but I don’t find what… If you can help ?

---

<div class="post-metadata">

**Author:** ![bradrydzewski](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/bradrydzewski/32/3513_2.png) [@bradrydzewski](https://drone.discourse.group/u/bradrydzewski)\
**Post date:** [May 25, 2019, 6:33pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/7 "2019-05-25T18:33:40Z")

</div>

Here are some tips:

1. you need to set `privileged: true` for your pipeline step. The `plugins/docker` image is whitelisted and automatically run in privileged mode, but your custom plugin will not.
2. where are you starting the docker daemon? If you look at `plugins/docker` source code you will see that we start the docker daemon in the Go code [1]

Please note that you can test plugins locally [2] which makes it easier to debug them and triage these sorts of issues. I recommend studying the docker plugin source code [3] more closely which may help you proceed.

[1] [https://github.com/drone-plugins/drone-docker/blob/master/docker.go#L70:L84](https://github.com/drone-plugins/drone-docker/blob/master/docker.go#L70:L84)  
[2] [https://github.com/drone-plugins/drone-docker#usage](https://github.com/drone-plugins/drone-docker#usage)  
[3] [https://github.com/drone-plugins/drone-docker](https://github.com/drone-plugins/drone-docker)

---

<div class="post-metadata">

**Author:** ![metal3d](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/metal3d/32/4750_2.png) [@metal3d](https://drone.discourse.group/u/metal3d)\
**Post date:** [May 25, 2019, 10:09pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/8 "2019-05-25T22:09:18Z")

</div>

> [@Enable TLS between drone server and k8s runner fails](https://drone.discourse.group/t/enable-tls-between-drone-server-and-k8s-runner-fails/4679/7):
>
> 1. where are you starting the docker daemon? If you look at `plugins/docker` source code you will see that we start the docker daemon in the Go code [1]

This is what I didn’t undertand until I’ve taken a look on the drone-docker sources (just minutes before your answer 🙂)

This is, IMHO, one point that Drone is missing… I’m pretty sure you’re very busy and you will not create such whitelisted plugin. But if one day you want to make it possible, please consider s2i to be a very useful plugin. I know Go and I probably can build such plugin myself, but I am running out of time (and I don’t know the structure as well as you, I will spend so much more time than you)

Even if I was able to find time to develop such of plugin, it will not be whitelisted and my teammate will not be able to activate it (no privilege) - so I have to use my workaround for now. It’s usable, that’s the essential goal 🙂

S2I would be very interesting to avoid my teammate to overwrite Dockerfile, and let a “base image” to assemble the image. The project sources doesn’t need to have a Dockerfile. It’s a bit more secured and it can make incremental build as well.

It’s not so bad, as I’m able to fake the s2i build by generating a Dockerfile, I will be able to leave that for now.

PS: Drone is very impressive, it works very well now in Kubernetes, and I’m very happy to use it. This little worry does not spoil my enthusiasm to use it at all.

---

<div class="post-metadata">

**Author:** ![bradrydzewski](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/bradrydzewski/32/3513_2.png) [@bradrydzewski](https://drone.discourse.group/u/bradrydzewski)\
**Post date:** [May 25, 2019, 10:42pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/9 "2019-05-25T22:42:29Z")

</div>

> [@Enable TLS between drone server and k8s runner fails](https://drone.discourse.group/t/enable-tls-between-drone-server-and-k8s-runner-fails/4679/8):
>
> Even if I was able to find time to develop such of plugin, it will not be whitelisted and my teammate will not be able to activate it

you can add to the white-list by setting the following environment variable:  
[https://docs.drone.io/reference/agent/drone-runner-privileged-images/](https://docs.drone.io/reference/agent/drone-runner-privileged-images/)

---

<div class="post-metadata">

**Author:** ![metal3d](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/metal3d/32/4750_2.png) [@metal3d](https://drone.discourse.group/u/metal3d)\
**Post date:** [May 25, 2019, 10:44pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/10 "2019-05-25T22:44:11Z")

</div>

Ho great !

OK, so maybe I wll be able to launch docker daemon my self, maybe in bash… I will take a look later.

Thanks a lot for all that help, for Drone, and so on 😉

---

<div class="post-metadata">

**Author:** ![metal3d](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/metal3d/32/4750_2.png) [@metal3d](https://drone.discourse.group/u/metal3d)\
**Post date:** [May 28, 2019, 11:08pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/11 "2019-05-28T23:08:56Z")

</div>

@bradrydzewski my god it’s now ok.

I now use “plugins/docker” as base image, and I can start docker daemon without privilege mode (I whitelisted my plugin)  
I can now do that:  
kind: pipeline  
name: default

```
steps:
  - name: s2i-build
    image: metal3d/drone-plugin-s2i
    pull: always
    settings:
      registry: docker-docker-registry.registry:5000
      insecure: true
      image: docker-docker-registry.registry:5000/metal3d/nginx:1.15-s2i 
      target: docker-docker-registry.registry:5000/metal3d/httptest

```

My entrypoint makes some tests, then starts daemon and waits docker to be ready (using `nc` on unix socket, and getting docker version)

After docker is ready, it launches:

```
s2i build ${DRONE_WORKSPACE_BASE} \
    --context-dir=${PLUGIN_CONTEXT-./} \
   ${PLUGIN_IMAGE} ${PLUGIN_TARGET}

```

So the build process is now able to get a “s2i base image” and build a target image.

I have to setup some params to push image or not, but it works great.

One more time, I’m so happy to have chosen Drone as CD solution 🙂

@bradrydzewski thanks a lot for your help

EDIT: I will give my plugin with opensource licence - wait some minutes

---

<div class="post-metadata">

**Author:** ![metal3d](https://yyz1.discourse-cdn.com/flex003/user_avatar/drone.discourse.group/metal3d/32/4750_2.png) [@metal3d](https://drone.discourse.group/u/metal3d)\
**Post date:** [May 28, 2019, 11:41pm UTC](https://drone.discourse.group/t/kubernetes-docker-build-failed/11018/12 "2019-05-28T23:41:26Z")

</div>

Plugin is ready:

[https://hub.docker.com/r/metal3d/drone-plugin-s2i](https://hub.docker.com/r/metal3d/drone-plugin-s2i)

I will need to implement docker login, of someone can help ?