Docker multi-stage build does not maintain image conents

Drone Support
1

I have one Dockerfile with two stages (dev, with necessary golang packages and a binary for testing) and prod (alpine linux, binary/scripts/configuration files only).

I’m trying to condense a Golang project down into a simple .drone.yml (tested on drone-cli 0.8.3 and 0.8.5) file that will do a few things:

  1. Builds both images and tags them
  2. Runs a test script
  3. Builds and/or runs a production container

The problem I’m facing is that when I do

drone exec --local

The DB and docker images build fine, but when I get to the test phase of the pipeline, the test script fails because the final binary can’t be found. I’ve confirmed in multiple ways that the binary does in fact exist, through running the container and ls’ing, etc.

This also goes for the godog binary in /go/bin, hence the go get commands in drome.yml

.drone.yml:

workspace:
      base: /go
      path: src/project
    context: .

    pipeline:
      build:
        image: docker
        commands:
            - docker build --tag project:latest --target project-dev --file ./docker/dev/Dockerfile.dev .
            - docker build --tag project:latest --target project-prod --file ./docker/dev/Dockerfile.dev .
        volumes:
          - /var/run/docker.sock:/var/run/docker.sock
      test:
        image: project-dev:latest
        pull: true
        commands:
          - go get github.com/DATA-DOG/godog/cmd/godog
          - ./scripts/test.sh

Dockerfile.dev:

FROM golang:1.10 AS project-dev

RUN apt-get update
RUN apt-get install -y protobuf-compiler

COPY . /go/src/project
WORKDIR /go/src/project

RUN go get ./...
RUN go get github.com/DATA-DOG/godog/cmd/godog
RUN go get github.com/derekparker/delve/cmd/dlv
RUN go get github.com/golang/protobuf/protoc-gen-go

RUN ls -hal
RUN ./scripts/build.sh
RUN ls -hal

FROM alpine:latest AS project-prod

#Install bash so we can run actual scripts
RUN apk update
RUN apk add bash

EXPOSE 9876

RUN mkdir -p /project
WORKDIR /project

COPY --from=project-dev:latest /go/src/project/scripts/test.sh /project/scripts/
COPY --from=project-dev:latest /go/src/project/bin/project /project/bin/
COPY --from=project-dev:latest /go/src/project/etc/config.json /project/etc/
COPY --from=cmapi-dev:latest /go/src/project/docker/prod/prod-entrypoint.sh /project/prod-entrypoint.sh

ENTRYPOINT ["/bin/bash", "/project/prod-entrypoint.sh"]
2

Is there are reason you aren’t using the Docker plugin to build and publish your image? I recommend looking at how we (the drone project) builds and publishes image for Go programs. Here are a few examples:

Typically we would recommend you build and test your Go code and create the image after the tests are passing. This results in more compact Docker images that contain only your Go binary.

As I side note, I would recommend you remove the pull flag, since this will instruct drone to pull a new image from the registry thus overwriting the one you just created.

        image: project-dev:latest
-       pull: true
3

No particular reason other than it was the first route that got me the furthest.

I’ve modified my .drone.yml to look like this:

workspace:
  base: /go
  path: src/project
context: .

pipeline:
  docker:
    image: plugins/docker
    dockerfile: docker/dev/Dockerfile.dev
    tags:
      - latest
  test:
    image: project-dev:latest
    commands:
      - ./scripts/test.sh

It builds the image, but now I’m getting another error:

[docker:L283:114s] invalid from flag value project-dev:latest: pull access denied for project-dev, repository does not exist or may require 'docker login'

Does this mean that the docker plugin requires a repository to push to? Or can I still use it to build and store images locally without needing a repo?

4

This would be the recommended approach:

pipeline:
  build:
    image: golang
    commands:
      - go get github.com/DATA-DOG/godog/cmd/godog
      - go get github.com/derekparker/delve/cmd/dlv
      - go get github.com/golang/protobuf/protoc-gen-go
      - bash scripts/build.sh

Executing docker build and docker run is not recommended (some exceptions apply) since drone is already executing your builds in a container environment. If you want to build and publish an image at the end of your pipeline, you can simplify your dockerfile and just add the static Go binary:

FROM alpine:3.6 as alpine
RUN apk add -U --no-cache ca-certificates

FROM alpine:3.6
EXPOSE 9876

COPY --from=alpine /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

ADD some-binary /bin/
ENTRYPOINT ["/bin/some-binary"]