Image pulled from AWS ECR is not available in the next step

Drone Support
1

We use a custom docker image for the build step and this image should be pulled from AWS ECR. I searched this forum and came across solutions that pull the image in one step to make it available in the next step but is doesn’t seem to work for me. Here is my .drone.yml:

---
kind: pipeline
name: default

volumes:
- name: docker
  host:
    path: /var/run/docker.sock

steps:
- name: Pull build-container-ansible
  image: docker.io/mludvig/docker-builder
  volumes:
  - name: docker
    path: /var/run/docker.sock
  commands:
  - aws --region ap-southeast-2 ecr get-login --no-include-email --registry-ids 123456789012 | sh
  - docker pull 123456789012.dkr.ecr.ap-southeast-2.amazonaws.com/build-container-ansible:latest
  - docker images

- name: Custom image
  image: 123456789012.dkr.ecr.ap-southeast-2.amazonaws.com/build-container-ansible:latest
  volumes:
  - name: docker
    path: /var/run/docker.sock
  commands:
  - ansible --version

The first step works and pulls the container from ECR:

+ aws --region ap-southeast-2 ecr get-login --no-include-email --registry-ids 123456789012 | sh
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
+ docker pull 123456789012.dkr.ecr.ap-southeast-2.amazonaws.com/build-container-ansible:latest
latest: Pulling from build-container-ansible
Digest: sha256:997671bbaea...9e21a94484d47
Status: Image is up to date for 123456789012.dkr.ecr.ap-southeast-2.amazonaws.com/build-container-ansible:latest
+ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
drone/drone 1.2.1 c9e98faa9680 5 days ago 63.2MB
plugins/slack latest 5dfdbac184c1 2 weeks ago 9.26MB
mludvig/docker-builder latest cb7cbfa59113 2 weeks ago 599MB
amazon/amazon-ecs-agent latest 267bac512a39 5 weeks ago 57.1MB
amazon/amazon-ecs-pause 0.1.0 54d8403124ce 5 weeks ago 954kB
plugins/ansible latest e43b2eee175f 7 weeks ago 201MB
plugins/git latest 90e2d8d126d9 3 months ago 66.3MB
drone/git latest 00d7b517c0b9 5 months ago 67.6MB
123456789012.dkr.ecr.ap-southeast-2.amazonaws.com/build-container-ansible latest dae4f9c3a6ac 11 months ago 1.19GB

On the last the image is there. When I login to the EC2 instance it’s indeed in the docker images list.

However the next pipeline step that should be using the builder-docker-ansible fails:

default: Error response from daemon: Get https://123456789012.dkr.ecr.ap-southeast-2.amazonaws.com/v2/build-container-ansible/manifests/latest: no basic auth credentials
default – Custom image: Failure

I can’t figure out why it doesn’t use the image pulled in the first step and keeps reaching out to ECR?

We are using Drone 1.2.1 in a single-server configuration, no agents.

Any ideas?

Thanks!

2

The image is likely available the the next step, however, you are using the :latest tag which means drone will always check the registry to see if a newer version is available. You can disable this behavior with the following:

  - name: Custom image
    image: 123456789012.dkr.ecr.ap-southeast-2.amazonaws.com/build-container-ansible:latest
+   pull: if-not-exists
3

Thanks, that works!

Any idea how to do it in the old Drone 0.8 manifest syntax? We’ve got a number of repos to be migrated from 0.8 to 1.2.1 and many of them use this private docker image for building. We will eventually rewrite their .drone.yml configs to the new syntax but for start would like to just migrate them with minimal changes.

Tried pull: if-not-exist and that makes the webhook fail (Can not unmarshal ‘if-not-exist’) and also tried pull: false but that doesn’t have any effect. The ECR still fails.

When running under 0.8 it doesn’t try to pull the image all the time. How do I make this work in 0.8 yaml file running in 1.2.1?

Cheers

4

you could run drone convert for those repositories to automatically convert the yaml and then manually add pull: if-not-exists. Or you could send a patch to the automatic conversion code [1] and set pull: if-not-exists by default as long as the step does not specify pull: true.

[1] https://github.com/drone/drone-yaml/blob/master/yaml/converter/legacy/internal/config.go#L195

5

Thanks for the hint that it’s caused by the :latest tag. I have now added a :stable tag to our builder image and that seems to fix the problem in the short term too. Eventually we will convert our yamls to the new format but for now we need to move it all across to the new Drone first.

Thanks for your help!