[solved] Docker build does not use cache (drone-cli exec)

Drone Support
1

Hello,

I’m trying to build a fairly simple angular2 project using a custom Docker file that runs “npm install” and “ng build”.
I’m using drone exec to test the pipeline locally. Drone is setup using the official installation instructions on local machine. It seems that there is no Docker Layer cache used at all. Every time I run drone exec it pulls the Image (node), and runs again all the steps which ends up taking a lot amount of time for something that hasnt changed. This is not the cache when I use docker locally, as it will correctly take advantage of the cached layers. Am I doing something wrong or is there no layer cache at all…? I’ve search in the forums but I only found outdated topics and github issues from 2016 that are solved.

I haven’t tested yet if this behavior is only related to drone-cli or not.

The pipeline is simple as well:

pipeline:
  build:
    image: node:9.11
    commands:
      - node --version
      - npm --version

  docker:
    image: plugins/docker
    repo: myregistry/myimage
    tags: latest
2

This is the expected behavior for this plugin.

Drone pipelines are isolated and ephemeral. Everything happens inside Docker containers, which are deleted when your pipeline completes. This means that any files or folders written inside these containers are deleted when the pipeline completes. The ephemeral nature of Drone is core to its design.

This design impacts caching and impacts the Docker plugin (plugins/docker) which runs docker-in-docker, and does not have access to your host machine. This is done for security and isolation purposes. This means anything that happens inside the container, including downloaded or cached layers, are deleted when the pipeline completes due to Drones ephemeral nature.

If you want to retain the build cache you have options:

1. Don’t use the Docker plugin

There is no requirement that you have to use the docker plugin. You can mount the host machine’s Docker socket into your build container using volumes [1] and then run docker build and docker push from the build section.

2. Create your own Docker plugin

The Docker plugin focuses on isolation and security. You can always create a plugin that focuses on different goals and use cases, and then share your plugin with the community.

3. Optimize your Image Build process

To quote Kelsey Hightower, “ship artifacts, not build environments”. Perhaps consider re-thinking how you build your images. You could run npm install and npm bundle outside of your Dockerfile and then use the ADD directive to copy your bundled javascript code into the image. This will allow you to ship a much smaller image, and you can use one of the cache plugins [2] to cache your node modules

4. Use a cache plugin with the Docker plugin

You can use a cache plugin [2][3] in conjunction with the Docker plugin to cache the layers and restore them during your build process.

[1] http://docs.drone.io/docker-volumes/
[2] http://plugins.drone.io/tags/cache/
[3] https://github.com/drone-plugins/drone-docker/issues/172#issuecomment-402566466

1 Like