Is Caching on Kubernetes Possible?

Drone Support
1

Hello! Just started using Drone and plan on migrating to it from Azure Pipelines. Is there a way to do caching utilizing the PVC provisioned to the droneci server so I can cache my large Github repo or perhaps docker image layers? My setup is agentless with the jobs spawning in a different namespace than the droneci-server which has a 50GB PVC allocated to it. I also got the cleanup operator running to keep things neat :slight_smile: Just curious if anyone has done this before and could give me some insight on how to do it.

Thank you!!

2

For Docker layer caching I recommend using cache-from, which strikes a pretty good balance between performance while still keeping the build environment ephemeral. This is covered in the following blog post https://laszlo.cloud/how-using-cache-from-can-speed-up-your-docker-builds-in-droneci

3

How about caching other things such as the repository, go modules, ect…? My idea is to perhaps create an AzureFile PVC, somehow get it mounted in the pipeline job, and then be able to share the cached repo, go modules, ect… across all of my pipelines, no matter what node they land on.

4

Drone delegates caching to plugins. Here are some existing plugin options:

5

For anyone stumbling across this, I ended up using this:

volumes:
- name: dockersock
  temp: {}

  - name: docker-go-cache
    image: plugins/docker
    depends_on: [ get_submodules ]
    volumes:
    - name: dockersock
      path: /var/run
    settings:
      registry: **REDACTED**
      username:
        from_secret: **REDACTED**
      password: 
        from_secret: **REDACTED**
      repo: REDACTED/go.cache
      cache_from: REDACTED/go.cache
      dockerfile: docker/drone/Dockerfile
      target: cache
      tags: 
        - latest
        - ${DRONE_COMMIT}
      dry_run: false
      auto_tag: false

Then I have a dockerfile that looks like this:

FROM REDACTED/dev:18.04 as cache

# Force the go compiler to use modules

ENV GO111MODULE=on

# We want to populate the module cache based on the go.{mod,sum} files.

COPY go.mod .

COPY go.sum .

#This is the ‘magic’ step that will download all the dependencies that are specified in

# the go.mod and go.sum file.
# Because of how the layer caching system works in Docker, the go mod download
# command will _ only_ be re-run when the go.mod or go.sum file change
# (or when we add another docker instruction this line)

RUN go mod download

Then any downstream builds will use the --from-cache=<repo>/go.cache:${DRONE_COMMIT}. This seems to effectively cache the go modules from one build to the nest.