Loving autoscale!
I’m trying to figure out an elegant solution for ccache - which is a caching mechanism for C++ software.
On my on-permise drone deployment, I mount a host directory for this cache.
What would be a good equivalent for autoscaled agents? Should I mount some network directory? Is there an elegant way to do this?
Currently I moved to my own rolled agents with a mounted Google Cloud Storage directory.
Any idea how to do the equivalent with autoscale?
I’m not familiar with ccache so the following solution may not work for you.
You could use s3 storage with drone-s3-cache plugin to allow caching across agents that isn’t dependent on mounted storage. It basically will create a tarball of files you want to save to use later and store it in s3 storage. Then subsequent build can pull/expand the tarball with the “cached” files.
If you don’t have access to s3 storage you can host your own with a minio server.
also note that I believe google cloud storage is compatible with the S3 protocol, so you should be able to use the drone-s3-cache plugin. But if not, a drone-gcs-cache-plugin would be a welcome addition.
linke to plugin http://plugins.drone.io/drone-plugins/drone-s3-cache/
Thanks (also @jmccann), I’m aware of the cache plugins. They are not fit, since the cache is pretty large (around 2GB for our codebase), so saving and restoring each time would be too long.
@kobigurk in this case, you should still be able to create a google cloud storage mount on your agent instances. I am not sure exactly what is required to create the mount, but I’m guessing it can be done by overriding the default cloud-init file used by the autoscaler to configure instances.
Note that overriding the cloud-init file is an advanced feature of the autoscaler. It is not documented yet, but I did write up some notes for someone using a custom cloud-init with amazon that you can use as reference for gcp: https://notehub.org/kx2kf
That’s actually exactly what I need! I already know how to do it on my self-rolled agents, so this is perfect.
awesome, once you get a cloud-init template working, please share 