This doesn’t seem to work?
build_args:
- GEMFURY_NPMRC=$GEMFURY_NPMRC
GEMFURY_NPMRC:
from_secret: GEMFURY_NPMRC
This is expected. GEMFURY_NPMRC=$GEMFURY_NPMRC
is bash syntax and build_args
is a Go []string
. A Go []string
will not expand bash syntax.
Hmm, I have this code working in Drone 0.8
docker build --build-arg GEMFURY_NPMRC=$GEMFURY_NPMRC
And it doesn’t seem to work with just using a commands:
, since it does a lot on startup.
Would it make sense to add a custom “commands:” where you can execute any docker command. In this example it feels like a thin abstraction on top of docker, that will always be one step behind.
Hmm, I have this code working in Drone 0.8
docker build --build-arg GEMFURY_NPMRC=$GEMFURY_NPMRC
the commands
section is converted to a bash file, which means it is able to process syntax. The build_args
section is Go string and is not able to process bash syntax. It probably makes a bit more sense when you understand the underlying implementation, but this is the expected behavior.
Would it make sense to add a custom “commands:” where you can execute any docker command. In this example it feels like a thin abstraction on top of docker, that will always be one step behind.
The docker plugin runs in privileged mode, and is restricted to docker build
and docker tag
commands. We cannot allow execution of arbitrary docker run
commands for security reasons.
It would be a valid usecase to use secrets in buildargs, even when buildkit introduces mounted secrets
opened 09:01PM - 26 May 15 UTC
status/needs-attention
area/security
Handling secrets (passwords, keys and related) in Docker is a recurring topic. M… any pull-requests have been 'hijacked' by people wanting to (mis)use a specific feature for handling secrets.
So far, we only _discourage_ people to use those features, because they're either provenly insecure, or not designed for handling secrets, hence "possibly" insecure. We _don't_ offer them real alternatives, at least, not for all situations and _if_, then without a practical example.
I just think "secrets" is something that has been left lingering for too long. This results in users (mis)using features that are not designed for this (with the side effect that discussions get polluted with feature requests in this area) and making them jump through hoops just to be able to work with secrets.
### Features / hacks that are (mis)used for secrets
This list is probably incomplete, but worth a mention
- **Environment Variables**. Probably the most used, because it's part of the ["12 factor app"](http://12factor.net). Environment variables are discouraged, because they are;
- Accessible by any proces in the container, thus easily "leaked"
- Preserved in intermediate layers of an image, and visible in docker inspect
- Shared with any container linked to the container
- **Build-time environment variables** (https://github.com/docker/docker/pull/9176, https://github.com/docker/docker/pull/15182). The build-time environment variables were **not** designed to handle secrets. By lack of other options, people are planning to use them for this. To prevent giving the _impression_ that they are suitable for secrets, it's been decided to deliberately **not** encrypt those variables in the process.
- **Mark .. Squash / Flatten layers**. (https://github.com/docker/docker/issues/332, https://github.com/docker/docker/pull/12198, https://github.com/docker/docker/pull/4232, https://github.com/docker/docker/pull/9591). Squashing layers will remove the intermediate layers from the final image, however, secrets used in those intermediate layers will still end up in the build cache.
- **Volumes**. IIRC some people were able to use the fact that volumes are re-created for each build-step, allowing them to store secrets. I'm not sure this actually works, and can't find the reference to how that's done.
- **Manually building containers**. Skip using a Dockerfile and manually build a container, commiting the results to an image
- **Custom Hacks**. For example, hosting secrets on a server, `curl`-ing the secrets and remove them afterwards, all in a single layer. (also see https://github.com/dockito/vault)
### So, what's needed?
- Add documentation on "do's" and "don'ts" when dealing with secrets; @diogomonica made some excellent points in https://github.com/docker/docker/pull/9176#issuecomment-99542089
- Describe the officially "endorsed" / approved way to handle secrets, if possible, using the _current_ features
- Provide roadmap / design for officially handling secrets, we may want to make this pluggable, so that we don't have to re-invent the wheel and use existing offerings in this area, for example, [Vault](https://www.vaultproject.io), [Keywiz](http://square.github.io/keywhiz/), [Sneaker](https://github.com/codahale/sneaker)
The above should be written / designed with both **build-time** and **run-time** secrets in mind
@calavera created a quick-and-dirty proof-of-concept on how the new Volume-Drivers (https://github.com/docker/docker/pull/13161) could be used for this; https://github.com/calavera/docker-volume-keywhiz-fs
> **Note:** Environment variables are used as the de-facto standard to pass configuration/settings, **including secrets** to containers. This includes official images on Docker Hub (e.g. [MySQL](https://registry.hub.docker.com/_/mysql/), [WordPress](https://registry.hub.docker.com/_/wordpress/), [PostgreSQL](https://registry.hub.docker.com/_/postgres/)). These images should adopt the new 'best practices' when written/implemented.
In good tradition, here are some older proposals for handling secrets;
- "Add private files support" https://github.com/docker/docker/pull/5836
- "Add secret store" https://github.com/docker/docker/pull/6075
- "Continuation of the `docker secret` storage feature" https://github.com/docker/docker/pull/6697
- "Proposal: The Docker Vault" https://github.com/docker/docker/issues/10310
It would be a valid usecase to use secrets in buildargs
Drone supports this. See my comment in this thread: Does the "verify" API endpoint work?
1 Like
kevinsimper
(Kevin Simper)
February 26, 2019, 10:31am
7
Nice it could be an idea to have something that generate documentation from the sourcecode so that you don’t have too