When I follow https://docs.drone.io/extend/secrets/vault/install/ and run the drone-vault command, I get
time=“2018-12-13T22:50:14Z” level=warning msg=“missing vault address”
time=“2018-12-13T22:50:14Z” level=info msg=“server listening on address :3000”
What is the environment variable I need to set? I only see SECRET_KEY.
Yes, sorry, you also need to pass the standard VAULT_ environment variables. See https://www.vaultproject.io/docs/commands/#environment-variables
I seem to be having a related issue that, even when a VAULT_ADDRESS is supplied, I’m getting a “Invalid or Missing Signature” error from requests to that address. Setting the VAULT_TOKEN doesn’t seem to make a difference, even when including the token in the header, i.e:
$ curl --header "x-Vault-Token: $VAULT_TOKEN" http://127.0.0.1:3000
Invalid or Missing Signature
I’m looking to prototype vault with a simple token based authentication, but I think I must be missing something about authentication methods that doesn’t translate from the vault docs to drone/vault.
thanks!
The endpoint is protected and cannot be directly access via curl without proper authorization. The header you are providing is Vault-specific, and is not used by Drone plugins. If you want to test a plugin, the easiest way is using the command line tools:
$ drone plugins secret get --help
NAME:
drone plugins secret get - get the named secret
USAGE:
drone plugins secret get [command options] secret
OPTIONS:
--ref value git reference (default: "refs/heads/master")
--source value source branch
--target value target branch
--before value commit sha before the change
--after value commit sha after the change
--event value build event
--repo value repository name
--endpoint value plugin endpoint [$DRONE_SECRET_ENDPOINT]
--secret value plugin secret [$DRONE_SECRET_SECRET]
--ssl-skip-verify value plugin ssl verification disabled [$DRONE_SECRET_VERIFY]
For example:
export DRONE_SECRET_ENDPOINT=http://127.0.0.1:3000
export DRONE_SECRET_SECRET=<plugin secret>
$ drone plugins secret get path/to/secret --repo=octocat/hello-world --branch=master
If you have any questions about the CLI command, you can see the underlying source code here: https://github.com/drone/drone-cli/blob/master/drone/plugins/secret/get.go
thanks for the information. How would you then set up authentication when running drone (agent and server) and the vault in concurrent docker containers?
we have install instructions at https://docs.drone.io/extend/secrets/vault/install/
Following those instructions led to the issue. E.g.
$ docker run -d \
--env=SECRET_KEY=${SECRET} \
--env=VAULT_ADDR=http://0.0.0.0:3000 \
--publish=3000:3000 \
drone/vault
$ docker run -d \
--env=DRONE_SECRET_SECRET=${SECRET} \
--env=DRONE_SECRET_ENDPOINT=http://0.0.0.0:3000 \
--name=agent \
drone/agent
$ curl http://localhost:3000
Invalid or Missing Signature
$ export VAULT_ADDR='http://0.0.0.0:3000'
$ vault kv put secret/hello foo=world
Error making API request.
URL: GET http://0.0.0.0:3000/v1/sys/internal/ui/mounts/secret/hello
Code: 400. Raw Message:
Invalid or Missing Signature$ curl http://localhost:3000
Invalid or Missing Signature
This is expected because you cannot manually curl the URL. You need a special client to communicate with the plugin, which is why we provide the drone plugins secret get command.
$ export VAULT_ADDR='http://0.0.0.0:3000'
$ vault kv put secret/hello foo=world
Error making API request.
I see you have configured all of these services to communicate with eachother using localhost or 0.0.0.0. This will not work because each one of these services is running in a container with an isolated network. The loopback address will only give you access to other services running inside the same container. Since you are trying to communicate between containers, you need to use the internal Docker IP or hostname for the container.
If you are new to Docker networking you might want to use something like Docker compose, which can simplify the setup process and will automatically configure the network and hostnames.
Thanks for the clue, and I apologize for the localhost diversion… but even when I use the loopback (e.g. exactly as written in the install instructions) I’m getting the same API error.
I’m not seeing anything different with docker-compose.
My assumption is that --publish should be doing all the heavy lifting for exposing the vault endpoint to the host. My setup is a small server running drone in a docker container and that I’d like to be able to add globally-persistent secrets that are available to all drone pipelines, as a more portable alternative to adding them one-by-one in the drone UI. Is there a way to provide token-based authentication through the host?
but even when I use the loopback (e.g. exactly as written in the install instructions) I’m getting the same API error.
I think the install instructions show an IP address, not a local IP address or loopback address. If you found documentation that shows otherwise please let me know.
My assumption is that
--publishshould be doing all the heavy lifting for exposing the vault endpoint to the host.
Publish will expose the port on the host machine, but 0.0.0.0 and localhost refers to the network inside of the container, not the host machine. You therefore cannot access a host-machine address and port from inside a container using 0.0.0.0 or localhost. Specifically, you cannot reach vault at 0.0.0.0 from inside the plugin container.
If you find yourself struggling with networking, you could just set the agent and plugin container to --network=host and avoid using an isolated network all together.
Is there a way to provide token-based authentication through the host?
Sorry I’m not sure I understand the question.
You are right of course, I confused myself in my last post and wrote down the incorrect thing. I am using the actual host IP address (instead of 1.2.3.4), as suggested in the install docs. I am still getting the same ‘Invalid or Missing Signature’ error when making the API request. VAULT_ADDR is the same as DRONE_SECRET_ENDPOINT, secret keys match, etc.
My assumption is that I still must be setting up authentication incorrectly, though I will try the same setup on another machine.
I am still getting the same ‘Invalid or Missing Signature’ error when making the API request.
Are you seeing this error in the agent logs? I just want to clarify because I know you were previously trying to curl the image, which is not possible.
My assumption is that I still must be setting up authentication incorrectly, though I will try the same setup on another machine.
You can pass DEBUG=true to the vault plugin to get more details. This might help you further debug the issue.
Are you seeing this error in the agent logs?
Yes: output from running docker-compose
Attaching to drone-secrets-1, drone-server-1, drone-agent-1
drone-secrets-1 | time="2019-01-25T22:06:05Z" level=info msg="server listening on address :3000"
drone-secrets-1 | time="2019-01-25T22:06:05Z" level=debug msg="vault: token rereshing disabled"
drone-secrets-1 | time="2019-01-25T22:07:04Z" level=debug msg="secrets: invalid or missing signature in http.Request"
The message is coincident with the API request, though unfortunately doesn’t contain much extra information.
It would be helpful to see your full configuration (docker-compose.yml file) but at this point I am not sure there is much more I can do to assist. I would therefore recommend you jump into the code and build the Vault plugin from source, which should help you triage this at a lower level. All the plugin source is available at github.com/drone/drone-vault
It turns out my error was quite simple, I was using the wrong vault address.
export VAULT_ADDR='http://0.0.0.0:3000'
should have been
export VAULT_ADDR='http://0.0.0.0:8200'