Problem:
We have a few organizations built in Gitea, it seems drone trusts only one organization right now, for others the pipline shows the following on the clone step:
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Trouble shooting:
The organization that works is called “Org1”. The organization that does not work is called “Org2”. If we try to use Org1 with the branches that were in Org2, drone works fine and is able to clone the branches in Or1. If Org2 is used, it seems drone does not trust the interaction and fails as above shows.
I think we narrowed this down to building more orgsecrets with the drone cli utility, however I keep getting the following error with the tool (Using add, ls, info, etc all give the same error):
$ ./drone --server 10.10.10.10 --token longadmintoken orgsecret ls
$ Get 10.104.8.28/api/secrets: unsupported protocol scheme “”
It seems that the drone utility requires me to specify a server and admin token, is this right? The documentation does not mention this at all (Maybe I am using this utility wrong?)
Our environment:
Gitea, drone, drone-runner, and database all live in docker containers brought up with docker-compose file.
We have a few organizations built in Gitea, it seems drone trusts only one organization right now, for others the pipline shows the following on the clone step:
I am not sure what is meant by trust. I think it would be helpful to describe the problem in terms of what you are trying to achieve and how (sample configuration, steps to reproduce, etc) as well as any other relevant details (complete build output, etc). Right now it is unclear where the provided error messages come from and why or what you are trying to accomplish.
It seems that the drone utility requires me to specify a server and admin token, is this right? The documentation does not mention this at all (Maybe I am using this utility wrong?)
Re: trust, that speculation. We did not deploy this system so I am not 100% on how the system as a whole.
If you need a more defined problem here is a picture of the error we see in drone for the branch being tested in a separate group:
Our goal is to allow this branch in a separate organization to verify keys and perform testing, just like all of our other branches in a different organization.
Drone uses git+https to clone the repository. Based on the example logs you provided we can see that someone configured Drone to use a custom clone plugin that overrides the default behavior to use git+ssh as well as some other custom logic. You will need to contact the clone plugin author (presumably the person within your company that setup your installation) since we cannot provide support for a third party plugin that we did not create.
@bradrydzewski
Thanks for the details. We have looked all over the system and talked to the admin that deployed it, we see no plugins being used. Is there a specific place we should be looking for custom plugins? How can you tell that a custom plugin was used?
Also why is the drone script not working properly to access the system, do I have the syntax wrong?
./drone --server 10.10.10.10 --token *longadmintoken* orgsecret ls
If you compare the default output with the output in your screenshot you will see they are significantly different. This is how I can be certain we did not write the code being used to clone the commit in your screenshot.
If I had to guess (since I don’t have a copy of your yaml or your configuration) I would say that someone disabled the default clone step in your yaml, and then added a custom clone step that executes custom shell script inside an alpine/git image. It is also possible they wrote a custom plugin or extension that sets default logic.
Here is the drone configuration file we are using, I believe under commands is what you are referring to as the custom shell script, is that correct?
It looks like the script is copying the $SECRET_KEY and $KNOWN_HOSTS to allow trusted secure access to our git repo without using a password. those variables seem to be linked to “secret_key” and “known_hosts”, how can I tell where they are linked to?
I think the problem is different organizations are using different keys, and I need to add those keys to our git repo’s .ssh/known_hosts file to trust them.
Config:
kind: pipeline
type: docker
name: Agave
clone:
disable: true
steps:
# Clone step: We have to clone manually as the gitea server is set up to
# disallow HTTP cloning. This forces the use of a key that to ensure the
# download is secure and encrypted. This secret key is available through
# the drones "secret" mechanism for sensitive data.
- name: clone
image: alpine/git
environment:
# Turn secrets into environment variables
SECRET_KEY:
from_secret: secret_key
KNOWN_HOSTS:
from_secret: known_hosts
commands:
- mkdir "$HOME/.ssh"
- echo "$SECRET_KEY" > "$HOME/.ssh/id_ed25519"
- chmod 600 "$HOME/.ssh/id_ed25519"
- echo "$KNOWN_HOSTS" > "$HOME/.ssh/known_hosts"
- chmod 600 "$HOME/.ssh/known_hosts"
- git clone "git@gitea.cybertax.cso.com:$DRONE_REPO.git" .
- git checkout "$DRONE_COMMIT"
# Build step: install and/or update node modules for project
- name: buildFrontend
image: node:12
depends_on:
- clone
commands:
- cd frontend
- npm install
- npm test
The values point to secret variables that someone configured. You would need to ask the person that created these commands and created these secret variables what their values are. Since these parameters are defined and created by the end-user, and not by Drone, we cannot really advise on their use or their contents (since we don’t know).
@bradrydzewski
Fair enough, lets table the discussion of the problem at hand for a moment.
I suppose I am really just looking for how to use the drone command to manage my system from afar to create a new orgsecret.
I want to perform “drone orgsecret ls”, to see what is listed first, however as drone is containerized I need to use the script outside of the container. I am running the script on the vm that hosts the docker container. The documentation from drone found here only goes over how to run the commands if drone was not a docker container (I think).
It looks like I need to specify the server address.
How can I list and then add a new orgsecret in a containerized environment?
Again, the command I use below keeps coming back with "10.10.10.10/api/secrets: unsupported protocl scheme """ Any idea why?
./drone --server 10.10.10.10 --token *longadmintoken* orgsecret ls
If I am going about this wrong please let me know.
Hi Dave, the command line tools are not intended to be run in a container. They are intended to be downloaded and run from your laptop, from your terminal, using your Drone API token. The command line tools connect to your Drone server using its remote API. You can find instructions at https://docs.drone.io/cli/install/ and https://docs.drone.io/cli/configure/
And of course the problem was SSH private key being used by the drone agent (I think that is correct term? Correct me if I am wrong) was not trusted by the git server. The work around was echoing the exact private key instead of “secret_key” as a private key already trusted in Gitea (Found in /data/git/.ssh/authorized_keys in the docker container)
This works, but is not for production as the key is viasble. I would like to use the drone cmd utility to be more secure.
I traced the “secret_key” and “known_hosts” environmental variables back to the orgsecrets. Using “orgsecret ls” shows there was none made for the other organization (only “Org 1”). “secret_key” was for Org1 and known_hosts was for Org1. So I created two new values “secret_key_org2” and “known_hosts_org2”, however using these values in the .drone.yml fails.
Is there a way I can look at the exact values for orgsecret entries? I see there is an “orgsecret info” but I keep getting a 404 page not found. If it is not possible to uncover these values, what is the correct format to add a id_ed25519 private key (e.g. do new line characters matter?)
Side question, it seems I have to keep using “–skip-verify” flag to get access with the drone command line tool. This was a hidden flag I found in the source code for the drone command line tool. Is there a way to specify a cert to trust?
And of course the problem was SSH private key being used by the drone agent
Technically speaking the agent is not using your ssh key. Your custom clone commands, executed inside of an alpine container, are using the ssh key. The agent is responsible for exposing this value to your container so that it can be used by your commands.
Is there a way I can look at the exact values for orgsecret entries? I see there is an “info” but I keep getting a 404 page not found.
This command requires system administrator privileges. The 404 error would indicate you have insufficient permissions. Learn more about configuring system admins here.
If it is not possible to uncover these values, what is the correct format to add a id_ed25519 private key (e.g. do new line characters matter?)
You should use the format required by your custom clone commands that your organization wrote. We did not write the clone commands, and therefore cannot comment on their required inputs.
what is the correct format to add a id_ed25519 private key (e.g. do new line characters matter?)
To preserve newlines we recommend uploading the secret from a file. You can find an example here: drone orgsecret add | Drone
Using the secret from a file as input for orgsecret worked!
I can now use the orgsecrets as environmental variables for separate organizations properly
Commands I used (For those finding this from search engines) :
Now if a new org is created, I just need to run these commands again for each org, but with new key files for each project.
This was the main purpose of what I was trying to accomplish. So technically since the person who created the original .drone.yml used those shell scripts to add keys and known_hosts, that section is considered a custom plugin?
This command requires system administrator privileges. The 404 error would indicate you have insufficient permissions. Learn more about configuring system admins here.
Still does not work, I tried using the token for the admin user created in my docker compose too. What does “info” provide?
Its been a while since this post was visited.
However I am running into the same issue again… I know its something I am forgetting.
Problem:
Still get the “Host key verification failed.” when drone tries to run the drone.yml file.
Trouble shooting
Created a new org in gitea
In drone, I used the orgsecret command with the drone binary to create a new secret and known_hosts entry for the new org. Confirmed the key and known_hosts file I am pointing to with “@” exist and have the correct permissions. Confirmed the orgsecret entries exist with “orgsecret ls” api command.
Created a new branch under the new org in gitea. Added the drone deploy key.
In Gitea, made sure the drone user had read/write access to the branch
Logged into drone web interface with my admin account, activated the branch.
added a .drone.yml file with the environmental variables of the new SECRET_KEY and KNOWN_HOSTS.
Drone can communicate with the webook and I get a 200 ok.
The test continues to fail with the “Host key verification failed.”
I figured out the problem. I know our pipeline cloning is not supported, but it does work for us. I want to share the problem and solutions I found today.
Make sure you check the orgsecret string name, it must be exact, case sensitive obviously.
I needed to update the "SECRET_KEY and KNOWN_HOSTS “from_secret” values, to the orgsecret string names I created, which I did not do originally.
I was creating a secret for the wrong files. “secret_key” worked with the private key. “known_hosts” worked with the known_host file. I was not passing the known_hosts file so that is why it failed in the first place.
I have made excessive notes on how our process works so I think I am good now!
