Cannot clone private gitea repo (could not read Username for...) 0.8.0-rc.3, gitea 1.2.0-rc2

Hello there, I have spent about 2 days digging through even the weird corners of the internet to get a picture of what’s going on, so I think I have a pretty good idea about the whole thing now.

I am unable to clone a private repository from gitea, when I set the repository to public, the repository is succesfully cloned.

After all the tests I have run I have a feeling that the login information is not properly passed to plugins/git

The setup

I have tested using drone 0.7, drone 0.8.0-rc.3 and even the latest image.
I have tested using sqlite and postgres db backend
Gitea is at version 1.2.0-rc2, I couldn’t get any previous versions to authenticate succesfully, I think the drone support was added recently?

The settings

As per many other topics here and elsewhere, I have confirmed the following:

  1. The repository (gitea) was private before I added it and is still private
  2. Drone settings for the repository are set to private as well - if I set the repository to public and keep drone settings as private, it still clones it succesfully
  3. The user account that added the repository has not revoked oauth access (can be seen in gitea and I am using the user to login to drone and I am still able to add/remove repositories with said account and I have started from scratch multiple times)
  4. I am able to clone the repository using a standalone plugins/git image provided I pass in the netrc authentication configuration (either another oAuth token or username/password)
discourse issue?

For some reason I cannot follow a list with codeblock on discourse - it will not format the code block unless I put something between the list and the block.

docker run --rm -i plugins/drone-git <<EOF 
{
    "repo": {
        "clone_url": "https://gitea.my.domain/Org/Repo.git",
        "private": true
    },
    "build": {
        "event": "push",
        "commit": "77b6f3e3da1cb94eb4ce8b6ce32ea89984a0409a",
        "ref": "refs/heads/master"
    },
    "workspace": {
        "root": "/drone/src",
        "path": "/drone/src/gitea.my.domain/org/repo",
        "netrc": {
            "machine": "gitea.my.domain",
            "**login**": "{LOGIN REDACTED}",
            "**user**": "{PASSWORD/OAUTH_TOKEN REDACTED}"
        }
    }
}
EOF
Drone Git Plugin built from 43dcd64
$ git init
Initialized empty Git repository in /drone/src/gitea.my.domain/org/repo/.git/
$ git remote add origin https://gitea.my.domain/Org/Repo.git
$ git fetch --no-tags origin +refs/heads/master:
From https://gitea.my.domain/Org/Repo
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> origin/master
$ git reset --hard -q 77b6f3e3da1cb94eb4ce8b6ce32ea89984a0409a
I was a bit confused with the property naming

This might be a question for @tboerger, the netrc Type is a bit weird in version 0.4 which is using the type from deprecated drone-plugin-go by @bradrydzewski

I have to pass in the username to “login” property, and password/oauth token to “user” property (there might be a different way to pass in oauth, but I haven’t found it

https://github.com/drone-plugins/drone-git/blob/release/0.4/vendor/github.com/drone/drone-plugin-go/plugin/types.go#L54

I also get the same result running with env variables

docker run --rm -e DRONE_REMOTE_URL="https://gitea.my.domain/Org/Repo.git" -e DRONE_NETRC_USERNAME="login_redacted" -e DRONE_NETRC_PASSWORD="oauthToken/Password_redacted" -e DRONE_NETRC_MACHINE="gitea.my.domain" -e DRONE_COMMIT_SHA="77b6f3e3da1cb94eb4ce8b6ce32ea89984a0409a" -e DRONE_COMMIT_REF="refs/heads/master" plugins/git
+ git init
Initialized empty Git repository in /.git/
+ git remote add origin https://gitea.my.domain/Org/Repo.git
+ git fetch --no-tags origin +refs/heads/master:
From https://gitea.my.domain/Org/Repo
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> origin/master
+ git reset --hard -q 77b6f3e3da1cb94eb4ce8b6ce32ea89984a0409a

The drone builds

Anyways, this brings me back to the builds
Repository (gitea): Private
Drone repository setting: Private
Even tried to add secrets for netrc, however they are not passed to the plugin directly without creating my own clone pipeline… I suppose? (tried netrc_username, netrc_password, netrc_machine, drone_netrc_username, drone_netrc_password, drone_netrc_machine)

Resulting log from such a build is

+ git init
Initialized empty Git repository in /drone/src/gitea.my.domain/Org/Repo/.git/
+ git remote add origin https://gitea.my.domain/Org/Repo.git
+ git fetch --no-tags origin +refs/heads/master:
fatal: could not read Username for 'https://gitea.my.domain': No such device or address
exit status 128

This seems to me, that .drone.yml is succesfully fetched from the repository, however the login information is not properly passed to plugins/git? I assumed this because if I improperly format the .drone.yml, I get

Invalid or missing pipeline section

Now when I change the settings to
Repository (gitea): Public
Drone repository settings: Public (same result if this setting is Private)

+ git init
Initialized empty Git repository in /drone/src/gitea.my.domain/Org/Repo/.git/
+ git remote add origin https://gitea.my.domain/Org/Repo.git
+ git fetch --no-tags origin +refs/heads/develop:
From https://gitea.my.domain/Org/Repo
 * branch            develop    -> FETCH_HEAD
 * [new branch]      develop    -> origin/develop
+ git reset --hard -q 1bb52e8a1fd68ad79a7075c46b63c804dc687fd1
+ git submodule update --init --recursive

So this is where I am stuck at the moment, I am more than happy to help with debugging if needed, I am just not sure where to go from here

I would recommend starting from a scratch environment.

FIRST I would recommend using the GOGS_ configuration. I believe there are known issues with the gitea driver, and whenever people have issues, switching to the gogs driver seems to solve things.

-     - DRONE_GITEA=true
-     - DRONE_GITEA_URL=http://gitea.mycompany.com
+     - DRONE_GOGS=true
+     - DRONE_GOGS_URL=http://gitea.mycompany.com

SECOND look at the gitea logs to see if you can find anything useful.

THIRD verify that drone knows your repository is private. This is important because drone will not provide git credentials to the pipeline if it thinks your repository is public.

$ drone repo info Org/Repo
Owner: Org
Repo: Repo
Type: git
Config: .drone.yml
Visibility: private
+Private: true
Trusted: true
Gated: false
Remote: https://gitea.my.domain/Org/Repo.git

FOURTH verify that credentials are being passed to the build environment. Drone will always pass credentials to a private repository. You can verify this using the exact yaml below. It will dump the contents of the netrc file to your build logs.

clone:
  default:
    image: alpine
    commands:
      - echo $DRONE_REPO_PRIVATE
      - cat /root/.netrc

pipeine:
  build:
    image: alpine
    commands:
      - echo hello

What are the results? This is important.

FIFTH test the credentials being used to clone your repository. These credentials are cached by drone and a simple gitea upgrade could have invalidated them. You can test locally, on your laptop, to verify the credentials are valid:

git clone http://<netrc-login>:<netrc-password>@gitea.my.domain/Org/Repo.git

Hello Brad, thank you for your help, I will try starting from scratch in the next week and report the results, so far I have tried:

  • Changing the driver from gitea to goggs as you suggested
  • I will check the logs next week with debugging enabled (if possible), current logs only show unauthorized access

[Macaron] 2017-09-08 17:01:32: Started GET /Org/Repo.git/info/refs?service=git-upload-pack for 172.19.0.2
[Macaron] 2017-09-08 17:01:32: Completed /Org/Repo.git/info/refs?service=git-upload-pack 401 Unauthorized in 13.545093ms
[Macaron] 2017-09-08 17:01:32: Started GET /Org/Repo.git/info/refs?service=git-upload-pack for 172.19.0.2
[Macaron] 2017-09-08 17:01:32: Completed /Org/Repo.git/info/refs?service=git-upload-pack 401 Unauthorized in 19.824002ms

  • I verified that drone knows it is a private repo:

Owner: Org
Repo: Repo
Type: git
Config: .drone.yml
Visibility: private
Private: true
Trusted: false
Gated: false
Remote: https://gitea.my.domain/Org/Repo.git

  • I checked the credentials being passed into the build environment

+echo $DRONE_REPO_PRIVATE
true
+cat /root/.netrc
machine gitea.my.domain
login {OAUTH_TOKEN_REDACTED}
password x-oauth-basic

  • I have tried to clone the repo using these credentials by calling

git clone http://&lt;netrc-login&gt;:&lt;netrc-password&gt;@gitea.my.domain/Org/Repo.git

using the following format (taken from the previous output)

git clone http://{OAUTH_TOKEN_REDACTED}:x-oauth-basic@gitea.my.domain/Org/Repo.git

This returned Authentication failed
using the following format succesfully clones the repository:

git clone http://{username}:{OAUTH_TOKEN}@gitea.my.domain/Org/Repo.git

So am I to understand there is an incorrect handling of oauth in gitea?

I am thinking yes.

The drone gitea driver is based on gogs, and the gogs driver does not have any issue with netrc authentication. Since gitea is a fork of gogs I would expect the same behavior, so perhaps this is a recent regression with gitea?

I would recommend creating an issue with gitea (or opening a thread on their discourse forum) to explain that cloning with x-oauth-basic no longer works. cc @appleboy

Thank you for helping me to debug this, as far as I can see, there is no mention about the change in the gitea changelog.
I can also confirm that gitea 1.1.4 works, I have no clue why I could not use the oAuth to login to drone in the first place, but switching to it now seems to fix the netrc login issue.

I have created an issue #2480 on github if somebody finds this and wants to follow the resolution to this

Would it maybe make sense to just use deploy keys in the future?

At face value this would seem reasonable, but there is a lot of history behind our decision to use tokens for cloning a repository. The very first version of drone (v0.1.0) used deployment keys, and we moved away from this approach. Why? They are less flexible and can only be used to clone a single repository. They do not work well for projects with private dependencies that need to be cloned at runtime, which we discovered was a very common use case (via go get, npm install, git submodules and more).

Tokens solve this problem very well, and are supported by GitHub, GitLab, Gogs, Coding and Bitbucket. It is therefore very unlikely we would change our current approach for one provider, especially since Gitea descends from Gogs, which is fully supported.

Okay, thanks for the insight on that decision, that does indeed make a lot of sense.

Hi all, (apologies for this; but being legally blind I don’t have a lot of time to read through all the backlog…) Can someone summarize where we’re at with Gitea + Drone? My understanding thus far is a regression exists in some “unknown” versions of Gitea vx.y+ (makes sense since I started running into trouble after upgrading a while back; I forget what eversion I was on :confused:); but using Drone + gogits is fine.

I switched to Gitea precisely because I ran into some operational issues with Gogits (perhaps maybe it was just a bug at the time). Shall I just switch back? Is that the work-around? What’s the status of the regression in Gitea? (my understanding is Gitea is more maintained than Gogits?)

Hello, i tried using DRONE_GITEA instead of DRONE_GOGS multiple times on multiple versions of Gitea/Drone, but always hit some problems, i recommend staying on DRONE_GOGS at this time :slight_smile: Hopefully that will change soon. (Im currently running Gitea 1.1.3 and Drone 0.8.0)

It seems the issue has been pushed to the 1.3.0 milestone, so the release of gitea 1.2.0 will probably break the authentication

I haven’t had time to dig deeper into it, but it is possible that gitea 1.1.x is fully compatible with drone 0.8 (using DRONE_GOGS env)

@jbub: Can you confirm that the combination of gitea 1.1.3 and drone 0.8.0 works with private repositories as well?

Ive been running it for two days now, seems to run fine, only thing i had to do was to recreate the webhooks.

@prologic of course, I am happy to provide a summary. I think there are two separate potential root causes for this error message, and I will try to describe both.

root cause 1: there seems to be some issues with the gitea integration code in drone. This integration was provided by a gitea maintainer, and was primarily a copy of the gogs integration code with some tweaks to support the gitea status API, which gogs did not support at the time.

The gitea integration code seems to have some issues. Unfortunately this is not something I’ve personally had the time to research. Most individuals seem to configure drone to connect to their Gitea instance as if it were gogs (with GOGS_ variables) which seems to work better.

root cause two: there seems to be a recent change to Gitea causing problems when cloning a repository with an oauth token (see https://github.com/go-gitea/gitea/pull/2184). The gitea team seems to be working toward a resolution, so falling back to a slightly older version of Gitea may solve the problem for you until 1.3 is released.

Finally regarding you question about gogs vs gitea they are both good projects with talented developers, so I think either option is great. For what it’s worth, I do have a large gogs instance that pre-dates Gitea. It is configured to auto-upgrade which means I personally test every new Gogs snapshot with Drone. Note that this is not an endorsement of gogs over gitea :slight_smile:

So what I’m reading (between the lines) is migrate back to gogits :slight_smile:

James Mills / prologic

E: prologic@shortcircuit.net.au
W: prologic.shortcircuit.net.au

The mistake happened on the Gitea side, they simply changed the personal access token flow. This PR will solve the issue for Gitea >= 1.2.0: https://github.com/drone/drone/pull/2241

Gitea also backported a fix to netrc so it should support both Token only and username+Token authentication methods.

I can confirm that private repositories now work fine on drone 0.8.1 and gitea 1.2.1

@bradrydzewski I think we can close the topic, thank you again for helping us debug the issue.

Hi there, I’m having a similar problem.

I’ve installed both gitea and drone on a docker stack. Running this piece of pipeline I noticed that in the netrc file, the machine name is the service name used for gitea in my stack configuration and not the real hostname I use to reach it, so drone can’t find the right username and password for my repository.

Now the question is, where does drone take the url used to create the netrc file? How can I fix this problem? Gitea, in the hook configuration, sends every url with the correct hostname.

Ok, I found out that the machine name pushed into the netrc file is the one configured with the DRONE_GOGS_URL env variable of the drone server container. So I solved the problem changing it and configuring another alias for the gitea server configuration. :grimacing: