Provide pull credentials to an extension

Drone Support
1

I want to create an extension which enables me to use an include mechanism. This include would allow to overwrite fields and reference pipeline steps from another project. This helps with architecting pipelines for microservices.

To avoid a mechanism like paths-changed where you have to define a service account to be able to pull the repository and only a subset of the providers are actually supported, it would be cool to be able to receive some credentials to actually pull the repository.

Is there a possible way? There must be some credentials as drone can clone the repository during the build itself. Or is it too risky to send the extensions? For me this would be only a small problem as the drone server and all extensions are running in the same docker network where only the server is exposed.

Would be glad for any feedback.

2

It is not possible today, however, I am open to a pull request to (optionally) include user token / credentials in the extension payload. However, it is important to note that credentials are provider-specific. For example, bitbucket server uses oauth1, bitbucket cloud uses oauth2 access + refresh token, github and gitlab use access tokens only. An extension would therefore still need provider-specific logic to authenticate requests.

3

Wouldn’t it be possible that drone provides the same credentials it does when cloning during the build. I mean the oauth2/oauth1 and others result in a token which then is used to clone the repository. For now HTTP(S) clone would be sufficient as SSH keys are another thing. Couldn’t the drone server go through this process and pass the obtained credentials? So it wouldn’t need to be reimplemented in every extension.

Meaning:

  1. Webhook received
  2. Handle authentication and obtain clone credentials xy
  3. Start to run the conversion extension while passing xy
  4. Start the build with the yaml
  5. Agent pulls job and then clones the repository using xy

Or am I missing a part on how the clone-process works?

4

Drone uses tokens to clone the repository, so by passing tokens to the extension, we would be providing the extension with the same credentials that Drone uses to clone the repository. Passing tokens is desirable because they can be used to access the API and to clone the repository using git+https. This would be useful to existing extensions, such as the paths changed extension, which requires a token to make API calls.

5

This token sounds like the “credentials” I meant and need. So this token would need to be passed to the extension. Basically the complete url á la https://user:password@my.git.server/org/repo.git would be ideal or the information to build such a url. The include logic for this would not need any provider specific steps as most information can be pulled from the existing variables + a snippet like:

steps:
  - include: 'another-org/or-another-repo'
    file: 'some-other-file-with-steps.yaml'
    variables:
      PARAM_1: 'SomeCoolValue'

Is this change (giving the token to the extension in additional field) complex?

And another topic: Is this token user-scoped? Meaning: Can only those repositories be accessed the user triggering the build has access to? Otherwise there might be a security issue.

6

I am not a go developer, but it seems like Drone creates a Netrc file which then contains the credentials to access the repository. This file seems to be used by the agent to clone the repository.

7

the token is stored in the netrc file, which is why sending the token to the extension is going to be the best option; it will be useful to extension authors that want to make authenticated API calls (which is what most extensions are doing) and will also be useful to extension authors that want to git clone (to my knowledge you would be the first).

8

Using git clone imo is more robust as I could also use the API of the corresponding provider to access file contents and so on. The problem here is every extension author has to do so for every provider which is kind of tedious. Nevertheless passing the token improves the situation of both git clone and API users. But I don’t think I need to explain it to you.

Using an include-like extension can enhance the whole drone usage with shared pipelines and so on which makes things more maintainable.

The only problem I see is regarding the scope/owner of the token and to prevent exploitation of the token with a malicious pipeline.

9

We have already done the work to abstract this for all providers with https://github.com/drone/go-scm. This includes useful methods for fetching diffs and changes associated with commits and pull requests:

I do not envision any scoping issues given the token is already beings used to make various API calls and to clone the repository (via the netrc file) in full.

I am not aware of any threat vectors associated with passing the token to the extension, assuming you are following best practices such as serving extension traffic using https, not writing tokens to your logs, etc.

10

Oh! Missed that one. Sorry. Looks great. Will take a look at it.

For the security issue I meant for my use case. Yeah sort of my problem I guess but would be glad having a little discussion with you about it as you know the system by heart. I don’t know who the owner of the token is. Is it the user who pushed or who activated the repository or some kind of temporary webhook token? In my use case (see above) you would be able to include a file from another repository.

Assume you have two private repos A and B. User C is the owner of both repos and activated repo A in drone. User D only has access to repo A. Repo A has a pipeline with a snippet similar to the above. In this snippet a yaml file from repo B is referenced. User D is pushing some changes. Now the failure of the pipeline should depend on the token owner. Pusher: Fail, Activator: Success, Temporary: Fail.

The problem with the activator being the token owner is that user D could potentially access repos/files which are not accessible otherwise by referencing some other repository or file in the pipeline in repo A.

11

you may be interested in a new feature that just landed in master yesterday, which allows you to create shared pipeline templates. We do not have any documentation in place yet since the feature just landed, but I expect this to be available relatively soon, perhaps as soon as next week. Not sure it will solve for your exact use case, but it may be worth looking into.

12

Shared pipeline templates sounds like the thing I want to build. Will look forward to the documentation as the code I found regarding templates only consists of database operations. But I am not a go developer and probably looked at the wrong place. Anyways, thanks for the feature! Maybe I can build upon it.

13

Okay, just read the documentation on the templates. Looks very cool and will look into adopting such a feature although I hope on further explanation of the topic in the docs in the future. Having to deploy the templates separatly mitigates the security issue on the repository access by having a different storage location. But this also creates problems with versioned pipelines as you have to know which version of the template is currently deployed.

Just out of interest: Who is the owner of the token? (Explained above)