1.0.0-rc.1 DRONE_REPOSITORY_FILTER not respected for bitbucket

Drone Support
1

I tested out 1.0.0-rc.1 over the weekend. It looks great and it is mostly going well.

I’m having trouble with repo import against bitbucket cloud. It appears that DRONE_REPOSITORY_FILTER does not function as expected in bitbucket. I can see and activate repositories owned by other projects. The server logs shows the correct filter during startup like so:

server_1_58c7ae82e950 | repository:
server_1_58c7ae82e950 |   filter:
server_1_58c7ae82e950 |   - [correct_project]

Also only (exactly) 100 repositories are synced from bitbucket. Some of the repositories I’d like to activate are missing. This already has a topic open at 1.0.0-rc.1: How increaselimit of repos?

I tried to dig into code to see if I could point at fixes or create a PR, but figured out that dev on 1.0.0 is in a private repo. Any pointers would be appreciated!

2

does not function as expected in bitbucket

You are correct, the filter was not hooked up in the code. I pushed an update to fix this, and pushed an updated image to DockerHub. Please note that the FILTER does not prevent you from activating a repository, it limits which repositories are synchronized and stored in the Drone database. This value needs to be configured before your first login (since you have already logged in, you will need start with a fresh database, or manually delete rows from the database)

Also only (exactly) 100 repositories are synced from bitbucket.

I do not have enough bitbucket repositories to test this, however, the code for listing all user repositories and pagination can be found here: https://github.com/drone/go-scm/blob/master/scm/driver/bitbucket/repo.go#L87

Note that this is not related to the existing topic you mentioned previously.

3

Your fix resolved the DRONE_REPOSITORY_FILTER issue. I’ll dig into the 100 repo limit a bit later. Thanks!

4

Well, only repos from the appropriate project are being synced. Some repos are still missing. I’m assuming the filter happens after the fetch and is still affected by the 100 repo limit problem.

5

The filter happens after all repositories are fetched, so filtering is unrelated to your Bitbucket issue.

The logic for fetching the user repository list is provider agnostic, meaning it is the same code for GitHub, Bitbucket, Gitlab, Gogs, etc. I am not having any issues with GitHub which would imply this issue is with the library we are using, and is related to either the Bitbucket API or Bitbucket pagination logic, which can all be found here https://github.com/drone/go-scm/blob/master/scm/driver/bitbucket/repo.go#L87

6

Is ListOption.Size being passed into the client from the upstream code? Is it set to 100 or more? It looks like the bitbucket api may have weird and silent pagination problems if pagelen is set to anything over 100 and sometimes if it is set to 100. I don’t have a great way to test any of that.

7

yes, we are using pagelen 100

8

You are using 100 in .8.x as well and this bug doesn’t happen there, so that is likely not the problem.

I’m working around the issue by setting up another bitbucket account that only has access to one organization and logging in initially with that account so I can keep testing. But if you figure anything out and need me to test a fix, let me know.

9

For other who are waiting on a fix for the “only syncing 100 repos from bitbucket” problem, there is a github issue where a potential fix is being discussed:

https://github.com/drone/drone/issues/2533