Problem with BitBucket Server - runtime error: index out of range, on event=push

undeadops · February 12, 2019, 5:17pm

Hi,

Just setting up drone, and I’m running into an issue after connecting it to our BitBucket Server (self hosted). I currently can use the same drone server, but give it settings for my github account and it all works fine. When I swap out the settings with the STASH settings for bitbucket server, I can load the git repos, but I have issues when I push into that repo. I can take the same repo, (including .drone.yml) and push to github and have it all work as expected

I get the following error:

drone_1  | DEBU[0236]                                               fields.time="2019-02-12T16:37:20Z" latency="12.7µs" method=GET remote="103.204.53.150:46240" request=/ request-id=1H5QNhBwtkT5NQ0wIZFdPA2pfz0
drone_1  | DEBU[0673] webhook parsed                                commit=<hash> event=push name=repo namespace=MY
drone_1  | DEBU[0673] trigger: received                             commit=<hash> event=push ref=refs/heads/master repo=MY/repo
drone_1  | WARN[0673] trigger: cannot find yaml                     commit=<hash> error="%!v(PANIC=runtime error: index out of range)" event=push ref=refs/heads/master repo=MY/repo
drone_1  | Panic: runtime error: index out of range
drone_1  | goroutine 457 [running]:
drone_1  | runtime/debug.Stack(0x29, 0x0, 0x0)
drone_1  | 	/usr/local/go/src/runtime/debug/stack.go:24 +0xa7
drone_1  | runtime/debug.PrintStack()
drone_1  | 	/usr/local/go/src/runtime/debug/stack.go:16 +0x22
drone_1  | github.com/drone/drone/vendor/github.com/go-chi/chi/middleware.Recoverer.func1.1(0xc4208ba800, 0x1a48c00, 0xc420610620)
drone_1  | 	/go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/middleware/recoverer.go:28 +0x1b6
drone_1  | panic(0x167c680, 0x25a95c0)
drone_1  | 	/usr/local/go/src/runtime/panic.go:502 +0x229
drone_1  | github.com/drone/drone/vendor/github.com/drone/go-scm/scm/driver/stash.(*Error).Error(0xc420657420, 0x15e74a0, 0xc4206521f0)
drone_1  | 	/go/src/github.com/drone/drone/vendor/github.com/drone/go-scm/scm/driver/stash/stash.go:132 +0x3c
drone_1  | github.com/drone/drone/internal/web.writeErrorCode(0x1a48c00, 0xc420610620, 0x1a2d6e0, 0xc420657420, 0x1f4)
drone_1  | 	/go/src/github.com/drone/drone/internal/web/writer.go:45 +0x35
drone_1  | github.com/drone/drone/internal/web.writeError(0x1a48c00, 0xc420610620, 0x1a2d6e0, 0xc420657420)
drone_1  | 	/go/src/github.com/drone/drone/internal/web/writer.go:51 +0x52
drone_1  | github.com/drone/drone/internal/web.HandleHook.func1(0x1a48c00, 0xc420610620, 0xc4208ba900)
drone_1  | 	/go/src/github.com/drone/drone/internal/web/hook.go:246 +0x6ca

bradrydzewski · February 12, 2019, 5:36pm

the panic aside, the error message is indicating that drone cannot find a .drone.yml file in your repository. Does such a file exist in ref=refs/heads/master repo=MY/repo?

bradrydzewski · February 12, 2019, 5:48pm

update, I quickly dug into the lines of code in the stacktrace and here is what I can see:

an error is being returned when trying to get your .drone.yml from the stash server. The request is returning an http code > 299
drone expects an http error code > 299 to return a valid error message in json format based on the Stash schema [1]
it appears the error response is an empty array of errors, which drone does not expect (this is where the panic occurs) [2]

So I think there are two things here:

I need to fix the panic
Stash is still returning an error and is not returning any error message

Note that fixing the panic will not solve the overall issue. The panic is the symptom of the larger root cause which is a malformed error from Stash when requesting the .drone.yml file. I recommend checking your Stash server logs to see if it can provide any more useful details as to why it is erroring.

[1] https://github.com/drone/go-scm/blob/master/scm/driver/stash/testdata/error.json
[1] https://github.com/drone/go-scm/blob/master/scm/driver/stash/stash.go#L132

undeadops · February 12, 2019, 6:44pm

yes, it does. Same repo pushed to github works fine.

could it be because the bitbucket server is at a subURL? ie: hostname/bitbucket ?

bradrydzewski · February 12, 2019, 7:42pm

could it be because the bitbucket server is at a subURL? ie: hostname/bitbucket ?

I doubt it, only because if this were the case it would have impacted other API operations such as your ability to login, get your repository list, create the webhook, etc. Those operations have to succeed to get to the point that Drone receives a webhook.

I recommend checking the Bitbucket Server logs as a next step, since it could surface some useful information regarding why it is returning an error when using the API to get the .drone.yml.

bradrydzewski · February 12, 2019, 10:26pm

I will also try to add some extra logging to the next release candidate so that we can capture the payload that is being returned by Bitbucket Server. But in the meantime, let me know what you find in the server logs.

undeadops · February 12, 2019, 10:44pm

Will do, Still trying to get access to the server

undeadops · February 13, 2019, 5:17pm

Server Logs look normal far as I can tell on the bitbucket server side. Webhook is reporting a 500 Server error on posting to drone.

I see in the access logs for bitbucket numerous successful (HTTP 200) accesses by browsers to the .drone.yml file in the repo, However, I don’t see anything for a go lang http client? I would think that means it’s not getting around to requesting the .drone.yml file?

undeadops · February 13, 2019, 8:57pm

can share a sanitized json body its sending if that would be helpful?

undeadops · February 15, 2019, 5:52pm

I think I found why its failing, there is an apache server sitting in front of bitbucket, which is how it gets to the sub url /bitbucket.

From the apache log, after a git push into the repo, I get two log entries

192.168.0.245 - - [15/Feb/2019:20:46:45 +0300] "GET /bitbucket/rest/api/1.0/projects/MY/repos/base-images/commits/<githash> HTTP/1.1" 200 "-" "Go-http-client/1.1"
192.168.0.245 - - [15/Feb/2019:20:46:45 +0300] "GET /rest/api/1.0/projects/MY/repos/base-images/raw/.drone.yml?at=<githash> HTTP/1.1" 404 "-" "Go-http-client/1.1"

Second line, has droped the prefix /bitbucket

undeadops · February 15, 2019, 6:02pm

Is it possible its just the leading / ?

github.com

drone/go-scm/blob/master/scm/driver/stash/content.go#L21




	"github.com/drone/go-scm/scm"
)


type contentService struct {
	client *wrapper
}


func (s *contentService) Find(ctx context.Context, repo, path, ref string) (*scm.Content, *scm.Response, error) {
	namespace, name := scm.Split(repo)
	endpoint := fmt.Sprintf("/rest/api/1.0/projects/%s/repos/%s/raw/%s?at=%s", namespace, name, path, ref)
	buf := new(bytes.Buffer)
	res, err := s.client.do(ctx, "GET", endpoint, nil, buf)
	return &scm.Content{
		Path: path,
		Data: buf.Bytes(),
	}, res, err
}


func (s *contentService) Create(ctx context.Context, repo, path string, params *scm.ContentParams) (*scm.Response, error) {
	return nil, scm.ErrNotSupported

bradrydzewski · February 15, 2019, 6:05pm

Great find, that certainly could be the issue. I just patched the code. The updated package will be included in our next release candidate (Monday/Tuesday).

undeadops · February 15, 2019, 6:11pm

awesome, thanks much

bradrydzewski · February 15, 2019, 6:12pm

thank you for helping debug this and for jumping into the code. I wish everyone did this

Topic		Replies	Views
Problem with BitBucket Server (v4.1.3) - runtime error: index out of range, on activate repository Drone Support	2	431	April 27, 2019
Panic: runtime error: index out of range [0] with length 0 with drone 1.9.1 and Bitbucket server v6.10.5 Drone Support	2	390	December 29, 2020
Drone 0.8.1 - Can't pull from a Bitbucket private repository Drone Support	6	2017	September 27, 2018
Drone + Kubernetes + Bitbucket Stash panic: runtime error Drone Support	1	669	March 9, 2018
Drone 0.8 broken due to bitbucket V1 API being decommissioned Drone Support	11	1650	July 7, 2019

Problem with BitBucket Server - runtime error: index out of range, on event=push

Related topics