Cron jobs seems not working with DRONE_GITEA_SKIP_VERIFY variable and SelfSigned Certificates

toni-moreno · November 9, 2019, 6:45am

Hi everybody I’ve seen in my lab platform working with self signed certificates that cron jobs are not working as expected because I’ve configured the DRONE_GITEA_SKIP_VERIFY and working ok with direct login on the UI, but I’ve seen errors with cron jobs as you can see in the following log lines.

{"branch":"master","cron":3,"error":"Post https://git.myplatform.org/login/oauth/access_token: x509: certificate is valid for 413075044928d7b2d8864b1ada61ba7c.e5c739ab2bc51b1f4a195056046c6168.traefik.default, not git.myplatform.org","level":"warning","msg":"cron: cannot find commit","repo":"myclient_probes/probe_firefox-google","time":"2019-11-08T08:06:15Z"}
{"branch":"master","cron":3,"error":"Post https://git.myplatform.org/login/oauth/access_token: x509: certificate is valid for 413075044928d7b2d8864b1ada61ba7c.e5c739ab2bc51b1f4a195056046c6168.traefik.default, not git.myplatform.org","level":"warning","msg":"cron: cannot find commit","repo":"myclient_probes/probe_firefox-google","time":"2019-11-08T08:09:15Z"}

Not sure if this can be a configuration error , or perhaps a bug ?

I’ve configured my drone-server and drone-runner as follows

drone-server:
    image: "${DRONE_SERVER_IMAGE:-drone/drone:latest}"
    networks:
      - net-public
    environment:
      DRONE_SERVER_HOST: drone.${WEB_DOMAIN}
      DRONE_SERVER_PROTO: "https"
      DRONE_LOGS_DEBUG: "true"
      DRONE_LOGS_TRACE: "true"
      DRONE_LOGS_PRETTY: "true"
      DRONE_RPC_SECRET: ${WEB_ADMIN_PASSWORD}
      DRONE_DATABASE_DRIVER: sqlite3
      DRONE_DATABASE_DATASOURCE: /var/lib/drone/drone.sqlite
      DRONE_TLS_AUTOCERT: "false"
      DRONE_USER_CREATE: username:${WEB_ADMIN_USERNAME},admin:true
      DRONE_GITEA_SERVER: "https://git.${WEB_DOMAIN}"
      DRONE_GITEA_CLIENT_ID:  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      DRONE_GITEA_CLIENT_SECRET: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
      DRONE_GITEA_SKIP_VERIFY: "true"
      DRONE_CRON_INTERVAL: 1m
      DRONE_CRON_DISABLED: "false"
     
    volumes:
      - drone-server-data:/var/lib/drone/
    deploy:
      placement:
        constraints:
          - node.labels.net-role.drone-server == true
      replicas: 1
      labels:
        - traefik.enable=true
        - traefik.frontend.rule=Host:drone.${WEB_DOMAIN}
        - traefik.docker.network=net-public
        - traefik.port=80
        - traefik.tags=net-public
        - traefik.redirectorservice.frontend.entryPoints=http
        - traefik.redirectorservice.frontend.redirect.entryPoint=https
        - traefik.webservice.frontend.entryPoints=https


  drone-runner:
    image: drone/drone-runner-docker:latest
    networks:
      - net-public
    environment:
      DRONE_RPC_HOST: drone-server
      DRONE_RPC_PROTO: http
      DRONE_RPC_SECRET: ${WEB_ADMIN_PASSWORD}
      DRONE_RUNNER_CAPACTIY: 8
      DRONE_DEBUG: "true"
      DRONE_TRACE: "true"
      DRONE_LOGS_TRACE: "true"
      DRONE_RUNNER_ENVIRON:  GIT_SSL_NO_VERIFY:"true"
      DRONE_RUNNER_NETWORKS: net-public
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    deploy:
      mode: global
      placement:
        constraints:
            - node.labels.net-role.drone-runner == true

How can I fix this error? There is any workaround we can follow?

And here the complete log

gist.github.com

https://gist.github.com/toni-moreno/d13244984fd8b7a2cc35528ce962c801#file-drone_server-log

drone_server.log

{"fields.time":"2019-11-07T17:29:20Z","latency":2338007,"level":"debug","method":"GET","msg":"","remote":"10.0.0.2:33392","request":"/api/stream/myclient_probes/probe_venta-asistida/33/1/3","request-id":"1TIVn7jDcvUs3zCEgQaOXZxVlJn","time":"2019-11-07T17:29:20Z"}
{"arch":"amd64","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2019-11-07T17:29:28Z","type":"docker","variant":""}
{"arch":"amd64","kernel":"","kind":"pipeline","level":"debug","msg":"manager: context canceled","os":"linux","time":"2019-11-07T17:29:28Z","type":"docker","variant":""}
{"arch":"amd64","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2019-11-07T17:29:38Z","type":"docker","variant":""}
{"arch":"amd64","kernel":"","kind":"pipeline","level":"debug","msg":"manager: request queue item","os":"linux","time":"2019-11-07T17:29:38Z","type":"docker","variant":""}
{"level":"debug","msg":"cron: begin process pending jobs","time":"2019-11-07T17:29:53Z"}
{"level":"debug","msg":"cron: finished processing jobs","time":"2019-11-07T17:29:53Z"}
{"commit":"cf89827246914f50a6a36e561c26894abec0448a","event":"push","level":"debug","msg":"webhook parsed","name":"probe_venta-asistida","namespace":"myclient_probes","time":"2019-11-07T17:30:06Z"}
{"commit":"cf89827246914f50a6a36e561c26894abec0448a","event":"push","level":"debug","msg":"trigger: received","ref":"refs/heads/master","repo":"myclient_probes/probe_venta-asistida","time":"2019-11-07T17:30:06Z"}
{"fields.time":"2019-11-07T17:30:07Z","latency":99928630,"level":"debug","method":"POST","msg":"","remote":"10.0.0.2:33392","request":"/hook?secret=BoSyk6cXgx5LSmvEYsOKBZyvgrO62A2v","request-id":"1TIVsuKlMJsTHbOOqKpaI7aSYWi","time":"2019-11-07T17:30:07Z"}

This file has been truncated. show original

toni-moreno · November 9, 2019, 9:52am

Hi again. I’m trying to fix this problem, but I’m new to drone and there is lots of things I unknown

I’ve reviewed code ( as good as I can ) and it seems that while the LoginProvider sets the value of DRONE_GITEA_SKIP_VERIFY at line 99 …

github.com

drone/drone/blob/33a8ecec916baa42b518b0d888b1cd5ecdc93544/cmd/drone-server/inject_login.go#L91-L104


func provideGiteaLogin(config config.Config) login.Middleware {
	if config.Gitea.Server == "" {
		return nil
	}
	return &gitea.Config {
		ClientID:     config.Gitea.ClientID,
		ClientSecret: config.Gitea.ClientSecret,
		Server:       config.Gitea.Server,
		Client:       defaultClient(config.Gitea.SkipVerify),
		Logger:       logrus.StandardLogger(),
		RedirectURL:  config.Server.Addr + "/login",
		Scope:        config.Gitea.Scope,
	}
}

…there is a LoginRefresher that doesn’t set this value.(could be?)

github.com

drone/drone/blob/33a8ecec916baa42b518b0d888b1cd5ecdc93544/cmd/drone-server/inject_login.go#L168-L174


		return &oauth2.Refresher{
			ClientID:     config.Gitea.ClientID,
			ClientSecret: config.Gitea.ClientSecret,
			Endpoint:     config.Gitea.Server + "/login/oauth/access_token",
			Source:       oauth2.ContextTokenSource(),
		}
	}

The error log cron: cannot find commit show that it happens when trying to refresh token , inside the FindRef (Lines 133) …

github.com

drone/drone/blob/e97b1e968bb9f729db995d1103bb593b01a2295b/trigger/cron/cron.go#L133-L143


		commit, err := s.commits.FindRef(ctx, user, repo.Slug, job.Branch)
		if err != nil {
			logger.WithFields(
				logrus.Fields{
					"error":  err,
					"repo":   repo.Slug,
					"branch": repo.Branch,
				}).Warnln("cron: cannot find commit")
			result = multierror.Append(result, err)
			continue
		}

… inside FindRef it first executes a Renew method (line 71)…

github.com

drone/drone/blob/e6ac2a53d4bc41d745fe708db30ec50dcd05a471/service/commit/commit.go#L70-L74


func (s *service) FindRef(ctx context.Context, user *core.User, repo, ref string) (*core.Commit, error) {
	err := s.renew.Renew(ctx, user, false)
	if err != nil {
		return nil, err
	}

…and after at line 57 execute the Refresh() method…

github.com

drone/drone/blob/e6ac2a53d4bc41d745fe708db30ec50dcd05a471/service/token/renew.go#L45-L65


func (r *renewer) Renew(ctx context.Context, user *core.User, force bool) error {
	if r.refresh == nil {
		return nil
	}
	t := &scm.Token{
		Token:   user.Token,
		Refresh: user.Refresh,
		Expires: time.Unix(user.Expiry, 0),
	}
	if expired(t) == false && force == false {
		return nil
	}
	err := r.refresh.Refresh(t)
	if err != nil {
		return err
	}
	user.Token = t.Token
	user.Refresh = t.Refresh
	user.Expiry = t.Expires.Unix()
	return r.users.Update(ctx, user)
}

… perhaps from the previously configured (without SKIP_VERIFY?) oauth.Refresher object .

github.com

drone/drone/blob/e6ac2a53d4bc41d745fe708db30ec50dcd05a471/service/token/renew.go#L32-L35


type renewer struct {
	refresh *oauth2.Refresher
	users   core.UserStore
}

Could be a code Bug? or perhaps something wrong in my configuration ?

Thank you everybody and sorry if I did some mistake in the analysis.

toni-moreno · November 10, 2019, 10:13am

Hi @bradrydzewski , I 'm ready to send you a PR with the fix for this problem , I’ve tested on my lab and it is working ok. Could you review it please? Should I open an Issue to your the github repo?

https://github.com/toni-moreno/drone/commit/9ae3930df9f498f51c0287969097c0dec86ade60

I’ve also pushed a fixed image here (https://hub.docker.com/repository/docker/tonimoreno/drone-server_fix_cron) if you would like to test.

Do you need something more that I can do to close this issue?

Thank you very much!

toni-moreno · November 12, 2019, 7:20am

Hi @bradrydzewski I’ve updated with my fixed image another test environment (shared with other users) and now also working fine .

I like to know if perhaps is time to open an Issue to the github project and make a PR ? ( I didn’t do before because of the message "please do not open a GitHub issue until you have discussed and verified with community support ").

thank you for your patience, and for this great project

Topic		Replies	Views
DRONE_WEBHOOK_SKIP_VERIFY does not work Drone Bugs	1	402	April 7, 2022
Nothing happens when my gitea repo has been updated Drone Support	6	1296	November 2, 2019
DRONE_GITEA_SKIP_VERIFY not taking effects? Drone Support	2	512	January 5, 2021
Drone Server docker image cannot make SSL requests Drone Support	5	626	August 5, 2020
Cannot clone private gitea repo (could not read Username for...) 0.8.0-rc.3, gitea 1.2.0-rc2 Drone Support	17	7553	October 25, 2018

Related topics