Drone on k8s: services aren't accessable at there hostname

Drone Support
1

I’m trying to setup a pipeline on drone installed on k8s. After having problems I created a test project that followed https://docs.drone.io/user-guide/pipeline/services/#incorrect-hostname

  1. I can’t find the repo that has the docs found on docs.drone.io.

  2. those docs show a pipeline w/o a name, and if I do that I get “default: cannot find named pipeline” in the webui

  3. if I give the pipeline a name I get this output in the ping step:

    + sleep 30
+ redis-cli -h cache ping
Could not connect to Redis at cache:6379: Name or service not known

ps - on the chance that the services are running in the same pod as the other steps, I tried localhost, which didn’t work either. Were are the services running?

pps - have you thought about using the HEALTHCHECK in docker images to test if a service is ready? that or a healthcheck command that you could add to the service that would exec in the service container to check if it’s ready to go?

2

Unlike Docker, Kubernetes requires specifying the port in the yaml, and the entire corpus of documentation has not been updated to reflect this. First, make sure you include the port (example below). Second, for whatever reason, we are seeing variation in behavior across Kubernetes offerings (Digital Ocean vs EKS vs GKE vs Minikube). I am unable to reproduce any issues with services on Digital Ocean or Minikube.

It is possible you are experiencing cluster-specific issues. We are unable to test every different provider for conformance, so we generally ask community members to get their hands dirty when triaging these sort of issues, and report their findings back to us. We do provide a guide to assist with debugging. See http://discuss.harness.io/t/contributing-to-drone-for-kubernetes/3159

It is also possible that it takes Kubernetes more than 5 seconds to schedule and start the container, and for Redis to begin accepting connections. All of our existing documentation is optimized for Docker and may need to be tweaked for Kubernetes. The Kubernetes scheduler is significantly slower and, because of scheduling differences, you may also need to account for image download times. So try adding more time to the sleep operation as well.

kind: pipeline
name: default

steps:
- name: test
  image: redis
  commands:
  - sleep 5
  - redis-cli -h $REDIS_SERVICE_HOST ping
  - redis-cli -h redis ping

services:
- name: redis
  image: redis
  ports:
  - 6379

Note that the above example uses REDIS_SERVICE_HOST, however, in my testing I am able to use the hostname as expected.

on the chance that the services are running in the same pod as the other steps, I tried localhost, which didn’t work either. Were are the services running

Each service is run in its own pod, with a Service resource also created.

have you thought about using the HEALTHCHECK in docker images to test if a service is ready

There is an open issue for exploring the use of healthcheck, see GitHub · Where software is built. This feature was considered more of a nice-to-have for our Docker implementation, since our Docker scheduler was very predictable and Drone had a lot of control over the scheduling process. Drone has limited control over the Kubernetes schedule, which also appears to be a bit less predictable, and may therefore benefit from such a feature.

3

Thanks for the help @bradrydzewski. Once I added a ports key to the service I was able to use both redis-cli -h redis ping and redis-cli -h $REDIS_SERVICE_HOST ping. I am doing this in my homelab with k8s running on microk8s

4

awesome, thanks for the update! I’m working to get the docs improved.

5

Hey guys,

This topic solved my problem too, but I’m worried about that it is still not presented anywhere else in the documentation, is it?
I was investigating hours after I found this topic :slight_smile:

6

@nandorholozsnyak did you install Drone using the chart in Helm stable? If so, this chart is unofficial and was deprecated. You should instead use our official chart. See Helm Stable Chart Deprecated

7

Hey, yes, unfortunately, I was experimenting with Drone, and so for I really like it! :smiley:
I will try to upgrade to the one you linked. Thank you. :wink:
BTW, the service port stuff I mentioned, that was for the documentation, but if I use this solution than the port could be skipped from the service definition to access it on the given name?

8

port is no longer required and was removed from the documentation.

also note:

If you installed Drone using Helm Stable with server.kubernetes.enabled enabled (which was the default) you have been using an experimental Kubernetes integration that was deprecated almost a year ago, is no longer maintained, and no longer matches our documentation.

you will definitely want to use our helm chart to ensure you are using the latest stable version of drone, otherwise you may encounter issues that we won’t fix or have already fixed in the latest stable version.