How to connect to a docker network, or how can a docker container connect to a drone service

Drone Support
1

Hi,

I want to set up a drone pipeline that does those things:

  1. (docker-compose build - unrelated)
  2. Start a MS SQL database.
  3. Run a script to initialize the table schema in the MS SQL.
  4. Start a container my-service which connects to the MS SQL.
  5. (Run component tests on my-service - unrelated)

However, if steps 2 starts the MS SQL in a container, the script in step 3 cannot connect to it. (On my local machine I can run the script with localhost:1433 but localhost doesn’t work on drone.)
If I start a MS SQL as a drone service instead, the script in step 3 works but my service in step 4 cannot connect to it.
How can I make one of the 2 approaches work?

Initially both the MS SQL container and my-service container are in an external network my-service, which is created in drone before starting the first container. But I tried removing all network specification and it still didn’t work.

mssql.yml:

version: '3'

services:

  mssql:
    image: mcr.microsoft.com/mssql/server:2017-GDR-ubuntu
    container_name: my-service-mssql
    environment:
      - SA_PASSWORD=test_123
      - ACCEPT_EULA=Y
      - MSSQL_MEMORY_LIMIT_MB=4096
#    networks:
#      - my-service
    ports:
      - "1433:1433"

#networks:
#  my-service:
#    external:
#      name: my-service

Part of .drone.yml when MS SQL is in a container:

  - name: start mssql
    image: 'docker/compose:1.24.0'
    commands:
#      - docker network create my-service || true
      - docker-compose -f mssql.yml up --detach
    volumes:
      - name: dockersock
        path: /var/run
......
  - name: deploy schema
    image: 'mcr.microsoft.com/dotnet/core/sdk:3.1'  # the script depends on dotnet
    commands:
      - ./local_deploy_schema_drone.sh  # the script connects to my-service-mssql:1433
    volumes:
      - name: dockersock
        path: /var/run
      - name: nuget-cache
        path: /root/.nuget
    depends_on:
      - my-service build
      - start mssql
  - name: my-service run component tests
    commands:
      - docker-compose up -d my-service  # also connects to my-service-mssql:1433
      - sleep 5
      - docker-compose run my-service-component-tests
    volumes:
      - name: dockersock
        path: /var/run
    depends_on:
      - my-service build
      - my-service build component tests
      - deploy schema

Part of .drone.yml when MS SQL is started as a drone service:

  - name: deploy schema
    image: 'mcr.microsoft.com/dotnet/core/sdk:3.1'  # the script depends on dotnet
    commands:
      - ./local_deploy_schema_drone.sh  # the script connects to my-service-mssql:1433
    volumes:
      - name: dockersock
        path: /var/run
      - name: nuget-cache
        path: /root/.nuget
    depends_on:
      - my-service build
  - name: my-service run component tests
    commands:
#      - docker network create my-service || true
      - docker-compose up -d my-service  # also connects to my-service-mssql:1433
      - sleep 5
      - docker-compose run my-service-component-tests
    volumes:
      - name: dockersock
        path: /var/run
    depends_on:
      - my-service build
      - my-service build component tests
      - deploy schema
......
  services
  - name: my-service-mssql
    image: mcr.microsoft.com/mssql/server:2017-GDR-ubuntu
    environment:
      ACCEPT_EULA: Y
      SA_PASSWORD: test_123
      MSSQL_MEMORY_LIMIT_MB: 4096

I know I can create another container that runs the script in step 3, but I’d like to avoid the additional setup of Dockerfile and docker-compose.yml if possible. I expect it shouldn’t be hard to connect between the host network and docker network in drone.

Thanks.

2

drone and docker-compose are both orchestration systems, and they are not really meant to be used together, without doing some really hacky stuff. Instead, the recommended approach is to define you service containers [1] directly in the drone.yml using the services section. The behavior is very similar to docker-compose.

alternatively, if you really want to use docker-compose, you might consider installing the exec runner and defining exec pipelines [2]. Exec pipelines are not containerized like docker pipelines, therefore, you would not run into nested docker issues.

[1] https://docs.drone.io/pipeline/docker/syntax/services/
[2] https://docs.drone.io/pipeline/exec/overview/

3

I tried starting mssql as a drone service instead (details in the original post), but my-service cannot connect to it from a container. Did you suggest starting my-service as a drone service as well? That sounds scary.

I just tried making the pipeline exec and update the script to connect using localhost:1433, still didn’t work.

4

Or is it just difficult to have something started by docker-compose connect with scripts/services run by drone directly?