We have been seeing intermittent failures in drone v1 kubernetes pipelines using the plugins/ecr plugin image. One or more of the parallel steps will fail with a message related to the docker socket not being available. This appears to be independent of the unix socket changes associated with the upgrade to docker > 18.09.01 as most build pass with the newer versions.
I am able to reproduce failures some what regularly, maybe 20% failure rate, with a simple test pipeline using the docker-ecr plugin.
---
kind: pipeline
type: kubernetes
name: default
trigger:
branch:
- master
steps:
- name: publish
image: plugins/ecr
depends_on: [clone]
environment:
PLUGIN_DEBUG: true
settings:
create_repository: true
registry: nopenopenope.dkr.ecr.us-east-1.amazonaws.com
repo: nopenopenope.dkr.ecr.us-east-1.amazonaws.com/reponame/${DRONE_REPO_NAME}
tags:
- git-${DRONE_COMMIT_SHA:0:7}
- latest
access_key:
from_secret: ecr_access_key
secret_key:
from_secret: ecr_secret_key
when:
event:
- push
- name: publish-also
image: plugins/ecr
depends_on: [clone]
environment:
PLUGIN_DEBUG: true
settings:
create_repository: true
registry: nopenopenope.dkr.ecr.us-east-1.amazonaws.com
repo: nopenopenope.dkr.ecr.us-east-1.amazonaws.com/reponame/${DRONE_REPO_NAME}
tags:
- also
access_key:
from_secret: ecr_access_key
secret_key:
from_secret: ecr_secret_key
dockerfile: build/1.Dockerfile
when:
event:
- push
- name: publish-also-also
image: plugins/ecr
depends_on: [clone]
environment:
PLUGIN_DEBUG: true
settings:
create_repository: true
registry: nopenopenope.dkr.ecr.us-east-1.amazonaws.com
repo: nopenopenope.dkr.ecr.us-east-1.amazonaws.com/reponame/${DRONE_REPO_NAME}
tags:
- alsoalso
access_key:
from_secret: ecr_access_key
secret_key:
from_secret: ecr_secret_key
dockerfile: build/2.Dockerfile
when:
event:
- push
- name: publish-also-too
image: plugins/ecr
depends_on: [clone]
environment:
PLUGIN_DEBUG: true
settings:
create_repository: true
registry: nopenopenope.dkr.ecr.us-east-1.amazonaws.com
repo: nopenopenope.dkr.ecr.us-east-1.amazonaws.com/reponame/${DRONE_REPO_NAME}
tags:
- alsotoo
access_key:
from_secret: ecr_access_key
secret_key:
from_secret: ecr_secret_key
dockerfile: build/2.Dockerfile
when:
event:
- push
The Dockerfiles are all the same.
FROM nginx:1.17.0-alpine
COPY nginx.conf /etc/nginx/nginx.conf
If parallel builds start docker within the window of a few milliseconds of each other, one of the builds fails. Here are two debug logs from parallel steps in the pipeline to show the timing.
Failing Step:
/usr/local/bin/dockerd --data-root /var/lib/docker --host=unix:///var/run/docker.sock
time="2020-07-07T13:07:58.250550296Z" level=info msg="Starting up"
time="2020-07-07T13:07:58.255901403Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
time="2020-07-07T13:07:58.257001082Z" level=info msg="libcontainerd: started new containerd process" pid=37
time="2020-07-07T13:07:58.258703032Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2020-07-07T13:07:58.258803189Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2020-07-07T13:07:58.258913501Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
time="2020-07-07T13:07:58.259026385Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-07-07T13:07:58.549097678Z" level=info msg="starting containerd" revision=7ad184331fa3e55e52b890ea95e65ba581ae3429 version=v1.2.13
time="2020-07-07T13:07:58.553073294Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
time="2020-07-07T13:07:58.553374469Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.554121118Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
time="2020-07-07T13:07:58.554245063Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.577635384Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
time="2020-07-07T13:07:58.578193805Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.583499817Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.586058005Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.587880037Z" level=info msg="skip loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.587896797Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
time="2020-07-07T13:07:58.588123232Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin"
time="2020-07-07T13:07:58.588137349Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
time="2020-07-07T13:07:58.588144027Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
time="2020-07-07T13:07:58.602911290Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1
time="2020-07-07T13:07:58.603056399Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1
time="2020-07-07T13:07:58.603242280Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.603372697Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.603457187Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.603556675Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.603674101Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.603787246Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.603924223Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.604005790Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1
time="2020-07-07T13:07:58.604728618Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2
time="2020-07-07T13:07:58.604942687Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1
time="2020-07-07T13:07:58.610683322Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.610915473Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1
time="2020-07-07T13:07:58.611089590Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.611266731Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.611354161Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.611571985Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.611655492Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.611813260Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.611909433Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.612034194Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.612132117Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
time="2020-07-07T13:07:58.626536188Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.626560571Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.626573813Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.626586392Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.632874776Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock"
time="2020-07-07T13:07:58.633043352Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock"
time="2020-07-07T13:07:58.633118155Z" level=info msg="containerd successfully booted in 0.084906s"
time="2020-07-07T13:07:58.686818187Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2020-07-07T13:07:58.687004539Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2020-07-07T13:07:58.687206815Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
time="2020-07-07T13:07:58.687334414Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-07-07T13:07:58.688473096Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2020-07-07T13:07:58.688668102Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2020-07-07T13:07:58.688775690Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
time="2020-07-07T13:07:58.688909310Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-07-07T13:07:58.796691666Z" level=warning msg="Your kernel does not support swap memory limit"
time="2020-07-07T13:07:58.798543904Z" level=warning msg="Your kernel does not support cgroup rt period"
time="2020-07-07T13:07:58.798690693Z" level=warning msg="Your kernel does not support cgroup rt runtime"
time="2020-07-07T13:07:58.799959675Z" level=info msg="Loading containers: start."
time="2020-07-07T13:07:58.857734717Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge 135168 1 br_netfilter\nstp 16384 1 bridge\nllc 16384 2 bridge,stp\nip: can't find device 'br_netfilter'\nbr_netfilter 24576 0 \nbridge 135168 1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
time="2020-07-07T13:07:58.990665129Z" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
time="2020-07-07T13:07:58.992580643Z" level=info msg="stopping healthcheck following graceful shutdown" module=libcontainerd
time="2020-07-07T13:07:58.997199034Z" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby
failed to start daemon: Error initializing network controller: error obtaining controller instance: unable to add return rule in DOCKER-ISOLATION-STAGE-2 chain: (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-2 -j RETURN: iptables: No chain/target/match by that name.
Passing Step:
/usr/local/bin/dockerd --data-root /var/lib/docker --host=unix:///var/run/docker.sock
time="2020-07-07T13:07:58.353263684Z" level=info msg="Starting up"
time="2020-07-07T13:07:58.354612555Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
time="2020-07-07T13:07:58.355514546Z" level=info msg="libcontainerd: started new containerd process" pid=42
time="2020-07-07T13:07:58.355726521Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2020-07-07T13:07:58.355811132Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2020-07-07T13:07:58.355899383Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
time="2020-07-07T13:07:58.355984921Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-07-07T13:07:58.550120381Z" level=info msg="starting containerd" revision=7ad184331fa3e55e52b890ea95e65ba581ae3429 version=v1.2.13
time="2020-07-07T13:07:58.551404426Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
time="2020-07-07T13:07:58.551598618Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.551931716Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
time="2020-07-07T13:07:58.552098743Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.577229071Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
time="2020-07-07T13:07:58.577733405Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.578420640Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.584438377Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.587650510Z" level=info msg="skip loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
time="2020-07-07T13:07:58.587784159Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
time="2020-07-07T13:07:58.588048734Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
time="2020-07-07T13:07:58.588172514Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
time="2020-07-07T13:07:58.588269802Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin"
time="2020-07-07T13:07:58.606376119Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1
time="2020-07-07T13:07:58.607853356Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1
time="2020-07-07T13:07:58.607971943Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.608016076Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.608044903Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.608086548Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.608121760Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.608148408Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.608189704Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.608226217Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1
time="2020-07-07T13:07:58.608405448Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2
time="2020-07-07T13:07:58.609818030Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1
time="2020-07-07T13:07:58.613949324Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1
time="2020-07-07T13:07:58.614071125Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1
time="2020-07-07T13:07:58.614205924Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.614272828Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.614343478Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.614385656Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.614414233Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.614454141Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.614488571Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.614513823Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.614537923Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
time="2020-07-07T13:07:58.615135080Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.615192332Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.615223506Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.615267810Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
time="2020-07-07T13:07:58.630747481Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock"
time="2020-07-07T13:07:58.630810403Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock"
time="2020-07-07T13:07:58.630822033Z" level=info msg="containerd successfully booted in 0.081383s"
time="2020-07-07T13:07:58.691748120Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2020-07-07T13:07:58.691765238Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2020-07-07T13:07:58.691816405Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
time="2020-07-07T13:07:58.691834780Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-07-07T13:07:58.695185317Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2020-07-07T13:07:58.695205072Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2020-07-07T13:07:58.695220177Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
time="2020-07-07T13:07:58.695232903Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-07-07T13:07:58.795196261Z" level=warning msg="Your kernel does not support swap memory limit"
time="2020-07-07T13:07:58.796436523Z" level=warning msg="Your kernel does not support cgroup rt period"
time="2020-07-07T13:07:58.796801070Z" level=warning msg="Your kernel does not support cgroup rt runtime"
time="2020-07-07T13:07:58.799812524Z" level=info msg="Loading containers: start."
time="2020-07-07T13:07:58.851630835Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge 135168 1 br_netfilter\nstp 16384 1 bridge\nllc 16384 2 bridge,stp\nip: can't find device 'br_netfilter'\nbr_netfilter 24576 0 \nbridge 135168 1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
time="2020-07-07T13:07:59.019406708Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
time="2020-07-07T13:07:59.063607687Z" level=info msg="Loading containers: done."
time="2020-07-07T13:07:59.182195740Z" level=info msg="Docker daemon" commit=afacb8b7f0 graphdriver(s)=overlay2 version=19.03.8
time="2020-07-07T13:07:59.182948077Z" level=info msg="Daemon has completed initialization"
time="2020-07-07T13:07:59.286691586Z" level=info msg="API listen on /var/run/docker.sock"
```
Any thoughts on if this is a race starting the docker dind containers within the same pod or ways to avoid this failure? This makes using the pipeline DAG features a bit more difficult as dependencies would be the only other way to stagger potentially long running build/publish steps.