Exec runner & support for xcode projects

Drone Support
1

I want to continue a discussion happening in this github issue related to the exec runner creates a per-pipeline workspace for every build, where each workspace has its own HOME directory.

On MacOS, building native apps with Xcode is to all intents and purposes impossible with a redirected $HOME. In the most common scenario (using fastlane, xcodebuild etc) keychain security, provisioning profiles etc will be looking in the local user’s ~/Library directory for data.

An immediate workaround is to set the environment variable in your yaml, like this:

steps:
- name: build
  environment:
    HOME: /Users/drone

I am also interested in long-term solutions for improved xcode support. Please note that we use a custom HOME per-pipeline by design. We will need to work within these existing constraints, as opposed to changing how the runner works.

I am open to ideas and hoping to spark discussion in this thread.

I also think it would help to post simple example projects to github that demonstrate where the runner falls short. This will help the Drone maintainers better understand the problem and will help us experiment with and test solutions.

2

@bradrydzewski, I setup this repo to demonstrate the issue. https://github.com/CubicNitrous/rvm_test
The crux of the issue is being unable to use RVM which is installed on the Mac. With the current config as is anything we call that uses ruby tries to use the built-in ruby that comes with the OS.

3

Thanks for providing the example! Just to clarify, is this related to HOME or PATH or both?

From what I can tell rvm has a configuration file installed in $HOME/.rvm. Would a solution to this problem be to allow you to populate the pipeline’s temporary HOME directory with files from another location?

For testing purposes, what happens if you do something like this:

steps:
  - name: setup
    commands:
      - cp -a /Users/foo/.rvm $HOME/.rvm
      - gem install bundler
      - bundle install
      - bundle exec fastlane

or this

steps:
  - name: setup
    commands:
      - cp -a /Users/foo/.rvm $HOME/.rvm
      - source $(rvm 2.6.3 do rvm env --path)
      - gem install bundler
      - bundle install
      - bundle exec fastlane

I noticed in the rvm documentation that you need to execute the source command when running in a non-interactive terminal session. https://rvm.io/rvm/basics#post-install-configuration.

Once we have an understanding of various problems and workarounds, we can start to figure out how to build permanent solutions into the system.

5

If you put rvm init code in .zshenv is actually will get executed Instead of setting the home again,

you can actually do rvm install {version} again to install ruby to this new home.

But that is a bit wasteful.

If we keeping the same home from same project, then we only need to install these tools once.

6

this is not an option, so we need another solution.

7

Can you elaborate on that?

8

If it’s for backward compatibility, we can have it under a new setting opt in ?

9

We need each pipeline to have a semi-isolated workspace for improved concurrency and to enable various per-repository authentication files written to the temporary home directory. A shared home would result in collisions. Changing the design is not an option, so we need to work within the design constraints highlighted above. I am sure there is a creative solution to this problem.

10

I am going to start working on this next week. I am almost certain there is a solution to this problem without having to change how things work. It will just take some experimenting. First I need to figure out how to get rvm installed on my macbook. It fails on my macbook and imac with cryptic errors! :thinking:

11

any update @ashwilliams1??

12

@CubicNitrous I spent some time trying to install and configure rvm but was having issues. It looks like they only support osx 10.12 and below, and I’m running 10.14. I’m getting all kind of compiler errors when trying to rvm install :anguished:

I struggled to find any solutions via googling. If you have scripts to install and configure rvm that you run on your machines, and you can share those script, that would be a huge help!

13

\curl -sSL https://get.rvm.io | bash -s stable --ruby

Thats what we run to install RVM

14

@CubicNitrous have you tried setting the home directory and then loading rvm as shown below? In a non-interactive shell environment (like Drone) this seems to be the recommended way to load rvm since the bash profile is not loaded in non-interactive shells.

steps:
  - name: setup
    commands:
      - export HOME=/Users/SomeUser
      - source $(rvm 2.6.3 do rvm env --path)
      - gem ...
15

I solved issue with xcode by adding

ln -s /Users/$${USER}/Library $${HOME}/Library

as first command. It symlinks Library from original home to temporary one and xcode has no problems finding it’s profiles, login keychain and whatever it needs. At the same time HOME variable is kept intact. Mac runner could do something like this by default.

16

@hypafrag thanks for the suggestion. If you are comfortable building the exec runner from source, we added a global symlink setting that you can test out. Please let us know how it works for you.

DRONE_RUNNER_SYMLINKS=/Users/xxxx/Library:/home/drone/Library

(please modify the above snippet and replace xxxx with the actual username)

17

Updated to latest master, added DRONE_RUNNER_SYMLINKS ... to config, removed ln -s ... from .drone.yml, started pipeline, …, profit.
Works as expected, thanks

18

Update: these changes are included in the v1.0.0-beta.9 release. Thanks @hypafrag for suggesting this solution and for testing our implementation.

19

For me the DRONE_RUNNER_SYMLINKS method does not work. It does mount the Library folder correctly, but not keychains appear inside the runner:

+ security list-keychains
    "/Library/Keychains/System.keychain"
    "/Library/Keychains/System.keychain"

All calls to security create-keychain or adding a keychain via security list-keychains -s key1.keychain ... finish successfully, but no chain got added. I always get the same response from list-keychains.

I tried the same also without DRONE_RUNNER_SYMLINKS and get the same results.
In a terminal on MacOS, everything works as expected, also via a ssh session.