1.0 Migration Utility Published

Announcements & Events
1

The 1.0 database migration utility is published and ready for use. I have personally tested migrating the beta.drone.io database (approximately 3 years of data) using sqlite, mysql and postgres. You can find the usage instructions here

  • Keep a backup of your database. The migration utility does not mutate the source database, but you should always keep a backup to be safe.
  • The source and target database should be the same type. For example, do not try to migrate from sqlite3 to mysql.
  • The target database should be empty before starting the migration process. No existing Drone tables or database resources should exist.
  • Execute the migration utility on a machine with significant ram or swap. The migration utility copies data from the source to target database, and stores data in-memory during the copy. This utility is liberal with memory usage. The available ram should exceed the size of your largest database table.
  • If a migration step fails you can resume the migration at that step once the problem is resolved.
  • You can run the migration utility as many times as you want. The setup-database step should only be executed once, unless you are starting with an empty database. The table migration steps can be executed multiple times as long as the target tables are empty.

Despite the fact that I successfully tested the migration utility, there may be edge cases or scenarios that have not been accounted for. If you encounter and issue please send a pull request or try and provide us with an isolated dataset that can be used to reproduce the issue.

Please note that 1.0 has some known bugs and it may take a few minor releases (1.0.1, 1.0.2, etc) before these issues are resolved. You should consider this prior to upgrading:

  • Live updates on website do not always work (may require browser page refresh)
  • Log stream performance issues with large log files. This is due to the agent protocol change from grpc to plain http and a lack of optimizations that will come in future releases. Practically speaking this has only been an issue when using ngrok or proxy or load balancing services with traffic rate limits.
  • Multi-pipeline executions can freeze if pipelines in the dependency graph are skipped
  • Multi-pipeline executions can freeze when the trigger.status == success

You can find usage instructions in the README.

5 Likes