- It didn't have the GitHub pull request plugin, so it was only capable of reporting test failures after a merge has happened instead of before. (Note: it is able to test Pagure pull requests, but Bodhi isn't migrated to Pagure yet.)
- It reused the system that ran the tests, rather than provisioning a fresh system for each test build. Due to this, previous tests could affect the environment for future tests.
- The tests didn't have root access on the test runner, which meant we had to request admin assistance to install new dependencies.
- The Jenkins job had to be edited in Jenkins, rather than in Bodhi's source control.
Now that Bodhi is on CentOS CI, the issues described above are all addressed:
- Bodhi's pull requests are now tested on GitHub.
- Each pull request gets a fresh VM to run the tests on, so they all have a common starting point.
- The test script can use sudo.
- CentOS CI uses Jenkins Job Builder, which allows Bodhi to store its Jenkins jobs in its own repo.
Many thanks to Brian Stinson and the CentOS project for setting this up and maintaining the system for Bodhi!