I've made some more progress with Bodhi's Continuous Integration test suite runner, bodhi-ci, over the past week or so. bodhi-ci is intended to be used by both Jenkies and human developers, so that humans have a way to reproduce test failures locally. As of my most recent blog post about this topic, the test suite was more efficient when run by the Jenkiesfile than when run locally, because the Jenkiesfile was an asynchronous and bodhi-ci was not.

My career has long kept me on Python 2, and so I've never seriously looked at Python 3's asyncio until now. Though Bodhi itself still runs on Python 2 on the server (the upcoming Bodhi 3.11 release is planned to fully support Python 2 and 3, and the client already supports Python 2 and 3), the CI script only needs to run in Bodhi's development environment and CentOS CI, both of which have access to Python 3. I noticed that EPEL was switching to Python 3.6 recently which would also allow me to use the much nicer async/await API, and thanks to Miro Hrončok, there is a build of click available for it too.

After a few days of reading and tinkering, I was able to convert bodhi-ci to be asynchronous like Bodhi's Jenkiesfile, and it's wonderful. This enabled several new features that will be helpful in development and essential for launching an integration suite:

• Bodhi's --failfast flag can now cancel all running tasks instantly, not just the unit tests.
• There is a new --concurrency/-j flag that lets you choose how many tasks to run in parallel, defaulting to the number of CPU cores detected.
• Tasks can now depend on other tasks. bodhi-ci used to run all the builds and then launch all the tests once all the builds were done. Now, the f27 tests can launch as soon as the f27 build is complete, regardless of whether the pip build is done (which typically takes a lot longer). This last one is essential for starting a CI suite, since we will need several set up tasks to complete before the CI suite can start.

Playing with asyncio has been a lot of fun, and I look forward to using bodhi-ci as I continue to improve Bodhi's test suite.