I've made some more progress with Bodhi's Continuous Integration test suite runner,
over the past week or so.
bodhi-ci is intended to be used by both Jenkies and human
developers, so that humans have a way to reproduce test failures locally. As of my most recent blog
post about this topic, the test suite was more efficient when run by the
than when run locally, because the
Jenkiesfile was an asynchronous and
bodhi-ci was not.
My career has long kept me on Python 2, and so I've never seriously looked at Python 3's
asyncio until now. Though Bodhi itself
still runs on Python 2 on the server (the upcoming Bodhi 3.11 release is planned to fully support
Python 2 and 3, and the client already supports Python 2 and 3), the CI script only needs to run in
Bodhi's development environment and CentOS CI, both of which have access to Python 3. I noticed that
EPEL was switching to Python 3.6 recently which would also allow me to use the much nicer
async/await API, and thanks to Miro Hrončok, there is a
build of click available for
After a few days of reading and tinkering, I was able to convert
bodhi-ci to be asynchronous
Jenkiesfile, and it's wonderful. This enabled several new features that will be
helpful in development and essential for launching an integration suite:
--failfastflag can now cancel all running tasks instantly, not just the unit tests.
- There is a new
--concurrency/-jflag that lets you choose how many tasks to run in parallel, defaulting to the number of CPU cores detected.
- Tasks can now depend on other tasks.
bodhi-ciused to run all the builds and then launch all the tests once all the builds were done. Now, the f27 tests can launch as soon as the f27 build is complete, regardless of whether the pip build is done (which typically takes a lot longer). This last one is essential for starting a CI suite, since we will need several set up tasks to complete before the CI suite can start.
asyncio has been a lot of fun, and I look forward to using
bodhi-ci as I
continue to improve Bodhi's test suite.