CKI updates and news
It’s been a while since the initial introduction of how and what we do and things have since changed. Let’s cover the high-level changes first, and then get into full technical details of our current setup in another post(s)!
First, let’s get the most important change out of the way: we are slowly migrating to GitLab! There’s still some code living in the old GitHub location (update 2020-06-03: not anymore!) which will stay there for a while. Welcome to the chaos of multiple repositories! More details about why we decided for this change will be provided in one the next posts.
And now, let’s get into some smaller, but more important changes. We modified how we test each patch series. You may remember from the intro post that we used to apply patches to an older git commit that didn’t fail testing. However, because the patches that caused the failures are not likely to be reverted, new patches we want to test end up depending on them and thus fail to apply to older revision. Nowadays, we apply the patches right on the top commit which allows us to test everything properly.
Because of the above change, we also had to change the way we track failures and bugs. We didn’t have to care about them too much before but now we need to mask them. While the implementation for a proper solution is largely still a work in progress, we make sure to not run the tests that are failing because of known issues to avoid confusion. We also track the known issues and regularly check whether they are still a problem. Thanks to this change, we actually managed to find a ton of test problems (and got most of them fixed upstream)!
That said, we extended the number of tests we run! We currently have more than 20 open source test suites (not every test is executed on each run) and are still onboarding new ones. Our end goal is to cover most of the kernel code and also have some integration tests to make sure user space keeps working with kernel changes.
We are still testing the stable kernels and added testing of the stable queue patches! We also implemented the ability to test Koji and COPR builds which allowed us to add Fedora 30 kernels into the list of upstream kernels we work with.
So, what’s next?
More upstream collaboration! We plan to open source most (if not all) of our code, documentation and processes. This would help with both project transparency and for other people to get involved if they’d like. There are multiple CI systems for the upstream kernel out there and it would be best if we could all work together to make kernel better and solve our common problems.
The first larger upstream involvement we’d like to work on would be a common place for any CI system to drop their results to. Aggregation of results in a common place would make it much easier for kernel maintainers to check what’s going on, what was actually tested and whom to contact if they have any questions.
This setup would also make it easy for CI systems to start working with upstream kernels – they would only need to add some API calls into their code. After this initial change, they can choose to modify their code for any closer involvement if they choose to, at any pace they need to. The bar for the entry needs to be low for people to get interested. Different companies and teams have different setups and it may not be feasible for them to fulfill extra arbitrary requirements in a short time frame, so they opt to not get involved at all, and that’s a shame and shouldn’t happen.
If you’d like to get involved in the discussion please let us know! We recently joined an upstream mailing list for automated kernel testing where we’d likely start the conversation.