What tests will run for my kernel ?

Test stages across a kernel’s life cycle

The Linux kernel can be tested in various ways throughout its stages of creation. These stages include when a patch set is posted, when a kernel is integrated with a collection of patches, and when a kernel is combined with other packages to form a compose. Each of these stages requires a definition of what will be tested and its duration. This page discusses these stages in detail.

Patches

This stage handles when a patch is posted to a mailing list or sent as a merge request to a git tree. The purpose of this stage is to focus the testing on the patches itself and not the whole kernel source tree. This minimizes the wastefulness of kernel testing by only focusing on the change itself.

  • Duration: 2-4 hours. Since we expect a heavy volume, this stage’s duration of testing should be minimized as we should only run testing which relates to proposed changes.
  • Tests: Kernel Tier0 tests including LTP + Targeted Patch Testing generated from KPET.
  • Resources: 1 machine per supported architecture (unless specific HW required)

Pre-built Kernel Binaries

This stage handles when the kernel maintainer has integrated a collection of patch sets in preparation of adding a kernel build to be added to a compose. The patch sets could have negatively impacted each other and need a broader set of tests to verify no regressions are found.

  • Duration: 4-6 hours. The volume is expected to be lower, multiple times a week, the duration can be higher.
  • Tests: Kernel Tier0 tests which span all subsystems. Since kernel maintainers expect results within a reasonable time frame, testing which requires special hardware is excluded if it not readily available to avoid long queue times.
  • Resources: 1 machine per supported architecture

Kernel Subsystem Tree

This stage handles when a kernel maintainer pushes to an upstream kernel tree. Depending on which upstream kernel tree we are testing, we will only run a select subset of tests which relate to the kernel tree. For example, net-next tree will only run networking related tests, while mainline will run all kernel Tier0 tests by default. It is up to the maintainer tell us which tests are applicable to their tree.

  • Duration: 4-6 hours. The volume can widely vary depending on which tree we’re testing.
  • Tests: See above, this depends on which upstream tree we’re testing and maintainer’s feedback for related tests.
  • Resources: The maintainer decides which architectures are sufficient

Compose (Future)

This stage handles when a new kernel build will integrate with a nightly RHEL compose. The testing here should ensure other packages did not break the kernel’s ability to install or function correctly.

  • Duration: 2-4 hours. The volume is expected to be low, once per night.
  • Tests: Broad package testing. It is not expected to have other packages cause internal kernel regressions. Instead this testing should focus more on user interfacing activities including install, glibc, API.
  • Resources: 1 machine per supported arch

Kernel Dependency (Future)

  • Duration: 2-4 hours. The volume is expected to be medium, 2-3 times per day (depending on kernel dependencies). It is also expected to be bursty. Weeks may go by with no activity, then multiple packages may change within a day or two. This stage handles when a new package that interacts with the kernel is built. It is expected to verify the new package does not interfere with the kernel’s ability to function. Examples of failures here are when grub2 or dracut changes its layout which prevents a kernel from a successful install. Those updated packages will test against a known kernel binary (from a successful compose) and quickly verify nothing broke.
  • Tests: Broad package testing, these tests are based on kernel dependent packages and their impact to the kernel. Perhaps a simple install, boot to a prompt and a network ping is considered good enough.
  • Resources: 1 machine per supported arch
Last modified June 3, 2021: Attempt to make the 'branch' config docs more clear (5a0d271)