stable URL: /l/testing-cki-builds

Testing CKI Builds externally

How to test CKI builds and submit results

Ready for test messages

CKI sends ready for test messages after a kernel is built and after CKI functional testing completes. This is meant to be used to trigger testing in specialized labs for testing that is not feasible to onboard into CKI (e.g. testing that requires a controlled environment or special hardware).

Ready for test details

Messages are sent to /topic/VirtualTopic.eng.cki.ready_for_test. All labs involved in extra testing will be notified if the topic is changed.

Onboarded labs can filter specific messages they are interested in. Some examples include:

  • Filtering on modified files to only test changes to their subsystem
  • Filtering on system/os to only test changes to specific release
  • Filtering on cki_finished == false to run testing in parallel
  • Filtering on cki_finished == true && status == 'success' to e.g. only run performance tests if CKI functional testing passed
  • Filtering on nonempty patch_urls to only test proposed changes instead of already merged ones
  • A combination of the above

Ready for test message schema

{
    "ci": {
      "name": "CKI (Continuous Kernel Integration)",
      "team": "CKI",
      "docs": "https://cki-project.org",
      "url": "https://gitlab.com/cki-project",
      "email": "cki-project@redhat.com",
      "irc": "Slack #team-kernel-cki"
    },
    "run": {
      "url": "<PIPELINE_URL>"
    },
    "artifact": {
      "type": "cki-build",
      "issuer": "<PATCH_AUTHOR_EMAIL or CKI>",  # Always CKI for merge request testing
      "source_package_name": "<kernel-source-package-name>",
      "source_package_version": "<kernel-source-package-version>",  # optional
      "source_package_release": "<kernel-source-package-release>"   # optional
    },
    "system": [
      {
        "stream": "<kpet_tree_name>"
      }
    ],
    "checkout_id": "<KCIDB checkout_id>",
    "build_info": [
      {
        "architecture": "<ARCH>",
        "build_id": "<KCIDB build_id>",
        "package_name": "<kernel-binary-package-name>",
        "package_version": "<kernel-binary-package-version>",  # optional
        "package_release": "<kernel-binary-package-release>",  # optional
        "kernel_package_url": "<LINK_TO_REPO>",
        "debug_kernel": bool
      },
      ...
    ],
    "patch_urls": ["list", "of", "strings", "or", "empty", "list"],
    "merge_request": {
      "merge_request_url": "link-or-empty-string",
      "is_draft": bool,
      "subsystems": ["list", "of", "strings"],
      "jira": ["list", "of", "jira", "links"],
      "bugzilla": ["list", "of", "bz", "links"],
      "qa_contacts": ["list", "of", "jira", "qa", "contact", "emails"]
    },
    "branch": "name-of-branch-or-empty-string",
    "modified_files": ["list", "of", "strings", "or", "empty", "list"],
    "pipelineid": <ID>,
    "cki_finished": bool,
    "status": "success|error|fail", # status NOT present if cki_finished is false
    "category": "kernel-build",
    "namespace": "cki",
    "type": "build",
    "generated_at": "<DATETIME_STRING>",
    "version": "0.1.0"
}

For documentation of kpet_tree_name, see Configuration options.

The status field contains the worst status after triaging, i.e. a build where all FAILed and ERRORed tests are triaged will be reported as success. For downstream pipelines that care about CKI test status via cki_finished=true, it should be safe to trigger for any builds that report as success or error.

Installing the kernel from these messages

The CKI kernels can be installed for testing in Beaker via the kpkginstall task:

<job>
  <recipeSet>
    <recipe ks_meta="redhat_ca_cert">
      <distroRequires>
        <distro_arch op="=" value="x86_64"/>
        <variant op="=" value="BaseOS"/>
        <distro_family op="=" value="CentOSStream9"/>
      </distroRequires>
      <hostRequires/>
      <task name="Kernel installation" role="STANDALONE">
        <fetch url="https://gitlab.com/redhat/centos-stream/tests/kernel/kernel-tests/-/archive/production/kernel-tests-production.zip#distribution/kpkginstall"/>
        <params>
          <param name="KPKG_URL" value="{{build_info.kernel_package_url}}#package_name={{build_info.package_name}}&amp;source_package_name={{artifact.source_package_name}}"/>
        </params>
      </task>
    </recipe>
  </recipeSet>
</job>

The distroRequires configuration needs to be adjusted based on the build_info.architecture, system.os and system.variant fields. For debug builds, it is also possible to use the proper variant name including the -debug suffix instead of the debug_kernel parameter.

Results messages

Test results should be sent back via UMB to /topic/VirtualTopic.eng.cki.results. Results must be sent in KCIDB v4 format, which has a detailed schema documented by kcidb-io.