CKI pipeline configuration options

All pipelines share the same code and make decisions based on the passed configuration options in form of trigger variables. Other tools interacting with the pipeline may also need to do the same, one such example would be reporter which needs to know where to send emails for the pipeline.

All the options below are in alphabetical order.

Note that this list does not contain internal pipeline variables that are technically possible to be overridden but are not meant to be touched, such as rpmbuild options or ccache related variables.

For variables required by triggers but not needed for the pipeline run itself, please refer to the pipeline triggers documentation.

Pipeline configuration option list

The pipeline and tooling handles both True and true (similarly with False/false). For simplicity, only capitalized values are mentioned below. The inconsistency is due to historical reasons and is planned to be resolved, progress can be followed in the GitLab issue.

• architectures: Space separated list of architectures to run the pipeline for, in RPM architecture format. Required in every pipeline.

• ARTIFACT_*: Information about where to find original pipeline artifacts if short pipelines are requested via tests_only=true. Used by the CI bot. MUST NOT BE USED IN PRODUCTION.

• artifacts_mode: Where pipeline artifacts should be stored, can be s3 or gitlab, defaults to gitlab.

• branch: Target branch of the kernel repository that should be built and tested. Used in reporting and messaging, not in the pipeline itself. commit_hash must be provided for a correct pipeline run.

• brew_task_id: Brew or Koji task ID. Only applicable for Brew/Koji pipelines. Defaults to False. Only applicable for kernels built as RPMs.

• build_selftests: True if any kernel selftests should be built, False otherwise. Defaults to False. Will cause make kselftest-merge configuration target to be called. Has to be used together with test_debug option (as the selftest config target pulls in debug options). Only applicable for kernels built as tarballs. NOTE selftests build is currently only supported for debug builds on x86_64 architecture, other architectures are skipped.

• builder_image: Container image used for kernel building. Defaults to registry.gitlab.com/cki-project/containers/builder-fedora.

• builder_image_tag: Tag of the builder container image. Format described in container documentation, or latest. Defaults to the value of image_tag variable.

• commit_hash: Commit to check out from the repository. Required if a kernel needs to be built.

• compiler: Which compiler toolchain to use to build the kernel. Can be gcc or clang, defaults to gcc. Specific behavior of clang differs on supported functionality with Fedora configuration for given architecture. Only applicable if a kernel needs to be built.

• config_target: make configuration target to use on top of Fedora configuration files. Defaults to olddefconfig. Only applicable for kernels built as tarballs.

• copr_build: Task ID of a COPR build, similar to brew_task_id. Only applicable for COPR pipelines.

• coverage: True to build a kernel with code coverage. Defaults to False.

• coverage_dirs: Space separated list of directories, captures coverage data only for these kernel directories. Only applicable when coverage=True. Default value in kpet-db repository.

• cover_letter: Link to a cover letter, if patch series are being tested and a cover letter exists. Optional.

• discovery_time: Time of revision discovery to track for KCIDB. Optional. NOTE may be removed.

• disttag_override: Override for the RPM %dist macro of the builder container image. This is necessary if the built kernel belongs to a different distribution release, e.g. .el8_3 for a RHEL 8.3 kernel built on the builder-rhel container image which has a %dist macro of .el8. Only applicable for kernels built as RPMs.

• domains: A space-separated list of regular expressions fully-matching names or slash-separated paths of kpet “host domains” to restrict test execution to. All tests targeting hosts in the matching domains will be executed once for each of the regular expressions. Run kpet domain tree to see available domains. If empty, tests will be executed once in all domains. Defaults to available.

• download_separate_headers: If true, download separate kernel-headers binary from koji. Defaults to false. Useful if the headers are not part of the base kernel package. Only applicable for Brew/Koji/COPR pipelines.

• git_url: URL of the kernel repository that should be built and tested. Only applicable if a kernel needs to be built.

• git_url_cache_owner: The file name for the git cache is determined as owner.repo from the git URL. Explicitly set this variable to override the owner component for the git cache used for git_url.

• image_tag: Container tag used for all generic containers in the pipeline. Regularly updated. Format described in container documentation, or latest.

• scratch: True if the kernel build is a scratch build (Brew/Koji) or is triggered by a merge request, False if not.

• kernel_type: upstream or internal. Used by DataWarehouse and KCIDB submitter to determine whether the kernel build is internal, or can be shared publicly. Defaults to internal.

• kpet_high_cost: Override for kpet’s --high-cost option. Defaults to triggered. See the output of kpet run generate --help command for possible values.

• kpet_tree_family: Value to pass to kpet tree selection script to pick appropriate test definitions.

• kpet_tree_name: Automatically determined from kpet_tree_family and kpet tree selection script by default. Only applicable as an override, e.g. for testing the CI pipelines. If defined, must match a kpet tree name as returned by kpet tree list.

• KPET_ADDITIONAL_VARIABLES: Additional environment variables that should be passed as variables to kpet. Only for pipelines triggered via the CI bot. MUST NOT BE USED IN PRODUCTION.

• mail_add_maintainers_to: Which email header to add test maintainers to when sending them notifications for their failed tests. If defined, can be cc, bcc or to. NOTE in the future it will be only possible to add test maintainers to cc, and the variable will be likely substituted by a new True/False value one.

• mail_bcc: List of email addresses delimited by ,. These will be added to the Bcc: header for email reports. NOTE recipient handling is being reworked and this variable will be removed later.

• mail_to: List of email addresses delimited by ,. These will be added to the To: header for email reports. NOTE recipient handling is being reworked and this variable will be removed later.

• mail_from: Email address to put into the From: header when sending email reports. NOTE email reports are being reworked and a default value might end up being put into a different place.

• mail_to_submitter: True if the submitter should be emailed the results. Defaults to False. NOTE recipient handling is being reworked and this variable may be removed later.

• make_target: targz-pkg if make targz-pkg should be used for building the kernel, rpm if rpmbuild should be used. Defaults to rpm. NOTE should be renamed to build_target in the future to reflect not all kernels are build using the make command.

• manual_review_mail_to: Comma separated list of emails to send the reports for review require_manual_review is True. If not specified and manual review is required, the email is sent to test maintainers and CKI list.

• merge_branch: Required if merge_tree is specified. Name of the branch which should be merged into the tested kernel tree before patch application.

• merge_tree: Optional. Link to a kernel repository that should be merged into the tested kernel tree before patch application. See merge_branch for specifying the branch name.

• merge_tree_cache_owner: The file name for the git cache is determined as owner.repo from the git URL. Explicitly set this variable to override the owner component for the git cache used for merge_tree.

• message_id: Value of a Message-ID header, typically of the last email of the patch series. If specified, email report will be sent In-Reply-To of this email. If missing, email report will be sent without any threading.

• mr_id: ID of the merge request that should be tested. Only applicable for kernels living in GitLab, where changes are not submitted as email patches but as merge requests.

• name: Pipeline type identifier, used e.g. in tree filtering in Data Warehouse. Typically contains upstream subtree name or distribution release, e.g. mainline.kernel.org or kernel-rt-rhel83. NOTE this variable is deprecated as it is ambiguous and must not be used in new code.

• native_tools: True if kernel tools should be built separately from kernel on a native architecture. Defaults to False. Only available for kernels built as RPMs which have the required kernel features backported.

• nvr: Kernel NVR. Required for Koji/Brew/COPR pipelines, ignored otherwise. NOTE given that the pipeline only uses it to determine kernel version it may be removed in the future in favor of a direct kernel_version variable.

• package_name: Name of the kernel package. Required for kernels built as RPMs (both ones built in the pipeline and by build systems) as boot test uses the value. Defaults to kernel.

• patch_urls: Links to raw patches or mboxes to apply, in a form of space separated string. Patches are applied using git am.

• patchwork_url: Link to a Patchwork instance, if Patchwork is the origin of the patches. Used to properly grab patch names, as all patches downloaded from patchwork are called mbox.

• POWER_BUILDER_SUFFIX: Request a ppc64le builder with a specific POWER version. Currently supported values are -p8 and -p9 (POWER 8 and POWER 9, respectively). Defaults to an empty string meaning any builder can be used.

• publish_elsewhere: If True, publish the kernel repository on a Red Hat internal server. Defaults to False. Useful when testing private kernels which are not accessible by the test environment.

• repo_name: COPR repository name in a format username/copr, if a COPR build is supposed to be tested.

• report_rules: JSON encoded list of reporting rules for the current checkout.

• report_template: full or limited. Signifies if internal links such as Beaker job links should be present in the email reports or not, to prevent sending useless information to external people. NOTE reporting is currently being reworked and this option may not be needed in the future and thus will be removed.

• report_types: Comma separated list of string representing how the pipeline results should be reported. Only supported type is email. NOTE email type will be removed in the future with the planned reporting rework.

• require_manual_review: True if the email report requires a manual review by test maintainers before being sent to the mailing list and contributors. Defaults to False. NOTE will be removed in the (remote) future after migration of reporting to DataWarehouse.

• retrigger: True if the pipeline is a retrigger for testing and should be ignored by all tooling. Defaults to False.

• rt_kernel: True if realtime kernel configuration options should be enabled for tarball packaged kernels. Defaults to False.

• selftest_subsets: Optional variable specifying a list of space separated selftest subsets to build. If it’s missing and build_selftests is True, CKI will build all selftest subsets. Only applicable for kernels packaged as tarballs.

• send_pre_test_notification: True if an email should be sent to a patch or build submitter before testing starts. Defaults to False. The email contains links to Beaker jobs for people to follow, thus is only useful for internal contributors. NOTE After migrating to UPT this notification will make no sense as there will be no test logs available in the Beaker jobs, and thus the option may end up being removed or reworked.

• send_ready_for_test_pre: True if the UMB messenger should send a “ready for test” message for parallel testing. Defaults to False.

• send_ready_for_test_post: True if the UMB messenger should send a “ready for test” message for post-CKI testing (e.g. performance testing). Defaults to False.

• send_report_to_upstream: True if the email report should be sent to the contributors. Causes the triggers to add emails to the mail_to, mail_cc and mail_bcc trigger variables and the reporter to send the email to them. NOTE recipient handling is being reworked and this variable will be removed later.

• send_report_on_success: True if email reports for successful test runs should be sent to the contributors, False if not. NOTE recipient handling is being reworked and this variable will be removed later.

• server_url: Value of the --server option of the koji command to use to retrieve builds. Required for Brew/Koji pipelines and for COPR pipelines using download_separate_headers.

• skip_beaker: True if only a dry run for submitting a test job is needed. Defaults to True in CI runs, to False otherwise.

• skip_build: True if the build stage should be skipped in the pipeline. Useful for only running testing with already existing kernels. Defaults to False.

• skip_createrepo: True if the createrepo stage should be skipped in the pipeline. Useful for only running testing with already existing kernels. Defaults to False.

• skip_merge: True if the merge stage should be skipped in the pipeline. Useful for only running testing with already existing kernels. Defaults to False.

• skip_publish: True if the publish stage should be skipped in the pipeline. Useful for only running testing with already existing kernels. Defaults to False.

• skip_results: True if result summary and known issue detection should be disabled. Defaults to False.

• skip_setup: True if the setup stage should be skipped. Defaults to False.

• skip_test: True if the test stage should be skipped. Defaults to False.

• srpm_make_target: make target to build SRPM. Defaults to rh-srpm.

• subject: Email subject to use with email reports for the test results, will be prefixed with the result summary. If missing, a generic subject mentioning the kernel version and pipeline name will be built.

• submitter: Email of the patch or build submitter.

• test_debug: True if the debug kernels should also be tested. Defaults to False. NOTE currently CKI cannot build debug RPMs due to artifact size limitations in GitLab, and s3 artifacts storage needs to be fixed up to make it work. Follow the GitLab epic for more details.

• test_priority: Priority for submitting test jobs. Can be low, medium, normal, high or urgent. Defaults to normal. NOTE not every provisioner supports test run priorities. USE RESPONSIBLY.

• test_runner: Where to run testing. Currently available runners are aws (limited testing available in virtual machines) and beaker. Defaults to beaker.

• test_set: Test set(s) to run for the pipeline, in the kpet generate -s option syntax. By default, all sets are included for patch runs (and tests are picked via patch analysis) and official kernel builds are limited to sanity kt0 testing. Also check out documentation for targeted testing for builds on how to limit test sets for a single build.

• tests_regex: If specified, tests names have to fullmatch() this regular expression to run. Can be combined with test_set to only run specific tests from the set(s). Not meant for production use as the test names can change, always use test_set for production. Current list of test names can be retrieved by running kpet test list.

• top_url: Value of the --topurl option of the koji command to use to retrieve builds. Required for Brew/Koji pipelines and for COPR pipelines using download_separate_headers.

• trigger_job_name: The CI_JOB_NAME which triggered the pipeline. Useful for consumers of downstream pipeline webhook events such as the kernel-webhook’s ckihook. A regular downstream pipeline is expected to be named using the following format: <RHELMAJOR>_<VARIANT>_<TYPE>_<SECLEVEL>:

  • <RHELMAJOR>: RHEL major version, i.e. rhel8 or for CentOS Stream c9s, c10s, etc.
  • <VARIANT>: optional and can be automotive, realtime or realtime[_-]check
  • <TYPE>: baseline or merge[_-]request
  • <SECLEVEL>: optional; only private is a valid value

  There are also special coverage and shadow builds enabled for specific setups. These conform to the schema of <RHELMAJOR>_<VARIANT>_baseline_coverage_build and <CENTOSMAJOR>_<RHELMAJOR>_compat_<TYPE>. Note that the <RHELMAJOR> in case <CENTOSMAJOR> is used as well matches the RHEL name, not the CentOS Stream name.

• watch_branch Git branch to watch to trigger testing on changes. Defaults to branch. Only applicable for use with baseline trigger.

• watch_url: Git repo URL to watch to trigger testing on changes. Defaults to git_url. Only applicable for use with baseline trigger.

• web_url: Value of the --weburl option of the koji command to use to retrieve builds. Required for Brew/Koji pipelines and for COPR pipelines using download_separate_headers.

• zstream_latest: True if this baseline marks the newest zstream release. Used by the CI bot to correctly pick testing pipelines if there is no ystream available (e.g. RHEL7), not used in the pipeline directly.

Last modified April 8, 2022: Add docs for new watch_{branch,url} (a114b25)