Adding a new gitlab-runner configuration
How to create a new gitlab-runner configuration and attach it to the correct projects
Operations handbook
Procedures for running and fixing the CKI setup
This page has an internal companion page which might contain additional information.
Adding a new Kubernetes deployment context
How to enable Kubernetes deployments to a new cluster and/or namespace in deployment-all
Administration Handbook
Instructions for sane DataWarehouse maintenance
Celebrating success
How to celebrate a success from IRC
Changing the Fedora release for the tested Koji builds
How to change the Fedora release that has its Koji kernel builds tested
Creating diagrams
How to add or modify diagrams
Creating DW service accounts
How to create service account for DataWarehouse
DataWarehouse Authorization and Permission
How DataWarehouse hides internal data and handles user access
Debugging a failing GitLab pipeline job
How to allow a kernel developer to investigate from the inside why a GitLab pipeline job failed
Debugging DataWarehouse triager and issue regexes
How to investigate why a certain test was not triaged in DataWarehouse
Disabling zstream testing
Steps to do when a variant or a complete kernel stream drops out of support
Enabling a new adhoc gitlab-runner
How to put an adhoc gitlab-runner into service
Enabling scratch build testing
How to enable Koji/Brew scratch build testing for a new RHEL major version
Fixing GitLab job system failures
How to investigate GitLab CI/CD job runner system failures
Fixing missing Koji/Brew builds
How to investigate a report of a missing Koji/Brew build
Fixing missing OSCI results
How to investigate a report of a missing OSCI results
Freezing deployments
How to stop continuous deployment for a while
Investigating UMB problems
How to investigate problems with interfacing to the Red Hat Universal Message Bus
Mirroring a centos-stream/tests/kernel into the VPN
How to set up mirorring of a repository in the centos-stream/tests/kernel group into the VPN
Profiling DataWarehouse API performance (time, query and memory)
How to profile and optimize slow and resource-intensive endpoints
Changing the configuration of the RabbitMQ nodes
How to mess with the RabbitMQ cluster without causing an outage
Refreshing recipe Beaker machines
How to solve lack of machines after an outage
Refreshing tokens
How to renew/rotate time-limited secret tokens
Removing a kernel zstream
Steps to do when support for a kernel stream has to be removed.
Purging a merge request from GitLab
How to remove all data associated with a merge request
Renewing AWS savings plans and instance reservations
How to safely navigate AWS for the scary task of committing for future spent
Syncing the data in the staging instance of DataWarehouse
How to restore a production backup into the staging instance of DataWarehouse
Adding an expression to the pipeline herder
How to get the pipeline-herder to retry GitLab jobs with certain characteristics
Rotating secrets
How to systematically rotate all secrets
Shutting down CKI
How to shut down CKI kernel testing, and communicate a planned or unplanned shutdown
Triggering test pipelines using the CKI bot
Features and details of CKI bot configuration
Updating cross-compilers
How to update the cross-compiler packages
Upgrading machines to a newer Fedora version
How to reprovision machines with a newer Fedora version without causing an outage
Updating pipeline images
How to update the container images used in the pipeline
Updating pipeline repositories
How to update the pipeline repositories
Updating y-stream composes
How to update y-zstream composes
Working with Kubernetes
How to access and work with production environments based on Kuberenetes
Zstreaming a kernel release
Steps to do when a kernel stream is to be released.