Fixing GitLab job system failures

How to investigate GitLab CI/CD job runner system failures

This page has an internal companion page which might contain additional information.

Problem

You get a problem of a CKI GitLab CI/CD job failing like

Determine the gitlab-runner responsible for the job. This can be derived from the gitlab-runner name in the job output. In the screenshot above, the wf-aws-aws-internal-b-dm-internal-build refers to the internal runner in AZ b.
Log into the gitlab-runner machine via ansible_ssh.sh.
Look at the output of the journal for the gitlab-runner via
```
journalctl --since today --all --unit gitlab-runner
```
Get started by looking for ERROR and red lines in the output.