Adding a new gitlab-runner configuration

How to create a new gitlab-runner configuration and attach it to the correct projects

Problem

A new gitlab-runner needs to be attached to a project.

Steps

  1. Verify that the deployment-all checkout works correctly including access to AWS and Beaker:

    aws_saml_login.sh --role poweruser
    echo "$BEAKER_CKI_TEAM_AUTOMATION_SSH_PRIVATE_KEY" | sed 's/\r//' | ssh-add -
    gitlab-runner-config/deploy.sh configurations diff
    
  2. (Optional) Reserve a new Beaker machine for an adhoc Beaker-based gitlab-runner for architecture ARCH (x86_64, s390x, ppc64le, aarch64) by running

    podman run --rm -it \
        -e BEAKER_URL \
        -e BEAKER_CKI_TEAM_AUTOMATION_KEYTAB \
        -e BEAKER_CKI_TEAM_AUTOMATION_SSH_PRIVATE_KEY \
        -e CGROUPS=v1 \
        -v .:/code \
        -w /code \
        registry.gitlab.com/cki-project/containers/python:latest \
        bash -c \
        'pip install --no-deps git+https://gitlab.com/cki-project/cki-lib.git &&
        ./beaker_provision.sh ARCH'
    

    Add the host name of the machine to ansible/inventory/beaker.yml, and configure it as a gitlab-runner via

    PLAYBOOK_NAME=gitlab-runner-config \
        ./beaker_deploy.sh \
        --limit HOSTNAME
    
  3. If a completely new gitlab-runner deployment is needed, set it up and add it to

    • Beaker machine: ansible/inventory/beaker.yml, all.hosts.children.gitlab_runner.hosts
    • Other machine: gitlab-runner-config/deploy.sh, RUNNER_HOSTS
    • Kubernetes: gitlab-runner-config/deploy.sh, CONTEXTS

    Also add it to gitlab-runner-config/runners.yml.j2 in runner_deployments.

  4. If a new configuration template is needed, add it to gitlab-runner-config/runners.yml.j2 in runner_templates.

  5. Instantiate the template in the correct runner deployment via the runner_configurations variable. If a new configuration is created, make sure it is attached to the correct GitLab projects via runner_registrations.

  6. Run gitlab-runner-config/deploy.sh registrations diff and check that the new runner configurations are reported and do not have a token assigned.

  7. Run gitlab-runner-config/deploy.sh registrations apply --create-missing and add the new tokens to the secrets via ./edit_secret.sh secrets.yml GITLAB_RUNNER_TOKENS.

  8. Deploy the new configurations via gitlab-runner-config/deploy.sh configurations apply.

  9. For non-Kubernetes gitlab-runner instances, restart the gitlab-runner service to enable the monitoring endpoint of gitlab-runner via

    ssh HOSTNAME sudo systemctl restart gitlab-runner
    
Last modified August 2, 2021: Document the setup of Beaker gitlab-runners (6695e51)