cki_tools.gitlab_repo_config

Automated GitLab repository and group configuration management

Overview

The gitlab_repo_config tool provides automated GitLab repository and group configuration management through declarative YAML configuration files.

The tool operates by reading YAML configuration files that describe the desired state of GitLab resources, comparing this against the current state via GitLab API calls, and optionally applying necessary changes to bring repositories and groups into the desired state.

Usage

CLI Interface

usage: gitlab_repo_config.py [-h] [--fix] [--project-url PROJECT_URL]
                             [--group-url GROUP_URL] [--config CONFIG]
                             [--config-path CONFIG_PATH]

options:
  -h, --help            show this help message and exit
  --fix                 Fix detected problems
  --project-url PROJECT_URL
                        Only run the checker for one project
  --group-url GROUP_URL
                        Only run the checker for one group
  --config CONFIG       Path to the config file
  --config-path CONFIG_PATH
                        Path to the config file

Operation Modes

Check Mode (default): Validates current GitLab configuration against desired state and reports discrepancies without making changes.
Fix Mode (--fix): Automatically applies necessary changes to bring GitLab resources into compliance with the desired configuration.

Configuration Files

General configuration file structure

The configuration file defines settings for GitLab groups and projects using a hierarchical YAML structure based on a top-level mapping. Mapping keys follow the format human-readable-group-prefix/suffix, with only the suffix being meaningful:

Group Settings (suffix group): Configure GitLab group properties, member access, and LDAP/SAML integration:

# Group configuration for https://gitlab.com/example-org/
org/group:
  url: https://gitlab.com
  group_path: example-org
  group:
    description: 'Example organization'
    project_creation_level: maintainer
    visibility: public

Project Settings (suffix equals the project path): Individual repository configurations including features, access controls, and CI/CD settings:

# Project configuration for https://gitlab.com/example-org/example-project/
org/example-project:
  url: https://gitlab.com
  group_path: example-org
  project:
    issues_access_level: enabled
    merge_method: merge
    visibility: public

Project Defaults (suffix default): Apply configuration templates to all projects directly within a group automatically:

# Applied to all direct projects in the group https://gitlab.com/example-org/
org/default:
  group_path: example-org
  project:
    only_allow_merge_if_pipeline_succeeds: true
    remove_source_branch_after_merge: true

Ignore Flag: Skip processing for specific projects:
```
org/example-project:
  ignore: true
```

Inheritance and Composition

On top of the YAML-provided features like anchors, a basic inheritance mechanism is provided:

Default Template (.default): Default configuration recursively applied to top-level mapping values:

.default:
  url: https://gitlab.com
  project:
    default_branch: main
    merge_method: merge
    only_allow_merge_if_pipeline_succeeds: true

Mixins: Reusable configuration fragments for common patterns, starting with a dot:

.org_project:
  group_path: example-org

.private_mixin:
  project:
    builds_access_level: private
    visibility: private

.security_mixin:
  protectedbranches:
    '*':
      code_owner_approval_required: true
      merge_access_levels: [access_level: 40]

Composition (.extends): Inherit from multiple configuration templates/mixins on top of the .default template:

org/example-project:
  .extends: [.org_project, .private_mixin, .security_mixin]
  # Inherits base settings, group membership, private visibility, and security controls
  project:
    description: "Some internal project"

GitLab API Coverage

The tool integrates with the applicable GitLab APIs to manage the repository and group settings. For detailed parameter information, refer to the GitLab API documentation.

Repository Management

Projects API: Repository visibility, feature toggles, merge settings, and general project configuration:

org/example-project:
  project:
    description: "Example project demonstrating project API settings"
    issues_access_level: enabled
    merge_requests_access_level: enabled
    visibility: public

Protected Branches API: Branch protection rules, push/merge access levels, and force push controls:

org/example-project:
  protectedbranches:
    main:
      merge_access_levels: [access_level: 30]  # developer level
      push_access_levels: [access_level: 0]    # not allowed
      allow_force_push: false
      code_owner_approval_required: true
    'feature/*':
      merge_access_levels: [access_level: 30]
      push_access_levels: [user_ids: [1234, 5678]]
      allow_force_push: true

Protected Tags API: Tag creation permissions and access controls:

org/example-project:
  protectedtags:
    '*':
      create_access_levels: [access_level: 40]  # Maintainer level
    'v*':
      create_access_levels: [access_level: 50]  # Owner level

Remote Mirrors API: Repository mirroring from upstream sources:

org/example-project:
  mirror:
    url: "https://github.com/example/upstream-repo.git"
    username: "mirror-user"
    password: "mirror-token"

Project Forks API: Fork relationship configuration:

org/example-project:
  forked_from_project: "upstream-org/upstream-project"

Group Management

Groups API: Group settings and LDAP/SAML group linking:

org/group:
  group:
    description: "Example organization"
    project_creation_level: maintainer
    require_two_factor_authentication: true
    shared_runners_setting: enabled
    subgroup_creation_level: owner
    visibility: public
  ldap_group_links:
    developers:
      group_access: 30
      provider: ldapmain
    admins:
      group_access: 50
      provider: ldapmain
  saml_group_links:
    gitlab-developers:
      access_level: 30
    gitlab-admins:
      access_level: 50

Access Control

Merge Request Approvals API: Merge request approvals and associated rules:

org/example-project:
  approvals:
    approvals_before_merge: 2
    reset_approvals_on_push: false
    merge_requests_author_approval: false
  approvalrules:
    "All Members":
      rule_type: any_approver
      approvals_required: 1
    "Security Team":
      rule_type: regular
      approvals_required: 1
      user_ids: [123, 456]

Push Rules API: Repository push restrictions like file size limits, and commit message validation:

org/example-project:
  pushrules:
    prevent_secrets: true
    commit_message_regex: '^(feat|fix|docs|style|refactor|test|chore):'
    branch_name_regex: '^(main|develop|feature/.+|hotfix/.+)$'
    file_name_regex: '\.envrc$'
    max_file_size: 100  # MB

CI/CD Integration

Project-level CI/CD Variables API: Project and group-level CI/CD variable management:

org/example-project:
  variables:
    DATABASE_URL: "postgres://localhost/myapp"
    API_TOKEN:
      value: "secret-token"
      environment_scope: production
      masked: true
      protected: true

Pipeline Schedules API: Automated pipeline triggers with cron-based scheduling:

org/example-project:
  pipeline_schedules:
    "Daily Build":
      description: "Run daily builds"
      ref: refs/heads/main
      cron: "0 2 * * *"
      active: true
      variables:
        BUILD_TYPE: daily
        ENVIRONMENT: staging
        API_TOKEN:
          value: "secret-token"
          variable_type: file

Protected Environments API: Environment-specific deployment access controls:

org/example-project:
  protectedenvironments:
    production:
      deploy_access_levels: [user_ids: [123, 456]]
    staging:
      deploy_access_levels: [access_level: 30]  # Developer level

Resource Cleanup & Retention

Branch and Tag Cleanup: Policies for notifications about unexpected branches and tags - they are never removed even with --fix:
```
org/example-project:
  branches: {}  # only keep the default branch
  tags:
    'v.*':  # regex fullmatch()
```

Environment Management: Environment retention policies:

org/example-project:
  environments:
    keep:
      - production/.*  # regex fullmatch()
      - staging/.*
    expire_days: 30

Container Registry Cleanup: Container image cleanup policies:

org/example-project:
  container_registry:
    default:
      'latest': {}
      'mr-\d+':   # regex fullmatch()
        keep_n: 10
        keep_younger: 1w