CKI-003: Reports recipients rules

Abstract

While working to improve and simplify the reporter automations, the problem of dynamically defining the list of recipients for a report resurged.

After several discussions, the decision of extracting that responsibility from the reporter was taken, creating the need for a new place where to define this data.

Motivation

Goals

This proposal is intended to simplify the current decisions taking algorithm, aspiring to a simpler reporter service by making the list of recipients more explicit, reproducible and testable.

The current system makes it impossible to identify who is going to receive a report without running the code. The result of this change needs to fix that.

User Stories

Tree maintainers

While testing a revision, the maintainers of this tree need to receive the results of the tests. Then can decide to get only failure notifications or also success ones.

Test maintainers

When tests fail, maintainers for those tests need to be informed of the failures.

Internal results list

The internal testing results list needs to receive all the reports sent, always.

Manual review

Gate keeping reports from reaching consumers need to be possible. Gated results should override all the recipients and only be sent to the manual reviewers.

This list of reviewers is kept separated from the list of recipients and it overrides them in case that a revision needs to be reviewed and the review has not happened yet.

The pipeline ‘review’ job notifies DataWarehouse that a revision’s results were reviewed and the reviewer recipients list is replaced with the real recipients.

Revision contacts

KCIDB schema defines a list of contacts as:

List of e-mail addresses of contacts concerned with this revision, such as authors, reviewers, and mail list

Currently this list is populated only with the brew task submitters. These should be treated as tree maintainers.

Approach

Guidelines

Define static rules

Using a syntax similar to Gitlab rules, define the reporting rules for each tree on the pipeline-data definitions.

---
tree:
  report-rules:
    - if: [and-conditions]
      send_to: [recipients]
      send_cc: [recipients]
      send_bcc: [recipients]

Templating

By using YAML anchors, powered by CKI’s config tree, define default rules and allow per-tree configurations.

---
.rule-foo: &rule-foo
  if: foo
  send_to: foo

.rule-bar: &rule-bar
  if: bar
  send_cc: bar

.rules-generic: &rules-generic
  - *rule-foo
  - *rule-bar

tree-1:
  report-rules: *rules-generic

tree-2:
  report-rules:
    - *rule-foo

Keywords

Use reserved keywords to represent groups of recipients, such as failed_tests_maintainers, submitter, etc. and conditions of the results, i.e always, success, has_failed_waived, etc.

---
tree:
  report-rules:
    - if: success
      send_to: submitter
    - if: failed_tests
      send_cc: failed_tests_maintainers
    - if: always
      send_bcc: some-mailing-list@redhat.com

Evaluation

All the rules must be evaluated and the final recipients list be the result of merging each rule’s outcome.

Single source of truth

These rules must define the final list of recipients (or groups of recipients) for a pipeline.

If a certain tree has no rules configured, that tree should have no reports sent.

Proposal

Recipient keywords

• submitter: The submitter of the change.
• failed_tests_maintainers: Maintainers of the failed tests.
• origin: An email/mailing list for the tested git tree.
• subscribers: Emails that have subscribed to this tree/revision via Datawarehouse.

Condition keywords

• always
• success: On overall success of the pipeline.
• failed: On overall failure of the pipeline (if any step of the pipeline fails).
• failed_tests: For when the pipeline fails in the testing step.

Recipient target keywords

• send_to
• send_cc
• send_bcc
• override_ignore: A list of recipients that will be filtered from all other fields before sending.

Benefits

The resulting rules need to be easy to read for humans and parse by machines, allowing us to understand the recipients by simple observation.

Drawbacks

Consistency

By using keywords for recipients groups, like test-maintainers or submitter the translation of these rules into email addresses needs to be done by the piece of code evaluating them.

This also generates the need to have keywords synchronized between the definition and the parsers.