cki_tools.cki_notifier
External testing is triggered via CKI notifier messages.
The service consumes DataWarehouse checkout notifications from RabbitMQ and
publishes ready-for-test messages to Kafka (kafka.tx.cki.ready-for-test)
for External testing.
The notifier can be deployed locally by running
python -m cki_tools.cki_notifier
Configuration via environment variables
| Name | Secret | Required | Description |
|---|---|---|---|
DATAWAREHOUSE_URL |
no | yes | DataWarehouse URL |
DATAWAREHOUSE_TOKEN |
yes | yes | DataWarehouse access token |
JIRA_SERVER |
no | no | JIRA server URL |
JIRA_TOKEN_AUTH |
yes | no | JIRA personal access token |
RABBITMQ_HOST |
no | no | AMQP host |
RABBITMQ_PORT |
no | no | AMQP port, TLS is used for port 443 |
RABBITMQ_VIRTUAL_HOST |
no | no | AMQP vhost, defaults to / |
RABBITMQ_USER |
no | no | AMQP user |
RABBITMQ_PASSWORD |
yes | no | AMQP password |
RABBITMQ_CAFILE |
no | no | AMQP CA file path |
WEBHOOK_RECEIVER_EXCHANGE |
no | no | AMQP exchange to receive messages |
RABBITMQ_QUEUE |
no | no | AMQP queue name to use on the exchange |
RABBITMQ_ROUTING_KEYS |
no | no | AMQP routing keys for the messages sent to the queue |
CKI_DEPLOYMENT_ENVIRONMENT |
no | no | Define the deployment environment (production/staging) |
CKI_LOGGING_LEVEL |
no | no | logging level for CKI modules, defaults to WARN; to get meaningful output on the command line, set to INFO |
CKI_METRICS_ENABLED |
no | no | Enable prometheus metrics. Default: false |
CKI_METRICS_PORT |
no | no | Port where prometheus metrics are exposed. Default: 8000 |
Message routing
flowchart TD
msg[DataWarehouse AMQP message] --> validate[Validate checkout]
validate --> route{Status}
route -->|build_setups_finished| pre_test[Pre-test ready-for-test]
route -->|ready_to_report| post_test[Post-test ready-for-test]
route -->|checkout_issueoccurrences_changed| retriage[Post-test on fail->pass only]
pre_test --> kafka[Publish to kafka.tx.cki.ready-for-test]
post_test --> kafka
retriage --> kafka
Incoming messages must have object_type: checkout. The diagram shows the
happy path; messages that fail validation or do not match a routing rule are
ignored (see Filtered and ignored messages).
Supported status values:
| Status | Message type | When a ready-for-test message is sent |
|---|---|---|
build_setups_finished |
pre-test | Builds are ready; test results are not included (cki_finished: false). |
ready_to_report |
post-test | Testing finished; includes test status (cki_finished: true). |
unknown |
pre- or post-test | Uses checkout misc flags from DataWarehouse (see below). |
checkout_issueoccurrences_changed |
post-test | Only on fail -> pass after the initial report (see below). |
unknown status and the race condition
unknown is used when the message does not name a checkout stage explicitly,
for example manual CLI runs or out-of-draft MR notifications. The notifier
loads the checkout from DataWarehouse and uses misc.ready_to_report or
misc.notification_sent_build_setups_finished to choose post-test or pre-test.
For ready_to_report and build_setups_finished, DataWarehouse may emit the
message before the matching misc flags are persisted. The notifier trusts the
message status in that case instead of waiting for the flags to appear in
DataWarehouse.
Issue occurrence updates
After a checkout reaches ready_to_report, DataWarehouse may send
checkout_issueoccurrences_changed when triage changes (waive, unwaive,
regression, and so on). The notifier sends another post-test message only when
all of the following hold:
- checkout
misc.ready_to_reportis true (initial report already happened) - message
misc.checkout_outcome_changedis true (pass/fail outcome flipped; set by DataWarehouse in top-level messagemisc) - the current overall test status is not fail or error (no pass→fail regression after a prior success notification)
Retriage that does not change the overall outcome, and pass→fail regressions, are ignored.
Post-test message status is derived from cki_lib.kcidb.checks.compute_overall_status
(success, fail, or error).
Filtered and ignored messages
Before routing, messages are dropped when:
- the status is unsupported or
object_typeis notcheckout - the checkout is retriggered (
object.misc.retrigger) or not from originredhat - the checkout is invalid, has no builds, or is stale (merged MR, or head pipeline no longer matches the MR)
Any other status that does not match the routing rules above is ignored as well.
Manual triggering
Messaging can also be manually triggered via the command line. First, make sure to have the required environment variables above configured correctly. Then, the sending of CKI notifier messages can be triggered with
CKI_LOGGING_LEVEL=INFO \
CKI_DEPLOYMENT_ENVIRONMENT=production \
python3 -m cki_tools.cki_notifier \
--status STATUS \
--checkout-id CHECKOUT_ID
STATUScan bebuild_setups_finished,ready_to_report,checkout_issueoccurrences_changed, orunknown. It represents the checkout stage, which determines whether testing has finished and whether test results are included. Useunknownto rely on the DataWarehouse checkout stage.CHECKOUT_IDis the KCIDB checkout id (e.g.redhat:12345).
Manual runs do not pass message misc, so
--status checkout_issueoccurrences_changed will not match the issue-occurrence
routing rules above unless the message body is constructed with
misc.checkout_outcome_changed.