Fixing missing OSCI results

Problem

You get a report of missing OSCI results, e.g.

Any idea why CKI results don't appear on OSCI dashboard?
https://dashboard.osci.redhat.com/#/artifact/brew-build/aid/35242921

Steps

1. From the OSCI dashboard, determine the Brew Task ID.

2. Check the pipeline-trigger logs in Grafana via {deployment="pipeline-trigger"} |= "Task ID" for messages related to the Task ID. Change the filter expression to a suitable part of the time stamp to see the context of the messages and whether a pipeline was actually triggered and or any errors occurred. Determine the Pipeline URL.

3. Check the pipeline URL obtained from the logs for any problems. Select any of the createrepo jobs, and visit the DataWarehouse link at the end of the logs. From there, determine the Checkout ID.

4. Check the gating reporter logs in Grafana via {deployment="gating-reporter"} |= "Checkout ID" for messages related to the Checkout ID. If there are any error messages, continue with investigating UMB problems.

5. Check that Datagrepper successfully received all messages related to the Task ID on the ci.cki.brew-build.test topics. Change the contains query string parameter to match the Task ID. If there are missing messages, continue with investigating UMB problems.

6. In the case that UMB messages were not sent because of gating-reporter configuration problems, DataWarehouse can be made to resend the RabbitMQ messages that in turn trigger gating-reporter to send the UMB messages. Log into the DataWarehouse production instance, and reset the ready_to_report field for the appropriate rows via something like

   KCIDBCheckout.objects.filter(id__in=(
      'redhat:brew-12345678',
   )).update(ready_to_report=False)
   

   Changes should be picked up within 10 minutes by the ReadyToReportCheckouts cron job.