Investigating UMB problems

How to investigate problems with interfacing to the Red Hat Universal Message Bus

This page has an internal companion page which might contain additional information.

Problem

Some piece of software has problems connecting to the Universal Message Bus (UMB) or with authentication.

Steps

Try to manually connect to the servers. The server URLs can be found in secrets.yml in cee/deployment-all, or via oc describe dc/amqp-bridge-umb. For example, look for errors in the output of

openssl s_client \
    -connect messaging-devops-broker01.web.prod.ext.phx2.redhat.com:5671 | \
    grep error
openssl s_client \
    -connect messaging-devops-broker02.web.prod.ext.phx2.redhat.com:5671 | \
    grep error

For “Connection refused”, check that the URLs are correct. The expected response is a “bad certificate error SSL alert 42”, which basically means that the client certificate is missing.

140197084915520:error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:../ssl/record/rec_layer_s3.c:1543:SSL alert number 42

Provide the appropriate client certificate. In a checkout of cee/deployment-all, run

echo "$UMB_MESSENGER_CKI_BOT_PROD_PEM" | \
    base64 --decode | \
    openssl s_client \
    -connect messaging-devops-broker02.web.prod.ext.phx2.redhat.com:5671 \
    -cert - | \
    grep error

The server might return another error message if it does not accept the certificate:

140405938968384:error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:../ssl/record/rec_layer_s3.c:1543:SSL alert number 42

Check the certificate, especially the validity interval:

echo "$UMB_MESSENGER_CKI_BOT_PROD_PEM" | \
    base64 --decode | \
    openssl x509 -text