During an upgrade, a secret is missing (rabbitmq-ha.forge)

Problem

During an upgrade, the results indicate a secret is not present.

I have a problem where the upgrade is stuck on or failing to install rabbit. kubectl describe on the pending pods may include:

WarningFailed3m45s (x12 over 5m52s)kubeletError: secret "rabbitmq-ha.forge" not found

Resolution

Try to create the secret. This is an example of a yaml file for this secret:

$ kubectl get secret -n domino-platform rabbitmq-ha.forge -o yaml
apiVersion: v1
data:
rabbitmq-password: BaseEncodedPassword # generate with echo newpassword |base64 ; echo
rabbitmq-username: Zm9yZ2U= # This maps to 'rabbitmq'
kind: Secret
metadata:
creationTimestamp: "2023-01-05T15:54:43Z"
name: rabbitmq-ha.forge
namespace: domino-platform
resourceVersion: "3801"
uid: Random string # probably skip this 
type: Opaque

Create this file - applying a new password and remove the uid line, kubectl apply -f <file>, namespace may be required although it is included in the file.

Root cause

This is seen in sandboxed environment with no know reason why this as been lost. The secret will be read later by the rabbitmq deployment.