Prometheus fails when upgrading to 5.3.x

Version:

Domino 5.3

Issue:

When upgrading from a version below 5.3.x, the install may fail on the prometheus release.

The error may look something like:

2022-11-21 21:53:00,798 - Error: UPGRADE FAILED: an error occurred while rolling back the release. original upgrade error: cannot patch "prometheus-node-exporter" with kind Service: Service "prometheus-node-exporter" is invalid: spec.clusterIPs[0]: Invalid value: []string(nil): primary clusterIP can not be unset: no NetworkPolicy with the name "prometheus-alertmanager" found

The install stops after failing to upgrade Prometheus.

Resolution:

There are two ways to resolve this installation issue. Try the first method before using the hammer approach.

1. Delete the resource(s)

   1. (Try this first) Delete the specific resources:

      • kubectl -n PLATFORM_NAMESPACE delete svc prometheus-node-exporter

      • kubectl -n PLATFORM_NAMESPACE delete netpol prometheus-alertmanager
        
        

   2. (Try this if installation still fails) Delete the Prometheus helm chart

      • helm -n PLATFORM_NAMESPACE delete prometheus
        
        

2. Restart the installation process

Notes/Information:

---