Prometheus has amnesia or missing metrics

After upgrading to Domino 5, some metrics are lost

I have labels but data is not reaching Prometheus?

It seems that kube-state-metrics has, since v2.0, added an option “--metrics-labels-allowlist” which allows you to whitelist labels for kube-state-metrics to report, with the default not matching the original behavior and instead blacklisting nearly everything.

Bring the labels back by editing the prometheus-kube-state-metrics deployment and add the argument to the arg list:

    spec:
      containers:
      - args:
        - --port=8080
        - --resources=daemonsets,deployments,endpoints,pods,replicasets,services,statefulsets
        - --telemetry-port=8081
        - --metric-labels-allowlist=pods=[dominodatalab.com/starting-user-username]

You can also specify multiple labels, or specify wildcards, though this will increase resource usage.