Datasets page fails to load with error: "Failed to fetch Dataset details."

Version/Environment (if relevant):

This applies to Domino version <5.4.

Issue:

In the UI, the Admin Datasets page fails to load with the error:

• Failed to fetch Dataset details. 

In the Chrome Developer Tools, the Console tab shows these errors:

• Failed to load resource: the server responded with a status of 403 (Forbidden) 
• Failed to load resource: the server responded with a status of 502 ()

Root Cause:

This is due to hitting the default timeout. This is a known issue where the nucleus-frontend is taking longer than the default 75-second timeout to retrieve the datasets and snapshots. This can happen for very large deployments. 

Resolution:

This issue is resolved in Domino version 5.4.0. Reference DOM-40828.

Workaround:

As a temporary workaround, Domino Admins with access to the Kubernetes cluster can increase the timeout value. Since this requires an edit to the frontend yaml file and a restart of the frontends, we recommend applying this change after business hours.

1. In the nucleus-frontend deployment yaml, go to Containers → nucleus-frontend → Environment → DOMINO_COMMAND_OPTS.
2. At the bottom of the DOMINO_COMMAND_OPTS section, add the line: Dplay.server.http.idleTimeout=200s
3. Save the change. This will trigger a restart of the nucleus-frontend pods with the new timeout.
4. In the UI, test loading the Admin Datasets page. Generally, 200 seconds should suffice. However, you may have to set this value higher if you continue to see the error.

Please note that this is not a long-term solution. This is a temporary workaround until you upgrade your Domino version to 5.4.0 or higher.  

Notes/Information:

Additional caveats: Although this change will survive a pod-restart, the setting will not persist after an upgrade or a new installation since the nucleus-frontend deployment yaml will be overwritten to not contain this new flag. Also, increasing the timeout could also keep other connections alive for longer. In extreme circumstances, this could exhaust resources.