Domino All Versions
Jobs / Apps / Models / Workspace present with 'failed to connect' messages in the logs:
2021-04-18 01:20:46 : curl: (7) Failed to connect to localhost port 9000: Connection refused
2022-07-09T07:28:16.918Z: Session terminated, terminating shell...curl: (7) Failed to connect to localhost port 9000: Connection refused
2022-07-09T07:28:16.918Z: Cleanup command failed. Attempt 1/10. Will try again in 6
2022-05-12T09:34:44.000Z: curl: (7) Failed to connect to localhost port 9000: Connection refused
Its important to note that this error message presents itself due to a number of reasons, some of these are documented below:
- Apps not running on the correct IP/Port, Apps in Domino must run with a host of
Exit code (143)- which generally indicates an issue with memory resources, these can also include Out Of Memory (OOM) events.
- ResourceChangeNotifications taking place on the backend Domino environment - This can be seen in support bundle events.json log.
- Apps are considered to be in a "running" state once the run container starts, even though it may actually be installing a bunch of packages, hence its not actually in a started state. To ensure a speedy running state, you can bake dependencies into the compute environment and remove any unneeded dependencies.
- Race conditions - Where there are instances of messages relating to '2 items found when at most one was expected' - This typically occurs when one app fails to terminate before the other is launched and is usually resolved with sleep statements placed in the code.
To ensure, we can get to the root cause of why you may be seeing the above message in the logs. It's highly recommended to review against the above causes first and provide Domino Support with a support bundle, details of how to obtain these are set out in the articles below:
Workspaces - https://tickets.dominodatalab.com/hc/en-us/articles/4407753434900-Obtaining-Support-bundle-Logs-on-a-Workspace-on-Domino-Version-4-3-up
Apps - https://tickets.dominodatalab.com/hc/en-us/articles/4411267187092-Obtaining-Support-bundle-Logs-on-an-App-on-Domino-Version-4-3-up
Please sign in to leave a comment.