Understanding Maintenance Mode Alerts in Domino 3.x

Occasionally, Domino Support may reach out to inform Domino Admins that we were alerted to a high number of Executors having entered Maintenance Mode.

Below, we will explain what this means, and the steps required to take the Executors out of Maintenance Mode.

An executor will enter maintenance mode if any one of the following conditions are met:

1. Dispatcher sends a health check request to the executor and doesn't get a valid response repeatedly fail for 15 minutes (most common).
2. Dispatcher receives no response or signals from the executor (i.e. network or system issue). Status will be labeled 'Unreachable'.
3. Executor fails its own health checks and reports unhealthy to the Dispatcher.
4. Admin's can manually place an executor into Maintenance Mode through the Dispatcher UI.

An executor in maintenance mode will exhibit the following effects:

1. Active runs on an executor in Maintenance Mode will complete as normal, uninterrupted.
2. No new runs are scheduled to the executor.
3. Dispatcher will stop executor if it's been idle (no active runs) for longer than 120 minutes (configurable through central config)
4. An executor cannot come out of Maintenance Mode unless manually unset by an admin through Dispatcher UI.
5. Executors in MM will not be terminated by dispatcher.

Domino Admins will be able to take the executor's out of maintenance mode from within the Dispatcher UI in Domino's Admin section.

As seen below, from the Domino Admin menu, select Dispatcher in the navigation header, scroll down to your Executors, and select the Actions link on the right hand side. 

Executors in maintenance mode will be highlighted in yellow:

To take the Executor out of maintenance mode, first access the Actions menu, then start the Executor, and use the MM off button to remove the maintenance mode tag:

If the Executor fails to start, fails to exit maintenance mode, or immediately re-enters maintenance mode, at that time the executor can be terminated.

The system will then spin up new executors as needed based upon the user run queue.

I