General Information

The Dispatcher is a core component of the Domino application and it has multiple responsibilities.

• Schedule runs to executors

• Get the state of each executor and run

• Scale-up and turn down executors

• Get active run information and update the database

• Execute health checks on executors, marking them unhealthy as needed

Troubleshooting

Restarting the Dispatcher

This is needed if there are changes made to the Dispatcher configuration or Central Config or as a general troubleshooting measure.

Domino ≤ 3.6.x

This process takes ~30 seconds and will be disruptive to any new runs attempting to be scheduled or runs that are syncing back during the restart period.

Active runs and executors are not affected.

docker restart domino-dispatcher

The Dispatcher should come back within 30 seconds. View logs to ensure it comes up properly.

docker logs --tail 20 --follow domino-dispatcher
...
...
[info] [Global] Application started. correlationId="PSjLQsw08d"

Domino ≥ 4.x

In 4.x deployments deleting the pod for nucleus-dispatcher-* is the standard way to restart the dispatcher, since there is a replica set associated with it, to rebuild. 

kubectl -n aws-staging-domino logs nucleus-dispatcher-55dcd5b87d-knwx6 --timestamps 
kubectl -n aws-staging-domino logs nucleus-dispatcher-55dcd5b87d-knwx6 --timestamps --tail=300