I’m wondering if the reason I could never get to the end of hyperparameter tuning is because the program stopped running after I left work for the day and disconnected my laptop from our office network. To troubleshoot this further, I wrote a simple sleep test program. If I start the program and then close the chrome tab, even though the workspace is still running the program stops running. If I go into Domino and open the still running workspace, the program is frozen at the spot where I closed the tab. Note the lack of * in the second screen shot.
Can you tell me if this is expected behavior? What is happening and how can I handle this situation?
Comments
1 comment
Hi Aman,
Thanks for the question. This is a behavior we've seen with Jupyter. The key thing to note here is while the execution seems to have "stopped", it is only the output that is interrupted, not the code execution. In your example, if you were to print(i) in a new (second) cell below the sample code while leaving the first cell running, closing the tab, and then returning to it, you'll see that the new cell does not execute immediately. What's happening here is that Jupyter is still executing the cell but has stopped writing the output. The new cell appears to "hang" until the time.sleep(10) x 20 has run to its end.
You can either wait for the time to run out or test this by hitting the stop button after you reopen the workspace and then just execute the second cell.
A couple of things you can do to handle this case:
1. Use
%%captureto store the output and print it at a later stage once the cell has finished executing.2. You can also convert the .ipynb to py and run code as a Job in Domino and look at the output.
Please sign in to leave a comment.