Jupyter overview

Version/Environment (if relevant):

This was written during release 3.x, a few of the concepts still apply to 4.x & 5.x, but some do not.

Jupyter (aka IPython Notebook) is a popular tool for working with Python, R, Julia and other languages interactively. Domino has implemented workspaces which allows you to run Jupyter Notebooks (and other tools) on powerful hardware easily or share your work with others with one click.

Table of Contents

• Basics
• Inline Plotting
• Versions and Packages
• Syncing Files from Jupyter Notebooks to Domino
• Viewing Notebook Files
• Comparisons and Commenting
• Using Notebooks for Reporting
• Troubleshooting

Basics

All Domino projects are configured to be able to launch IPython Notebook sessions natively. Click on the Workspace icon in the left navigation bar:

Click Jupyter icon, select your hardware, and click launch

Domino will start Jupyter on a machine with your selected hardware and copy your project files there. Your Jupyter Session will be in a new tab

Our default Jupyter installation has kernels for Python 2.7, R, and Julia. It also supports creating Terminal sessions. We support customizations to the Jupyter installation, so you can use whatever kernels you want.

Heads up!

Pro Tip: You can also ssh to the instance hosting the Jupyter notebook, which can be found by clicking on the runs icon in the left navigation. See SSH Into Running Instance for more information. 

Workspaces are like any other executing Run in Domino: they execute inside a secure, isolated environment, we bill by runtime, and any changes to the project are committed when the run completes or when you sync with Domino. 

Inline Plotting

By default, we’ve configured the Notebooks to plot inline, so no extra configuration is required. Please reach out if you need to configure a different plot style.

We also support Plotly's interactive widgets, as described in our blog post.

Versions and Packages

If you need other 3rd party packages install, they can be specified as in any other run. Please see our “Dependencies” documentation for details on how to do this.

Syncing files from Jupyter Notebook to Domino

When you start your Jupyter session, Domino put all your project's files onto the server were it’s running. Frequently, though, you may want to get updated files into your Jupyter server — or you may want to get results out of the server.

Domino makes it easy to synchronize your project files with the files on the Jupyter server. You can open up the Sync menu for your options.

1. A “Full Sync” will first get the latest files from your Domino project, then it will commit changed files from your Jupyter server to your Domino project, where you can access them through the “Files” section of your project, download them to your computer, etc.
2. “Get Latest Files” will copy files from your project on Domino down to the Jupyter server (but local files on the Jupyter server won’t get committed back to Domino with this action) 

You can see what files you'll be committing back to Domino before you sync by opening the Session Details in the sidepanel on the left - open up the side panel by clicking this icon: 

Viewing files

It can take a minute to spin up a server, and in many cases, it’s important to be able to quickly get a view of what notebooks are available.

Domino now renders .ipynb files in your project directly through the web UI, so you can see the contents of a notebook without running a whole server. The image below shows what happens if you simply browse to view an ipynb file — there is no Jupyter server running here.

This lets you turn your Domino projects into a powerful notebook gallery to share with your colleagues. If someone sees a notebook they like, they can spin it up with one click on a running server. Or they can fork your project to make their own changes.

Comparison and commenting

Domino already provides powerful collaboration tools for data science work, such as comparing results between experiments and facilitating discussion. Now these features work great with ipynb files, too. For example, you can compare two different sessions you worked in and see the differences between the two versions of your notebook.

And like any other file or result, you can leave comments about notebooks, which will be shared with your colleagues.

Using notebooks for reporting

In addition to running notebooks interactively and viewing them statically, Domino lets you run notebook files as batch jobs: we’ll calculate the notebook and save the result as HTML, which we’ll host on the web so your colleagues can see it. To run a notebook as a "batch" job, you can either (a) click the "run" button next to the notebook in the files view; (b) or do a normal "domino run" command from your CLI, just specify the notebook name (e.g., "domino run foo.ipynb")

Alternatively, you can set your notebooks to run on a schedule using our Scheduled Runs feature, so your calculated, rendered notebook can be sent out as a report.

Troubleshooting

Notebook authentication
As of version 4.3, Jupyter notebooks enable token-based authentication by default, resulting in this log-in page when starting an interactive session:

While this level of security protection is recommended, notebooks running inside Domino already handle user authentication and access restrictions, making this additional step superfluous. To disable this feature, simply add the following line of commands to the pre-setup script of a custom Compute Environment, or as a RUN command inside your docker file: 

printf "\nc.NotebookApp.token = u'' \n\n" >> /home/ubuntu/.jupyter/jupyter_notebook_config.py && chown -R ubuntu:ubuntu /home/ubuntu/.jupyter

Open Session button does not appear when importing project files

There is a known issue where running later versions of Jupyter can result in the "Open Session" button never appearing, only when files are being imported from another project. This is addressed in a later version of Domino, but in the meantime if you are experiencing this, you can workaround by adding the following lines to the pre-setup script of a custom environment definition:

if [ -f .domino/launch-ipython-server ]; then
    sed -i.bak "s#c.NotebookApp.default_url = '/tree\${MAIN_PROJECT_REDIRECT_DIR}/'#c.NotebookApp.default_url = '/tree\${MAIN_PROJECT_REDIRECT_DIR}'#" .domino/launch-ipython-server
fi