Skip to content

How to Run Jupyter notebook off Xanadu cluster?

Jump to bottom
Noah Reid edited this page Jan 29, 2024 · 15 revisions

Introduction

This tutorial will show you how to run Jupyter Notebook on the Xanadu cluster. There are four steps:

  1. Start the Jupyter Notebook server as a batch job.
  2. Establish an SSH tunnel from your local machine to the cluster node hosting your Jupyter Notebook server.
  3. Connect to the server using a web browser.
  4. Turn off the server when you are done using it.

NOTE 1 : You must be connected via UCHC VPN. This is pulsesecure VPN, not Cisco.

NOTE 2 : You must use the "internal" submit node xanadu-submit-int to connect to the cluster. The instructions won't work using the "external" submit node xanadu-submit-ext.

Step 1: Start the Jupyter Notebook server.

First, install Jupyter in your home directory on the cluster. We recommend you use conda (which you may need to install if you have not already).

This conda environment will need to contain all the python modules you want to use in your notebook. If you already have a python environment you're using, you can install Jupyter in that environment and use it instead of a new one. It's possible to use many other languages, including R, in a Jupyter Notebook. If you can't extrapolate from these directions and the Jupyter docs how, please reach out for assistance.

To create a new environment and install Jupyter:

# create a new environment named "jupyter"
conda create -n jupyter
# activate the environment
conda activate jupyter
# install jupyter in the environment
conda install -c anaconda jupyter

With Jupyter installed, we can start a batch job that will run the Jupyter Notebook server on a compute node.

Batch Script

Create the following script and name it xanaduJupyterNotebook.sh. This script can be re-used, you won't need a new copy for each new Jupyter Notebook session.

#!/bin/bash
#SBATCH -J jupyNBook
#SBATCH --partition general
#SBATCH --qos general
#SBATCH -n 1
#SBATCH -N 1
#SBATCH -c 1 # processors
#SBATCH --mem=20G # memory
#SBATCH -t 0-12:00:00 # Run time. here for 12 hours
#SBATCH -o %x_%j.out
#SBATCH -e %x_%j.err

# get info required to establish tunnel
XDG_RUNTIME_DIR=""
node=$(hostname -s)
user=$(whoami)
cluster="xanadu-submit-int"
port=$(shuf -i 2222-22222 -n 1)
CondaEnvironment=${1}

# print tunneling instructions to jupyter_tunnel_info.txt

echo -e "

###   Command to create ssh tunnel:

ssh -N -f -L ${port}:${node}:${port} ${user}@${cluster}.cam.uchc.edu

###    Use a Browser on your local machine to go to:

localhost:${port}  (prefix w/ https:// if using password)

" >jupyter_tunnel_info.txt

# load modules or conda environments here
source ~/.bashrc   # when using a local conda install, this is required to add conda to your path
conda activate ${CondaEnvironment}  # loading conda environment containing jupyter and relevant software

# Run Jupyter
jupyter-notebook --no-browser --port=${port} --ip=${node}

NOTES ON RESOURCE USAGE

  • TIME LIMITS #SBATCH -t 0-12:00:00 This will run the notebook server for 12 hours. Please do not start Jupyter servers and leave them idle. Not setting a time limit will cause the server to run for the full 3 week job limit on the cluster. Running a server constantly that you pop in and use periodically over several days or weeks is NOT an appropriate use of this shared resource. Please contact us if you are feeling like this is the only way to accomplish what you need to do and we will figure something out.
  • CPU AND MEMORY Select appropriate CPU and memory as with any other batch job. Unless you are doing parallel computations in your notebook, 1-2 CPUs should be plenty. If you don't know how much memory your notebook will need, start with 10-20G and increase it in future sessions if it's not enough for your application.

Once you've prepared your conda environment and have the batch script ready, cd to the directory you want to be the notebook's working directory. Then submit the script with an sbatch command.

cd /path/to/my/workdir
sbatch ~/path/to/Xanadu_jupyNBook.sh CondaEnvironment

While submitting the script change the CondaEnvironment to appropriate environment name.

Now monitor the job with squeue and wait till the job starts running i.e, STATUS changes to R. Barring any errors (check the .err and .out files), you have started the Jupyter Notebook server.

Step 2: Establish an SSH tunnel.

Now we need to establish and SSH tunnel from your local machine to the node running the Jupyter Notebook server. Open the jupyter_tunnel_info.txt file created in the directory you launched the batch script. You will see something similar to the following message.

Command to create ssh tunnel:

ssh -N -f -L 8889:xanadu-06:8889 [email protected]

Use a Browser on your local machine to go to:

localhost:8889  (prefix w/ https:// if using password)

NOTE: The port numbers (in this case 8889) are randomly generated. Yours will be different.

Open a new terminal. Do not connect it to the cluster. Copy the line ssh -N -f -L 8889:xanadu-06:8889 [email protected] and enter it in your new terminal window. Provide password if requested.

Step 3: Open the notebook in your web browser.

Copy localhost:8889 from jupyter_tunnel_info.txt. Now open a browser on your local machine, paste this address, and hit enter. Jupyter will ask for a token. Back on the cluster, it will have printed a message to jupyNBook_<SLURMJOBID>.err:

[I 16:06:27.217 NotebookApp] Serving notebooks from local directory: /home/FCAM/nreid/jupyter
[I 16:06:27.217 NotebookApp] Jupyter Notebook 6.5.4 is running at:
[I 16:06:27.217 NotebookApp] http://xanadu-34:4796/?token=fd09dd01dea103a4ce85f1b9a96ec1e97bd4bc8e70f11127
[I 16:06:27.217 NotebookApp]  or http://127.0.0.1:4796/?token=fd09dd01dea103a4ce85f1b9a96ec1e97bd4bc8e70f11127

Either copy the text string after token= and enter it in the box, or just copy one of the URLs and put it in your web browser. You should now be able to open a new or existing notebook and start working.

Step 4: Ending your session.

  • Step 1: Save the python notebook.

  • Step 2: Close the notebook tab on the browser.

  • Step 3: Close the ssh tunnel with the commands below (replace 8889 with your port number)

    lsof -i tcp:8889

    this will list process IDs (PID). Kill process

    kill -9 <PID>

  • Step 4: Logon to Xanadu cluster and cancel the job with scancel

Please be considerate and do not spin multiple sessions.

Source: https://researchcomputing.princeton.edu/support/knowledge-base/jupyter

Clone this wiki locally