Add MONAILabel HoVerNet tutorial notebook (Project-MONAI#1317)

Pathology classification segmentation tutorial per PRD request.

---------

Signed-off-by: tangy5 <[email protected]>

monailabel/README.md

@@ -28,6 +28,7 @@ Each tutorial provides MONAI Label plugin installation steps associated with the
 - **Pathology App**:
   - Viewer: [QuPath](https://qupath.github.io/) | Datastore: Local | Task: Segmentation
     - [MONAILabel: Nuclei Segmentation with QuPath](monailabel_pathology_nuclei_segmentation_QuPath.ipynb) Nuclei segmentation with QuPath setup and Nuclick models.
+    - [MONAILabel: HoVerNet Nuclei Classification and Segmentation](monailabel_pathology_HoVerNet_QuPath.ipynb.ipynb) Nuclei classification and segmentation with QuPath.
 - **Endoscopy App**:
   - Viewer: [CVAT](https://github.com/opencv/cvat) | Datastore: Local | Task: Segmentation
     - [MONAILabel: Tooltracking with CVAT](monailabel_endoscopy_cvat_tooltracking.ipynb): Surgical tool segmentation with CVAT/Nuclio setup.

monailabel/monailabel_pathology_HoVerNet_QuPath.ipynb

@@ -0,0 +1,275 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Copyright (c) MONAI Consortium  \n",
+    "Licensed under the Apache License, Version 2.0 (the \"License\");  \n",
+    "you may not use this file except in compliance with the License.  \n",
+    "You may obtain a copy of the License at  \n",
+    "    http://www.apache.org/licenses/LICENSE-2.0  \n",
+    "Unless required by applicable law or agreed to in writing, software  \n",
+    "distributed under the License is distributed on an \"AS IS\" BASIS,  \n",
+    "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  \n",
+    "See the License for the specific language governing permissions and  \n",
+    "limitations under the License. \n",
+    "\n",
+    "# MONAI Label Pathology App - HoVerNet-Nuclei Classification and Segmentation\n",
+    "\n",
+    "***The pathology cell classification and segmentation analysis with MONAI Label***\n",
+    "\n",
+    "In this notebook, we show the application of cell classification and segmentation at the same time with MONAI Label. The nuclei classification and segmentation is predicted via HoVerNet. This tutorial uses QuPath as the client viewer, and we demonstrate the end-to-end workflow with MONAI Label + HoVerNet integration.\n",
+    "\n",
+    "![hovernet1](./figures/monailabel_pathology_HoVerNet_QuPath/hovernet1.png)\n",
+    "\n",
+    "## 1. Prepare MONAI Label"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup environment\n",
+    "\n",
+    "### Prerequisites\n",
+    "- **Install MONAI Label** weekly preview release: "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install monailabel"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1.1 Download pathology app\n",
+    "Choose \"pathology\" as the app"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!monailabel apps --download --name pathology --output apps"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1.2 Download sample data\n",
+    "Download the sample data from digital slide archive with wget and save to the **datasets** folder for further use."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!mkdir datasets\n",
+    "!wget \"https://demo.kitware.com/histomicstk/api/v1/item/5d5c07509114c049342b66f8/download\" -O \"datasets/JP2K-33003-1.svs\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "HoVerNet model is trained with CoNSeP datasets, the original dataset can be downloaded from <https://warwick.ac.uk/fac/cross_fac/tia/data/HoVerNet/>."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1.3 Starting MONAI Label Server\n",
+    "\n",
+    "The command will start MONAI Label server with the pre-trained HoVerNet model. \n",
+    "\n",
+    "Such as\n",
+    "\n",
+    "```bash\n",
+    "monailabel start_server --app <path to pathology app> --studies <path to downloaded datafolder> --conf models hovernet_nuclei\n",
+    "```\n",
+    "In the command, ```--app``` specifies the pathology app path, ```--studies``` specifies the dataset folder path."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!monailabel start_server --app apps/pathology --studies datasets/ --conf models hovernet_nuclei"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Install QuPath and MONAI Label Plugin\n",
+    "\n",
+    "[QuPath](https://qupath.github.io/) is an open software for bioimage analysis. it can be used as an efficient tool for annotation and visualization of whole slide images. \n",
+    "\n",
+    "Note: Nvidia does not maintain the thrid-party visualization tools. Please refer to [installation guide](https://qupath.readthedocs.io/en/stable/docs/intro/installation.html) for details or trouble shooting. \n",
+    "\n",
+    "In this tutorial, we provide minimum steps for QuPath setup. \n",
+    "\n",
+    "**Step 1**: Get QuPath from website.  \n",
+    "\n",
+    "Go to the [QuPath website](https://qupath.github.io/), click \"Download for Linux\" to pull the *tar.xz* file\n",
+    "\n",
+    "Extract the QuPath software by:\n",
+    "\n",
+    "```bash\n",
+    "# E.g., QuPath 0.3.2\n",
+    "tar -xf <QuPath-0.3.2-Linux.tar.xz>\n",
+    "```\n",
+    "\n",
+    "**Step 2**: Start QuPath with excutable launcher file.\n",
+    "\n",
+    "Start QuPath by opening a **Terminal**, Use ```chmod u+x /path/to/QuPath/bin/QuPath``` to make the extracted QuPath launcher executable.\n",
+    "If you suffer from problem, please refer to QuPath [QA](https://github.com/qupath/qupath/releases/tag/v0.3.2).\n",
+    "\n",
+    "Then type or drag the launcher file: ```/path/to/QuPath/bin/QuPath``` to the terminal, and enter to open QuPath UI.\n",
+    "\n",
+    "**Step 3** Install MONAI Label Plugin for QuPath\n",
+    "\n",
+    "Download [qupath-extension-monailabel-0.3.1.jar](https://github.com/Project-MONAI/MONAILabel/releases/download/data/qupath-extension-monailabel-0.3.1.jar)\n",
+    "and **drag the jar** on top of the running QuPath application window (black screen area) to install the extension.\n",
+    "If you have previously installed then make sure to **_remove/uninstall_** the extension before updating.\n",
+    "\n",
+    "<p align = \"center\"><img src=\"./figures/monailabel_pathology_qupath/qupath_plugin.png\" alt=\"drawing\" width=\"700\"/></p>\n",
+    "<p align = \"center\">Drag the downloaded MONAI Label QuPath plugin JAR file into QuPath viewer UI.</p>\n",
+    "\n",
+    "Note: Make sure MONAILabel Server URL (Host+Port) is correct through `Preferences`.\n",
+    "\n"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Nuclei Classification and Segmentation with Interactions\n",
+    "\n",
+    "Below is the instruction of using HoVerNet for labeling nucleis given WSI or other digital pathology images using QuPath. Pre-trained model is provided and loaded by MONAI Label server.\n",
+    "\n",
+    "\n",
+    "### 3.1 Start QuPath, MONAI Label Plugin, and Fetch Next Image\n",
+    "\n",
+    "- On the menu bar, navigate and click MONAI Label. Click *Next Sample/Patch* to load data specified in the start_server command. If the QuPath starts from beginning, the button will automatically load next whole slide image.\n",
+    "\n",
+    "<p align = \"center\"><img src=\"./figures/monailabel_pathology_qupath/qupath2.png\" alt=\"drawing\" width=\"800\"/></p>\n",
+    "\n",
+    " - Users can also use the **Next Sample/Patch** button to select a patch ROI automatically, by selecting **Next Patch**. （Uncheck the **Next Patch** box will load next whole slide image.）\n",
+    "\n",
+    "![hovernet1](./figures/monailabel_pathology_HoVerNet_QuPath/hovernet2.png)\n",
+    "\n",
+    "### 3.2 Select ROI by Rectangle Tool\n",
+    "\n",
+    "First, let's show patch inference with the rectangle ROI tool.\n",
+    "\n",
+    "- Users can automatically select the rectangle ROI by fetching \"Next Patch\", or use the QuPath rectangle tool to pick an ROI.\n",
+    "\n",
+    "![hovernet1](./figures/monailabel_pathology_HoVerNet_QuPath/hovernet3.png)\n",
+    "\n",
+    "### 3.3 Run Nuclei Classication and Segmentation Simultaneously\n",
+    "\n",
+    "- After selecting the rectangle ROI, users can navigate to the MONAI Label plugin and click **Annotations**.\n",
+    "Then select models in the pop-up interface, and click **OK** to run inference. \n",
+    "\n",
+    "![hovernet1](./figures/monailabel_pathology_HoVerNet_QuPath/hovernet4.png)\n",
+    "\n",
+    "MONAI Label server will run auto segmentation with the selected patch data and return the segmentation with polygons.\n",
+    "\n",
+    "### 3.4 Submit ROI Label for Training\n",
+    "\n",
+    "- In the next step, users can edit the predicted nuclei segmentation. Once the annotation is done, users can navigate to MONAI Label plugin and click **Submit Label** to save the ground truth.\n",
+    "\n",
+    "![hovernet1](./figures/monailabel_pathology_HoVerNet_QuPath/hovernet6.png)\n",
+    "\n",
+    "\n",
+    "### 3.5 Run Interactive ROI-based Segmentation\n",
+    "\n",
+    "- Users can also use the interactive ROI tool to get auto-segmentation with one ROI draw:\n",
+    "- Draw any rectangle ROI, and MONAI Label server will return segmentation immediately:\n",
+    "\n",
+    "![hovernet1](./figures/monailabel_pathology_HoVerNet_QuPath/hovernet7.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Training Nuclei Segmentation\n",
+    "\n",
+    "Once users finish annotations on several patch images, MONAI Label provide training option to fine-tune the model.\n",
+    "\n",
+    "- Navigate to MONAI Label plugin and click **Training**\n",
+    "\n",
+    "![hovernet1](./figures/monailabel_pathology_HoVerNet_QuPath/hovernet9.png)\n",
+    "\n",
+    "- In the training option pop-up interface, select **hovernet_nuclei**, select training customized training options. Uncheck **pretrained** to start training model from scratch. Finally, click **OK** to start training in MONAI Label server.\n",
+    "\n",
+    "![hovernet1](./figures/monailabel_pathology_HoVerNet_QuPath/hovernet10.png)\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Repeat the Interactive Labeling Process \n",
+    "\n",
+    "Repeated fetch data and interactive learning process to section 3.1, until a satisfied model is achieved or entire datasets/WSIs are annotated. \n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Conclusion\n",
+    "\n",
+    "This tutorial demonstrates the basic usage of HoVerNet, nuclei classification and segmentation at the same time with MONAI Label. We introduced several inference strategies including ROI tool, interactive ROI tool. We show how to submit annotated labels, then the steps of training are demonstrated.\n",
+    "\n",
+    "In the MONAI Label HoVerNet integration, we highlight the interactive labeling experience with classification and segmentation at the same time from a whole slide image. MONAI Label achieves annotation actively and gradually improves the model's classification and segmentation performance during labeling. \n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.10"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

runner.sh

@@ -58,6 +58,7 @@ doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" monailabel_patholog
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" monailabel_radiology_spleen_segmentation_OHIF.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" monailabel_wholebody_totalSegmentator_3dslicer.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" monailabel_monaibundle_3dslicer_lung_nodule_detection.ipynb)
+doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" monailabel_pathology_HoVerNet_QuPath.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" example_feature.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" ssl_train.ipynb)
 doesnt_contain_max_epochs=("${doesnt_contain_max_epochs[@]}" ssl_finetune.ipynb)
@@ -93,6 +94,7 @@ skip_run_papermill=("${skip_run_papermill[@]}" .*monailabel_pathology_nuclei_seg
 skip_run_papermill=("${skip_run_papermill[@]}" .*monailabel_radiology_spleen_segmentation_OHIF*)
 skip_run_papermill=("${skip_run_papermill[@]}" .*monailabel_wholebody_totalSegmentator_3dslicer*)
 skip_run_papermill=("${skip_run_papermill[@]}" .*monailabel_monaibundle_3dslicer_lung_nodule_detection*)
+skip_run_papermill=("${skip_run_papermill[@]}" .*monailabel_pathology_HoVerNet_QuPath*)
 skip_run_papermill=("${skip_run_papermill[@]}" .*ssl_train*)
 skip_run_papermill=("${skip_run_papermill[@]}" .*ssl_finetune*)
 skip_run_papermill=("${skip_run_papermill[@]}" .*transform_visualization*)  # https://github.com/Project-MONAI/tutorials/issues/1155