notebooks/3D_annotator.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Annotation tool to create/correct 3D training labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import holoviews as hv\n",
    "from improc.io import parse_collection, DCAccessor\n",
    "DCAccessor.register()\n",
    "from holoviews import opts\n",
    "\n",
    "from inter_view.dashboards import OrthoAnnotationDashBoard\n",
    "\n",
    "hv.extension('bokeh', logo=False, width=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Annotations\n",
    "\n",
    "Nuclei and lumen annotations are needed to train the segmentation models.\n",
    "\n",
    "1. First place labels to be corrected in a correction folder, .e.g `nuclei_corrections`, `lumen_corrections`\n",
    "1. Once annotated/corrected, move the files in the corresponding annot subfolder so that it can be used by the training task.\n",
    "\n",
    "**Nuclei annotations**\n",
    "- If no nuclei segmentation exist yet, use the partial annotations generated from the tracked seed as starting point (e.g. nuclei_weak_annot)\n",
    "    - start the `NucleiSegmentationTrainingTask` to trigger the generation of the partial labels\n",
    "- After a first prediction has been generated, use the segmentation viewer to pick and automatically copy files for correction\n",
    "\n",
    "\n",
    "**Lumen annotation**\n",
    "- If no lumen segmentation exist yet, set `n_blank_annot` in the config below to generate empty label from n randomly picked images\n",
    "- After a first prediction has been generated, use the segmentation viewer to pick and automatically copy files for correction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# correction_layer = 'nuclei_corrections'\n",
    "correction_layer = 'lumen_corrections'\n",
    "n_blank_annot=0\n",
    "\n",
    "nuclei_ch = 'Channel1-Deconv'\n",
    "cell_ch = 'Channel0-Deconv'\n",
    "spacing = (2,0.26,0.26)\n",
    "\n",
    "\n",
    "basedir = '../data'\n",
    "data_pattern = '{dataset_id}/{layer}/{fname_head}T{time:04d}.{ext}'\n",
    "index = ['dataset_id', 'layer', 'time']\n",
    "\n",
    "\n",
    "channel_config = {cell_ch:{'cmap':'red'},#, 'intensity_bounds':(100,60000), 'slider_limits':(0,6000)},\n",
    "                  nuclei_ch:{'cmap':'gray'},#, 'intensity_bounds':(1400,20000), 'slider_limits':(0,6000)},\n",
    "                  correction_layer:{'cmap':'glasbey_hv_16bit', 'raster_aggregator':'first', 'intensity_bounds':(-2**15,2**15-1), 'slider_limits':(-2**15,2**15-1), 'bitdepth':'int16', 'opacity':0.2}}\n",
    "\n",
    "def set_inactive_tool(plot, element):\n",
    "    plot.state.toolbar.active_inspect = None\n",
    "\n",
    "opts.defaults(opts.Image('channel.{}'.format(correction_layer), clipping_colors={'min': (0, 0, 0, 0)}, clim=(0,2**16-1), tools=['hover'], hooks=[set_inactive_tool]))\n",
    "\n",
    "opts.defaults(opts.Image('channel', frame_width=700))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# parse files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def initialize_blank_annot(df, ref_layer, correction_layer, n_samples, seed=741):\n",
    "    '''\n",
    "    Note: this does not check if matching corrected annoations already exist.\n",
    "    '''\n",
    "    \n",
    "    subdf = df.query('layer==\"{}\"'.format(ref_layer)).sample(n_samples, random_state=seed)\n",
    "    for _,row in subdf.reset_index().iterrows():\n",
    "        row_out = row.copy()\n",
    "        row_out['layer'] = correction_layer\n",
    "        img = row.dc.read()[0]\n",
    "        row_out.dc.write(np.zeros(img.shape, dtype=np.int16)-1, compress=9, exist_ok=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = parse_collection(os.path.join(basedir, data_pattern), index)\n",
    "df = df.dc[:,[nuclei_ch, cell_ch, correction_layer]]\n",
    "\n",
    "if n_blank_annot > 0:\n",
    "    initialize_blank_annot(df, nuclei_ch, correction_layer, n_samples=n_blank_annot)\n",
    "    df = parse_collection(os.path.join(basedir, data_pattern), index)\n",
    "    df = df.dc[:,[nuclei_ch, cell_ch, correction_layer]]\n",
    "\n",
    "# filter samples needs to be corrected\n",
    "df = df.groupby(['dataset_id','time'],).filter(lambda x: correction_layer in x.reset_index().layer.tolist())\n",
    "\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# interactive dashboard\n",
    "To start drawing select the 'Freehand Drawing Tool' in the toolbar on the right of the image.\n",
    "\n",
    "- drawing label:\n",
    "    - -1: un-annotated (does not contribute to the training loss)\n",
    "    - 0: background\n",
    "    - for nuclei:\n",
    "        - [1...] instance labels. last one in the list is always unused\n",
    "    - for lumen:\n",
    "        - 1: lumen\n",
    "        - 2: epithelium\n",
    "- on click\n",
    "    - \\-: does nothing\n",
    "    - pick label (pipette): sets label at the clicked position as drawing label\n",
    "    - fill label (pot filling): fill the label at the clicked label with the current drawing labels (discontinuous)\n",
    "- lock\n",
    "    - background: prevent from drawing over the existing background\n",
    "    - foreground: prevents from drawing over the existing labels\n",
    "- draw in 3D: draw with thickness in the 3rd dimension. best used to draw an initial \"ball\" at the center of each nuclei\n",
    "- save: saves the current annotation. The current annotation is also automatically saved when loading a new image\n",
    "- discard changes: Cancels all changes made since the last save (i.e. not a granual ctrl+z!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "db = OrthoAnnotationDashBoard(df=df,\n",
    "                           multi_select_levels=['layer'],\n",
    "                           channel_config=channel_config,\n",
    "                           composite_channels=[nuclei_ch, cell_ch],\n",
    "                           overlay_channels=[correction_layer],\n",
    "                           spacing=spacing,\n",
    "                           annot_channel=correction_layer)\n",
    "\n",
    "db.panel().servable()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:tf2.3]",
   "language": "python",
   "name": "conda-env-tf2.3-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}