Skip to content

Generate consensus 3D cells segmentations by combining 2D cell segmentations from any combination of xy, xz, yz views, compatible with outputs of any 2D segmentation method.

License

Notifications You must be signed in to change notification settings

DanuserLab/u-segment3D

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

u-Segment3D ("Turn 2D segmentations into 3D")

A Python library for generating 3D consensus cell segmentation from 2D segmented stacks

May 22, 2024

u-Segment3D is a Python library to merge 2D slice-by-slice segmented 3D volumes in x-y, x-z, or y-z views into a single consensus 3D segmentation. u-Segment3D can use the probabilities and gradients directly predicted by popular existing Neural Network segmentation models such as Cellpose (Direct Method) or operates on the provided 2D segmentations directly (Indirect Method). u-Segment3D does not require all 3 views, any number can be used, nor do each view need be generated by the same algorithm. u-Segment3D also provides a number of postprocessing algorithms to recover missing features after obtaining the 3D segmentation. Therefore it can also be used as a postprocessing module for 3D segmentations in general. Despite the name, the postprocessing and segmentation functions in u-Segment3D have 2D equivalents which can be used to enhance 2D segmentation too. Please see the tutorials.

The primary motivation in developing u-Segment3D is to take advantage and leverage existing state-of-the-art 2D segmentation models for 3D segmentation without further data training.

It is associated with the BioArxiv paper, A general algorithm for consensus 3D cell segmentation from 2D segmented stacks, bioRxiv, 2024, written by Felix Y. Zhou, Clarence Yapp, Zhiguo Shang, Md Torikul Islam, Benjamin Nanes, Edward Jenkins, Gabriel M. Gihana, Bo-Jui Chang, Andrew Weems, Michael Dustin, Sean Morrison, Reto Fiolka, Kevin Dean, Andrew Jamieson, Peter K. Sorger and Gaudenz Danuser.

The current library is still a work-in-progress. It is fully functional but documentation still require work and more tutorials are upcoming.

A universal 2D-to-3D segmentation algorithm

Dependencies

u-Segment3D has a number of dependencies detailed in the requirements.txt. GPU dependencies are based on installing Cellpose to use as the default 2D segmentation method in u-Segment3D.

Installation

u-Segment3D can be installed for all three major OS from PyPI. We recommend installing to a new conda environment:

conda create -n u_Segment3D_env python=3.9
pip install u-Segment3D

u-Segment3D can also be installed by git cloning the repository and running pip in the cloned folder with python>=3.9. We have developed on Python==3.9 on Red Hat Enterprise Linux Server 7.9. pyproject.toml configures the individual dependencies for each OS.

Linux

We suggest first creating a new conda environment for install and use conda to install cudatoolkit and cudnn first:

conda create -n u_Segment3D_env python=3.9 cudatoolkit=11.8.* cudnn==8.* -c anaconda
conda activate u_Segment3D_env
pip install .

If on a HPC cluster, depending on the way it is setup, you may need to module load the cuda corresponding to the install, module load cuda118/toolkit/11.8.0 prior to activating the conda environment to use the installed cupy library functions for image resizing. Otherwise, u-Segment3D will fall back to a pytorch version of image resizing. Don't worry, this still uses gpu, but is slower than using cupy.

Errors we have encountered:

  1. When running, we get an icu error, AttributeError: module 'icu' has no attribute 'Locale'. Fix this by installing pyicu. This can be done using pip (requires gcc compiler, gcc>=6.3.0) or alternatively installation of a precompiled version with conda from the conda-forge channel
pip install pyicu # Requires compilation, check that gcc>=6.3.0 to be available on your linux distribution
conda install -n u_Segment3D_env pyicu -c conda-forge # tries to install a precompiled version. 
  1. pytorch does not have gpu available. Try installing a previous version for the respective cudatoolkit version. Alternatively, try installing a precompiled version using anaconda from the conda-forge channel. Refer to instructions for Windows below.

  2. You see a print out along the lines of 'cuda not available. Falling back to pytorch'. This doesn't mean you don't have cuda. It means cuda drivers is likely installed in non-standard location or not installed properly so that not all cupy function is not available. Consequently u-Segment3D is using CUDA (if available) via pytorch.

Windows

conda create -n u_Segment3D_env python=3.9 cudatoolkit=11.8.* cudnn==8.* -c anaconda
conda activate u_Segment3D_env
pip install .

On machine with NVIDIA graphics card, check to see if torch has been correctly installed with GPU by importing the library in Python.

import torch
print(torch.cuda.is_available())

If True, nothing needs to be done, you are set. If False, you need to install an earlier version of pytorch compiled for cuda11.8

pip install torch==2.0.1 --index-url https://download.pytorch.org/whl/cu118

See https://pytorch.org/get-started/previous-versions/ to check out other pytorch versions.

Alternatively, you may try to install pytorch through anaconda from the conda-forge channel:

conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

Errors we have encountered:

  1. Installing scikit-fmm fails, because it cannot be compiled. To resolve, install precompiled binary version through conda-forge before running pip install .
conda install -n u_Segment3D_env scikit-fmm -c conda-forge

MacOS

NVIDIA CUDA is not available for MacOS with Apple Silicon chips. Therefore don't install any of these dependencies.

conda create -n u_Segment3D_env python=3.9
conda activate u_Segment3D_env
pip install .

Getting Started

The simplest way to get started is to check out the included notebook tutorials which aims to showcase various use cases of u-Segment3D for 3D segmentation and how parameters may be tuned and algorithms adapted. A pictorial diagram is provided to show how tutorial scripts relate:

The easiest way to use u-Segment3D is the indirect method. If you have 2D slice-by-slice instance segmentation masks of cells in xy, xz, and yz views, then you can translate these into one 3D instance segmentation mask with a few lines of code:

import segment3D.parameters as uSegment3D_params
import segment3D.usegment3d as uSegment3D

# instantiate default parameters
indirect_aggregation_params = uSegment3D_params.get_2D_to_3D_aggregation_params()

# integrate labels_xy, labels_xz, labels_yz into one single 3D segmentation. Give a single-channel volume image, img we define its xy view as img, its xz view as img.transpose(1,2,0) and its yz view as img.transpose(2,0,1)
segmentation3D, (probability3D, gradients3D) = uSegment3D.aggregate_2D_to_3D_segmentation_indirect_method(segmentations=[labels_xy,
                                                                                                                        labels_xz,
                                                                                                                        labels_yz], 
                                                                                                                  img_xy_shape = labels_xy.shape, 
                                                                                                                precomputed_binary=None, 
                                                                                                                params=indirect_aggregation_params,
                                                                                                                savefolder=None,
                                                                                                                basename=None)

Example Data

Please download the zipped folder containing example data from the link. The following examples assume you have unzipped the data to the example_data/ directory of this repository, and is running the examples after installation from their current location in the repository. Please adjust filepaths accordingly, if otherwise.

Example Scripts

Examples for 3D Single Cell segmentation with morphological motifs:

  1. tutorials/single_cells/segment_blebs_3D.py : "demonstrates direct method, how to run on downsampled version to get base segmentation, and additionally recover cellular protrusions at original image resolution for efficiency"
  2. tutorials/single_cells/segment_lamellipodia_3D.py : "demonstrate direct method and basic usage"
  3. tutorials/single_cells/segment_filopodia_3D.py : "demonstrates direct method and basic usage, and using a median filter to pre-smooth salt-and-pepper like noise which may affect guided filter postprocessing"
  4. tutorials/single_cells/segment_ruffles_3D.py : "demonstrates direct method and basic usage, and extra use of padding and median filter, if single cells are cropped too close to the border"

Examples for 3D Multiple Cell segmentation:

  1. tutorials/multi_cells/segment_Tcells_3D.py : "demonstrates the same direct method with Cellpose backend for single cells can be applied to resolve multiple cells and their subcellular protrusions"
  2. tutorials/multi_cells/segment_zebrafish_macrophages_3D.py : "demonstrates more extensive preprocessing prior to running Cellpose 2D and then running u-Segment3D aggregation"
  3. tutorials/multi_cells/segment_zebrafish_macrophages_3D_with_independent_fg_binary.py : "demonstrates how to 'paint' an external binary with cellpose gradients within the gradient descent aggregation in u-Segment3D. Regions not covered by gradients are segmented as a single cell or fragment"

Examples for 3D Tissue segmentation:

  1. tutorials/tissue/segment_organoid_3D.py : "demonstrate direct method and basic usage, with minimal processing, how to resize images according to voxel resolution and eval performance"
  2. tutorials/tissue/segment_ovules_3D.py : "demonstrate direct method and basic usage, with minimal processing, how to resize images according to voxel resolution and eval performance"

Examples for 2D segmentation for anisotropic: These examples demonstrate the indirect method, whereby we first generate the 2D segmentation masks and also showcases how to aggregate on only one view i.e. 'xy' to segment 3D.

  1. tutorials/anisotropic/segment_single-cell_tracking_challenge_2D_stacks.py : "demo segmentation of MDA231 cells from the single cell tracking challenge, using an independent Otsu thresholded foreground binary to mask and get cell shape boundary"
  2. tutorials/anisotropic/segment_epithelial_organoid_2D_stacks.py : "demo segmentation of an air-liquid interface grown skin cell culture that is thin"

Overview of Library Features

For those interested, u-Segment3D is a library organized by function similar to scipy / numpy / opencv allowing re-usable functions that can be used to build-up complex processing pipelines. A brief summary is below.

Module Functionality
file_io.py Functions for reading and writing images and intermediate outputs including pickled objects
filters.py Functions for 'filtering', 'enhancing' and 'cleaning-up' images and segmentations e.g. label diffusion, keep largest component
flows.py Functions for computing different 2D flows used for 2D-to-3D segmentation
gpu.py Functions for GPU-based processing, primarily array resizing to speed up I/O operations if GPU is available and CUDA is installed.
meshtools.py Functions for basic mesh processing, e.g. reading/writing meshes, extracting surface mesh from binary segmentation and computing mesh properties
metrics.py Functions for evaluating segmentation performance, such as average precision
parameters.py Functions for generating the default parameters for key u-Segment3D steps
plotting.py Functions for plotting, such as coloring the gradient and segmentations
segmentation.py Functions for running Cellpose with automatic parameter tuning
watershed.py Functions for doing suppressed gradient descent in 2D/3D
usegment3d.py Wrapper Functions that align with the conceptual steps in the paper - most often will use functions from here

Questions and Issues

Feel free to open a GitHub issue or email me at [email protected].

Danuser Lab Links

Danuser Lab Website

Software Links

About

Generate consensus 3D cells segmentations by combining 2D cell segmentations from any combination of xy, xz, yz views, compatible with outputs of any 2D segmentation method.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published