SegCLR

Segmentation-Guided Contrastive Learning of Representations (SegCLR)

Segmentation-guided Contrastive Learning of Representations (SegCLR) is a method for learning rich embedding representations of cellular morphology and ultrastructure. See the blog post and the updated preprint fully describing the method for more details. The embeddings for two large-scale cortical datasets, one from human temporal cortex (explore in Neuroglancer) and one from mouse visual cortex (explore), are publicly released on Google Cloud Storage.

SegCLR code release and demo notebooks

Open-source release of SegCLR code to the connectomics repo is in progress. Note that the API should currently be considered unstable.

Colab notebooks demonstrating how to use the code are available here:

• Run a pretrained SegCLR model to predict embeddings for a data cutout. This notebook shows how to instantiate a SegCLR model, load weights from a pretrained model, and run inference.
• Access SegCLR embeddings from public CSV ZIP releases for h01 (human cortex) and MICrONS (mouse cortex). This notebook shows how to read the data remotely and parse it. It also demonstrates how to run dimensionality reduction to inspect embedding clusters (as in paper figure 4).
• Run a pretrained SegCLR subcompartment classifier. This notebook shows how to load a pretrained subcompartment classifier model and run it on embeddings for a test cell (as in paper figure 2).
• Train a cell type classifier with out-of-distribution (OOD) detection. This notebook shows how to load ground truth cell type labels for the mouse cortex dataset, and train a lightweight cell type classifier on top of SegCLR embeddings from scratch (as in paper figure 3). In this demo, the classifier is trained on glia cell types, while the neuron types are only used for evaluation, so the classifier must learn to reject the OOD neuron types. We do this by training a classifier with calibrated uncertainty estimates via SNGP (SNGP paper, as in paper figure 5).

Further demo notebooks will be provided as additional code is released.