Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi project classification #107

Merged
merged 81 commits into from
Jul 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
eabf5f8
Rename method getDistanceMatrix to getAverageDistanceMatrix
stefanhahmann May 28, 2024
e8d4e84
Add a unit test to getAverageDistanceMatrix
stefanhahmann May 28, 2024
65e45df
Make ClassificationUtils getAverageDistanceMatrix use List of Lists i…
stefanhahmann May 30, 2024
0b6eced
Add some logging to getAverageDistanceMatrix method
stefanhahmann Jun 5, 2024
26c66f9
Make methods getDistanceMatrix and getAverageDistanceMatrix more generic
stefanhahmann May 30, 2024
ecd5854
Extract 2 new methods in the ClassifyLineagesCommand: updateParams() …
stefanhahmann May 29, 2024
ac621e5
Add some missing final modifiers to method params in ClassifyLineages…
stefanhahmann May 29, 2024
34b6460
Add the possibility to specify a list of projects to ClassifyLineages…
stefanhahmann May 29, 2024
84ac31c
Move time measurements logs for computing distance matrix from method…
stefanhahmann May 30, 2024
a783e50
Add param ProjectModel to getRoots() method in ClassifyLineagesContro…
stefanhahmann May 30, 2024
add2091
Rename model and projectModel to referenceModel and referenceProjectM…
stefanhahmann May 30, 2024
114a200
Pass distance matrix as param to method classifyLineageTrees()
stefanhahmann May 30, 2024
f191373
Add a list of external project files to ClassifyLineagesController an…
stefanhahmann May 30, 2024
9aad78a
Add getName() method BranchSpotTree which returns the first the label…
stefanhahmann May 30, 2024
f58ccd9
Add new class ProjectAccessor that encapsulates loading project model…
stefanhahmann Jun 11, 2024
7e5e35b
Add new method getRootsAndDistanceMatrix()
stefanhahmann May 30, 2024
62c8f0b
Remove lengthy documentation from ClassifyLineagesCommand and instead…
stefanhahmann Jun 5, 2024
559cfe6
Change layout of ClassifyLineagesCommand
stefanhahmann Jun 5, 2024
fa24f07
Make number of spots the default crop criterion
stefanhahmann Jun 5, 2024
64e9e8b
Init projects choice in ClassifyLineagesCommand
stefanhahmann Jun 5, 2024
eca7fb3
Add Model as param to method applyClassification()
stefanhahmann Jun 6, 2024
9a7fe32
Remove init project choice, since it interferes with adding further p…
stefanhahmann Jun 6, 2024
192499b
Add checks for crop start/end for all projects when crop criterion sp…
stefanhahmann Jun 6, 2024
84acfb3
Change check for max classes from not more than roots to not more tha…
stefanhahmann Jun 11, 2024
e26f16e
Add commands of ClassifyLineagesPlugin to command descriptions
stefanhahmann Jun 13, 2024
6f7ea36
Change Classification variable from class to local variable
stefanhahmann Jun 17, 2024
ed7a527
Add time measurement to ProjectAccessor
stefanhahmann Jun 18, 2024
a822d30
Add a method to demo utils that wraps BDV data as model and create ma…
stefanhahmann Jun 21, 2024
d9a8a4e
Add a new Service MastodonProjectService
stefanhahmann Jun 21, 2024
301c30c
Add a new class Notification that contains some shortcuts to notify u…
stefanhahmann Jun 21, 2024
8a4f2fc
Remove input branch duration from the ClassifyLineagesCommand
stefanhahmann Jun 21, 2024
e13d0ff
Retrieve the MastodonProjectService from the context and init the Cla…
stefanhahmann Jun 21, 2024
f91f3d8
Replace ProjectAccessor by using MastodonProjectService
stefanhahmann Jun 21, 2024
89d0dbd
Add possibility to specify whether generated tag sets should also be …
stefanhahmann Jun 21, 2024
748749f
Add detection of external projects that could not be opened to the UI
stefanhahmann Jun 21, 2024
3a25307
Let method createTagSet return the name of the created tag set
stefanhahmann Jun 21, 2024
6235cdf
Only apply read lock to graph, when writing the tag set to it
stefanhahmann Jun 21, 2024
28383c4
Add some debug message
stefanhahmann Jun 21, 2024
7611154
Change tag set name
stefanhahmann Jun 21, 2024
0475721
Change ClassifyLineagesCommand from DynamicCommand to Interactive com…
stefanhahmann Jun 21, 2024
12fefb0
Remove button create Tag set from ClassifyLineagesCommand
stefanhahmann Jun 21, 2024
740c315
Show parameter feedback as a list
stefanhahmann Jun 21, 2024
c1ca424
Reduce complexity of setExternalProjects method by introducing some s…
stefanhahmann Jun 21, 2024
f7e57dd
Improve debug log message
stefanhahmann Jun 21, 2024
c507874
Create new method in DemoUtils to save an app model to a temp file
stefanhahmann Jun 24, 2024
2fe6553
Set setVisualisationParams() to true to increase test coverage
stefanhahmann Jun 24, 2024
67802fe
Add new unit test testCreateTagSetWithExternalProjects()
stefanhahmann Jun 24, 2024
aa9e167
Remove unrequired method getProjectModel from ClassifyLineagesController
stefanhahmann Jun 24, 2024
4893ddc
Add new unit test for ClassifyLineagesController setExternalProjects(…
stefanhahmann Jun 24, 2024
392c69d
Change debug log message
stefanhahmann Jun 24, 2024
ad8f8ac
Increase xmx for build job to 3g
stefanhahmann Jun 24, 2024
352ffb8
Avoid concurrent modification exception when removing entries from ex…
stefanhahmann Jun 24, 2024
c9e0319
Add start end end timepoints to BranchSpotTree class
stefanhahmann Jun 27, 2024
3f995a0
Remove unused initializer from ClassifyLineagesCommand
stefanhahmann Jun 27, 2024
6c28996
Use List instead of Set to store objectClassifications in the Classif…
stefanhahmann Jun 27, 2024
99ecee1
Correctly apply classification to external projects
stefanhahmann Jun 27, 2024
cdb73a5
Remove redundant handling for the case of resetting the external proj…
stefanhahmann Jun 27, 2024
edf25be
Make search for root branch spots in project model consistent with br…
stefanhahmann Jun 27, 2024
ea29aad
Extend unit test for classification with external projects by checkin…
stefanhahmann Jun 27, 2024
c2a7415
Update scijava-common version to 2.99.1-SNAPSHOT
stefanhahmann Jul 18, 2024
3faf6b1
Override scijava-ui-swing version to 1.0.2
stefanhahmann Jul 18, 2024
217689b
Update heap size for tests to 3gb
stefanhahmann Jul 18, 2024
34887fc
Set log level to INFO in general and to DEBUG for org.mastodon.mamut
stefanhahmann Jul 18, 2024
381dd82
Declare prefService and projectService as a private field with @Param…
stefanhahmann Jul 18, 2024
cceb23d
Extract new method classifyUsingExternalProjects()
stefanhahmann Jul 18, 2024
01eda6a
Rename method setVisualisationParams to setShowDendrogram
stefanhahmann Jul 18, 2024
d26d428
Add hierarchical to the class javadoc of the classification class
stefanhahmann Jul 18, 2024
8c0b872
Refactor variable from expectedClassCount to expectedSpotsPerClass
stefanhahmann Jul 19, 2024
e51b3a8
Remove generation of test dataset from ClassifyLineagesControllerTest
stefanhahmann Jul 19, 2024
ee9476a
Refactor method openProjectSessionsCount to activeSessions()
stefanhahmann Jul 22, 2024
193db41
Introduce new class ExternalProjects
stefanhahmann Jul 22, 2024
cbcafba
Make constructor of ProjectSession package protected
stefanhahmann Jul 23, 2024
83e35ca
Change methods runClassification and classifyExternalProjects in Clas…
stefanhahmann Jul 23, 2024
eefe2fe
Replace ProjectSession and MastodonProjectService by simpler Classifi…
stefanhahmann Jul 24, 2024
ff76f59
Remove Dendrogram test from ClassifyLineagesControllerTest
stefanhahmann Jul 24, 2024
de2a8dd
Add name of current project to ClassifyLineagesCommand
stefanhahmann Jul 24, 2024
180c3f6
Change label of further projects parameter
stefanhahmann Jul 24, 2024
52a4292
Fix messages for crop start and end
stefanhahmann Jul 24, 2024
4a3a94b
Ensure that only distinct projects are part of the analysis
stefanhahmann Jul 24, 2024
96b09e5
Ensure that current project is not added again as further project
stefanhahmann Jul 24, 2024
cdaf1f7
Fix unit test bug
stefanhahmann Jul 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 22 additions & 13 deletions doc/classification/readme.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
## Lineage Tree Classification

* This plugin is capable of grouping similar lineage trees together.
* This command is capable of grouping similar lineage trees together.
* The linage classification operates on Mastodon's branch graph.
* Lineage trees are considered similar, if they share a similar structure and thus represent a similar cell division
pattern. The structure of a lineage tree is represented by the tree topology.
This tree topology consists of the actual branching pattern and the cell lifetimes,
i.e. the time points between two subsequent cell divisions.
* The similarity of a pair of lineage trees is computed based on the Zhang edit distance for unordered
trees ([Zhang, K. Algorithmica 15, 205–222, 1996](https://doi.org/10.1007/BF01975866)). This method captures the cost
of the transformation of one tree into the other.
* The cost function applied for the tree edit distance uses the attribute branch spot duration, which is computed as a
difference of time points between to subsequent divisions reflecting the start and the end of a spot's lifetime.
* Thus, the linage classification operates on Mastodon's branch graph.
* The Zhang unordered edit distance allows the following edit operations. The edit operations are defined in a way that
they satisfy the constraints elaborated in section 3.1 ("Constrained Edit Distance Mappings") of the paper:
they satisfy the constraints elaborated in section 3.1 ("Constrained Edit Distance Mappings") of the
paper: [Zhang, K. Algorithmica 15, 205–222, 1996](https://doi.org/10.1007/BF01975866)

```
Note: The prefix T may represent a node or a complete subtree. Nodes without this prefix are just nodes.
Expand Down Expand Up @@ -98,26 +101,32 @@ Tree2
```

* Edit distance of 69, because:

* one node has a difference of 1
* two nodes have a difference of 24 each
* two extra nodes are added with a weight of 10 each
* ![zhang_example.gif](zhang_example.gif)
* The tree edit distances are computed between all possible combinations of lineage trees leading to a two-dimensional
matrix. The values in this matrix are considered to reflect similarities of lineage trees. Low tree edit distances
represent a high similarity between a discrete pair of lineage trees.
* This similarity matrix is then used to perform
an [agglomerative hierarchical clustering](https://en.wikipedia.org/wiki/Hierarchical_clustering) into a specifiable

* The similarity measure uses the attribute cell lifetime, which is computed as a difference of time points between to
subsequent divisions. There are multiple ways to compute the similarity measure between two lineage trees (cf. below).
* The similarities are computed between all possible combinations of lineage trees leading to a two-dimensional
similarity matrix. The values in this matrix are considered to reflect similarities of lineage trees. Low tree edit
distances represent a high similarity between a discrete pair of lineage trees. This matrix is then used to perform
an [Agglomerative Hierarchical Clustering](https://en.wikipedia.org/wiki/Hierarchical_clustering) into a specifiable
number of classes.
* For the clustering three
different [linkage methods](https://en.wikipedia.org/wiki/Hierarchical_clustering#Cluster_Linkage) can be chosen.

### Parameters

* Crop criterion:
* Time point (default)
* Number of spots
* Number of spots (default)
* Time point
* Crop start
* At which number of spots or time point the analysis should start
* Crop end
* At which number of spots or time point the analysis should end
* Number of classes
* The number of classes the lineage trees should be grouped into
* Must not be greater than the number of lineage trees
* Minimum number of divisions
* The minimum number of divisions a lineage tree must have to be considered in the classification
Expand Down
14 changes: 8 additions & 6 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,10 @@

<mastodon.version>1.0.0-beta-30</mastodon.version>
<mastodon.group>org.mastodon</mastodon.group>
<!-- with pom-scijava >= 37.1.0, this can be removed again -->
<scijava-common.version>2.97.1</scijava-common.version>
<!-- when a pom-scijava exists that references the scijava-common 2.99.1 release, this can be removed again -->
<scijava-common.version>2.99.1-SNAPSHOT</scijava-common.version>
<!-- with pom-scijava >= 38.0.2, this can be removed again -->
<scijava-ui-swing.version>1.0.2</scijava-ui-swing.version>

<releaseProfiles>sign,deploy-to-scijava</releaseProfiles>

Expand Down Expand Up @@ -210,13 +212,13 @@
</activation>
<build>
<plugins>
<!-- Configure the maven-surefire-plugin to use a heap size of 2gb while running tests. -->
<!-- Configure the maven-surefire-plugin to use a heap size of 3gb while running tests. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<argLine>-Xmx2g</argLine>
<argLine>-Xmx3g</argLine>
</configuration>
</plugin>
</plugins>
Expand All @@ -226,13 +228,13 @@
<id>coverage</id>
<build>
<plugins>
<!-- Configure the maven-surefire-plugin to use a heap size of 2gb while running tests for jacoco coverage analysis. -->
<!-- Configure the maven-surefire-plugin to use a heap size of 3gb while running tests for jacoco coverage analysis. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<argLine>@{argLine} -Xmx2g</argLine>
<argLine>@{argLine} -Xmx3g</argLine>
</configuration>
</plugin>
<plugin>
Expand Down
Loading
Loading