-
Notifications
You must be signed in to change notification settings - Fork 65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Drawing on oblique images is really bad #1407
Comments
Leaving this here as notes: According to Sarthak this has to do with how we handle direction changes. For loading into the viewer, typically we have an image with some original direction matrix that has to get collapsed to identity. We also store the "original" direction information so we can convert back. But we aren't doing this for ROIs/masks right now. Also of note: We need to put this "original direction" information in anything that is produced from running on a loaded image.That way, when the derivative image gets saved, it also gets reverted back to the original direction. This needs to happen for any operation performed on loaded images, because directionality changes and registration don't operate on images loaded into the viewer anyway (same is true for all CLI apps). |
More notes: First principle: If a user loads an image into our viewer, regardless of its direction, any derivative output should match up with the same image they started from. That is, the user doesn't care about what image data is being manipulated in our viewer, they care about the data they had when they came in (i.e. on disk). Any output we produce to disk should be in the same space, unless the user is deliberately changing the space. Tools on the GUI could potentially be separated into several groups that need to be handled separately with regard to this issue.
Category 1 doesn't need this correction to be considered -- the algorithm can directly operate on the image. If image obliqueness breaks the algorithm, that is the fault of the algorithm and should be fixed there (or otherwise noted in its documentation that it requires some preprocessing). I might (read: will definitely) need some help/pointers figuring out exactly what we need to do here. |
@sarthakpati Do we need to change ALL image/mask loading/unloading to do the registration process you described? Or does this only need to be done for masks/ROIs? |
I would imagine that this would need to be done for all images and masks. But before looking at the registration solution (which would probably be more stable), I would recommend looking at changing the rotation matrix and directions so that everything gets squished to unity to see if that works, because that would involve the least amount of interpolation... |
Describe the bug
Drawing a mask using the GUI drawing tools on a loaded, oblique image is basically impossible because the drawing doesn't appear where you draw. You might draw on one slice and the mask won't be visible until you scroll through more slices, and depending on what you drew, it might appear in pieces as you scroll through slices.
To Reproduce
Load an oblique DICOM image into the images panel
Go to drawing tab
select rectangle or freehand or whatever
Draw on any slice.
See that the mask doesn't appear where your cursor was.
Scroll around confused and notice that your mask is there, just not where you put it, and sometimes on a diagonal.
Expected behavior
The drawing should appear under my cursor.
Screenshots
If applicable, add screenshots to help explain your problem.
CaPTk Version
1.8.1
Desktop (please complete the following information):
Seen on Windows. Probably present on other platforms.
Additional context
I guess it's a result of our drawing space being mapped differently than the image in the viewer. I'm not currently able to determine what exactly is causing this or if oblique images are loaded properly into the viewer (they can look kind of odd.) A collaborator is asking for some sample NIfTI segmentations on an oblique DICOM image generated by CaPTk -- they previously tried loading up a mask like that in ITK-Snap and it was way off-alignment. It's not really possible for me to check that the mask is being saved correctly, because I can't really draw it decently in the first place.
I guess it would be sufficient to have some way to correct the oblique-ness at load time, have all drawing/rendering take place on the corrected image, then "undo" the correction when saving or running code. Not sure.
Another point: if I run geodesic training segmentation on a mask I drew on an oblique image, when it finally completes, CaPTk will immediately say that it doesn't match the dimensions of the currently loaded image and opens the registration window.
Pinging @sarthakpati @ashishsingh18 for any insight on this.
The text was updated successfully, but these errors were encountered: