Attempt at Uniformity task optimization.

[luissantosHCIT]

Is your feature request related to a problem? Please describe.
Out of all of the tasks, the Uniformity task appears to run the slowest. It feels significant as an observer.

To Reproduce
Steps to reproduce the behavior:

1. Run the hazen test battery and you will notice that the Uniformity tests take longest.
2. Run hazen against phantom data and you might notice this test takes relatively longer than the others to complete.

Expected behavior
I expect each task to take no longer than the average time of the other tasks. 2-3 seconds are passable. I have done complex image processing (registration, gaussian filtering, subtractions, etc...) on much bigger data than the single phantom slices these tasks typically require, so I have a feeling there might be areas for improvement here.

Screenshots

For comparison, here is the profile for the acr_geometric_accuracy task, which seems to consume about 3 seconds with a third of the time spent on saving the images.

Desktop (please complete the following information):

• OS: GNU/Linux [Ubuntu]
• Python version [3.10]
• Hazen version [1.3.2]
• Deployment [terminal]

Additional context
I am going to profile and attempt to optimize the code if possible.
My idea is to make sure these tasks perform in relatively comparable ways.

[luissantosHCIT]

New Changes

Removed the two stages performing the ROI look up using nested loops (O[n2]). In exchange, we now have a large ROI and a small ROI pair of steps. My code now looks for the min and max pixels within the large ROI using masked arrays. Then, generate a pair of small ROIs centered around these pixels using masked arrays. It then computes the mean value in those small ROIs.

The original approach spent too much time scanning the region of interest trying to determine where to place the small ROIs.
The reason I can get away with my approach is that we get most of the performance lost back and the results mirror more closely what a human is expected to get if following the ACR guidelines.

Something that I hope to bring in my changes is a better approximation of what is expected out of users per the guideline, which should make results easier to validate and reason about.

New Profile

Validating Results

Reference (ACR Large Phantom)

Report

Performance Improvement

My code takes this task from 12.5 seconds to 1.70 seconds of execution. This is 7 times faster and makes this task match the performance of the other tasks. This combined with parallelization should guarantee that all ACR tasks can be achieved within a couple of seconds, thus making it possible to minimize the time spent by technologists getting these results ready.

[luissantosHCIT]

Now, I can start yielding some of the performance back in exchange for correctness.

Per ACR guidelines, we really want the mean intensity value.
Computing the center and then mean around it is not quiet accurate and indeed yields errors because that approach is not based on the true means. The previous slower approach was correct in that regard.

My next step is to generate a 2D kernel that is as close to circular as possible (covers as much of the area an ROI made by a human would) and compute a mean large roi from which we can identify a min and max point using np.argmin and np.argmax. Those points should be reflecting accurate 1cm2 means and thus be true local minima/maxima in the pixel space.

A convolution here might be optimal and maintain performance around < 3 s of execution time.

[luissantosHCIT]

Ok, so I implemented the smoothing step and recalculation based on it.
Doing a convolution here was extremely fast and exposed a fault with the original Uniformity implementation results.

Reference ACR Example

Reference Dataset

For this report, I focused on the GE Medium phantom dataset available in the project since it adds a notch at the top of the large ROI and thus impose extra challenges in image processing (can become a local minima region that can bias results towards a smaller uniformity).

Before

Main Logic for Small ROIs

Report

Below is the results from running the original implementation of the hazen's uniformity task.

Benchmark

After

Approach

The new approach is to take the original image and convolve it with a circular kernel with the exact dimensions as the small ROI.
The idea is to generate a weighted average at each pixel and leverage the vectorization of convolution for a fast generation of mean intensities.

Main Logic for Small ROIs

Kernel

The kernel is generated based on the in-plane voxel/pixel resolution and the small ROI radius

Intermediate Visualization

Report

Benchmark

Items Addressed

1. Speed optimization.
2. Accuracy of mean value calculation.
3. Improved approximation of human behavior and ACR guideline spirit.
4. Corrected inaccuracy with reporting (There was an offset being applied to the large ROI drawn in the report which did not match the true center of the ROI used for computation).

Summary

I have a solution to the Uniformity task performance issues. It revealed more critical issues that put into question the validity of results. I took a convolution approach to identify the mean min and max intensity regions for the calculation of PIU. My current solution appears to lower the execution time from 122s to 2.53s for the GE medium phantom. That is a 48x improvement in this case. Mileage may vary since I have seen the original implementation run 10x faster against acquisitions with half the matrix size. My approach seems to keep execution under the target 3s. More importantly, I believe that the new results are much closer to the true homogeneity of the acquisition, which is very homogeneous in this case. A validation report vs 500 phantom acquisitions is in my todo.

[luissantosHCIT]

There was a mistake in my logic. Addressing the mistake in mean() calculation yielded results that were comparable to the original implementation (although there was still some differences in results). It looks more stable now after running some tests. We still keep our speed and other bug fixes wins, but now it puts into question the homogeneity of my in-house data. More validation needs to happen.

[luissantosHCIT]

I am ready to push changes to this project.

[luissantosHCIT]

I did further cleaning of my code and this is the final write up on my changes.

Reference ACR Example

Reference Dataset

For this report, I focused on the GE Medium phantom dataset available in the project since it adds a notch at the top of the large ROI and thus impose extra challenges in image processing (can become a local minima region that can bias results towards a smaller uniformity).

Before

Main Logic for Small ROIs

Report

Below is the results from running the original implementation of the hazen's uniformity task.

Benchmark

After

Approach

The new approach is to take the original image and convolve it with a circular kernel with the exact dimensions as the small ROI.
The idea is to generate a weighted average at each pixel and leverage the vectorization of convolution for a fast generation of mean intensities.

Main Logic for Small ROIs

Kernel

The kernel is generated based on the in-plane voxel/pixel resolution and the small ROI radius.
In reality we generate something like the kernel screenshot below which encodes the contributions from each element to the mean value of the ROI space at each center point.

Intermediate Visualization

Report

Benchmark

Items Addressed

1. Speed optimization.
2. Accuracy of mean value calculation.
3. Corrected inaccuracy with reporting (There was an offset being applied to the large ROI drawn in the report which did not match the true center of the ROI used for computation).
4. Path name sanitization

Summary

I have a solution to the Uniformity task performance issues. It revealed more critical issues that put into question the validity of results. I took a convolution approach to identify the mean min and max intensity regions for the calculation of PIU. My current solution appears to lower the execution time from 122s to 2.53s for the GE medium phantom. That is a 48x improvement in this case. Mileage may vary since I have seen the original implementation run 10x faster against acquisitions with half the matrix size. My approach seems to keep execution under the target 3s. More importantly, the main adjustment to the accuracy of the uniformity calculation comes from better center detection of the large ROI (See #452 ) A validation report vs 500 phantom acquisitions is in my todo.