Gradcam Visualisation

[Meddebma]

Dear All,

I am trying to classify two diseases of the brain, the training went very well (accuracy 0.83), and I wanted to use gradcam to find out which were the regions of interest that classifying. In the tutorials you provide, Gradcam and activation map are always displayed beside of the original image. Is there a way to transpose the gradcam with the original image? and maybe export it as a gif where each slice is showing the anatomy and the superposed gradcam results?

Thanks!

[Nic-Ma]

Hi @Meddebma ,

Thanks for your interest and feedback.
I think you can treat the gradcam as an image, then refer to the visualize tutorial for GIF or blend operations:
https://github.com/Project-MONAI/tutorials/blob/master/modules/transform_visualization.ipynb

Thanks.

[Meddebma]

Thanks @rijobro
That is true.. when I put cam_result[None] I get:
print(img.shape)
print(cam_result.shape)
torch.Size([1, 1, 96, 96, 96])
torch.Size([1, 1, 1, 96, 96, 96])

how can I best change the dimensions so that it matches not only for one slice but also for the whole image?

[rijobro]

I'm lost... if cam_result[None] has the shape torch.Size([1, 1, 1, 96, 96, 96]), then just use cam_result as this will have the shape torch.Size([1, 1, 96, 96, 96]). This matches your input image and so you shouldn't have any problems. Remember that wherever you slice img, you'll also have to slice cam_result the same way.

[Meddebma]

Actually the problem is that cam_result has the shape torch.Size([1, 1, 96, 96]), so maybe that is why it works for only one slice (see picture). But as soon I want to implement for blendimage it doesnt work. Maybe is the problem in this line?
ret = blend_images(image=img[:, :, :, :, depth_slice], label=cam_result[:, :, :, depth_slice], alpha=0.5, cmap="hsv", rescale_arrays=False)
print(ret.shape)
https://github.com/Meddebma/pyradiomics/blob/master/Classification_HBI.ipynb

[rijobro]

Every time I look at your notebook it's changed, I'm not sure how useful I can be.

You've successfully visualised the image, gradcam and occlusion sensitivity. If you do this: cam_result = cam_result[..., depth_slice] and end up with a 2d image, then that implies that your cam_result was previously 3d (96, 96, 96 spatially). I'm not sure why you slice your cam_result straight away, but only slice your img for visualisation (in the for loop). This means that outside your for loop they are going to have different dimensions.

Again, make sure your image dimensions match. Then treat all images the same -- whatever you do to one, you do to the others.

[Meddebma]

Oh sorry, I was trying to implement your recommandations that is why I was changing the notebook
Thank you very much @rijobro , it worked well with the cam_result!
appreciate your help :)