Class activation map on Unet

[JumpLK]

Hi, I'm using a U-net to do some segmentation on 3D volumes and I would like to see where it focuses.

I tried using GradCAM but it tries to allocate thousands of Gb of memory and I would like to know if it's because of the size of my data or because I made a mistake.

here is the relevant part of my code :
cam = monai.visualize.GradCAM(nn_module=model, target_layers="model.0.conv")
img = batch_data["image"] #img.shape = (4,1,96,96,96)
cam_res = cam(x=img, class_idx=None)

[rijobro]

Could you make a minimum working example? You should be able to initialise the UNet without training and use torch.rand to create your image.

[joho84]

Hi @rijobro ,
I'm facing the same problem.
I used the U-Net tutorial code as minimum working example (but had the same problem on my other net trained with lightning):

I created a simple U-Net:

device = torch.device("cpu")

model = UNet(
    dimensions=3,
    in_channels=1,
    out_channels=2,
    channels=(16, 32, 64, 128, 256),
    strides=(2, 2, 2, 2),
    num_res_units=2,
    norm=Norm.BATCH,
).to(device)

loss_function = DiceLoss(to_onehot_y=True, softmax=True)

optimizer = torch.optim.Adam(model.parameters(), 1e-4)

And then tried to adapt the code from the interpretability notebook:
cam = GradCAM(nn_module = model, target_layers="model.2.0.conv")

ran = torch.rand(1,1,96,96,96)

result = cam(x=ran, class_idx = None)
RuntimeError: [enforce fail at CPUAllocator.cpp:65] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 3131031158784 bytes. Error code 12 (Cannot allocate memory)

What am I doing wrong?
Thank you in advance for your help!

[rijobro]

What output do you get from:

from monai.config import print_debug_info
print_debug_info()

[joho84]

Hi rijobro,
here is the output of the debug_info:

================================
Printing MONAI config...
================================
MONAI version: 0.5.2
Numpy version: 1.18.5
Pytorch version: 1.7.1
MONAI flags: HAS_EXT = False, USE_COMPILED = False
MONAI rev id: feb3a334b7bbf302b13a6da80e0b022a4cf75a4e

Optional dependencies:
Pytorch Ignite version: 0.4.4
Nibabel version: 3.2.1
scikit-image version: 0.18.1
Pillow version: 8.1.0
Tensorboard version: 2.3.0
gdown version: 3.13.0
TorchVision version: 0.8.2
ITK version: 5.1.2
tqdm version: 4.55.2
lmdb version: 1.0.0
psutil version: 5.8.0

For details about installing the optional dependencies, please visit:
    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies


================================
Printing system config...
================================
System: Linux
Linux version: Ubuntu 18.04.5 LTS
Platform: Linux-3.10.0-957.5.1.el7.x86_64-x86_64-with-glibc2.10
Processor: x86_64
Machine: x86_64
Python version: 3.8.5
Process name: python
Command: ['/opt/conda/bin/python', '-m', 'ipykernel_launcher', '-f', '/root/.local/share/jupyter/runtime/kernel-5f124e32-68c5-42c6-9741-aefc9cfdf37f.json']
Open files: [popenfile(path='/root/.ipython/profile_default/history.sqlite', fd=35, position=12288, mode='r+', flags=688130), popenfile(path='/root/.ipython/profile_default/history.sqlite', fd=36, position=212992, mode='r+', flags=688130)]
Num physical CPUs: 40
Num logical CPUs: 40
Num usable CPUs: 1
CPU usage (%): [33.2, 2.7, 1.2, 1.3, 1.7, 4.1, 6.2, 9.3, 1.1, 1.2, 1.1, 1.1, 5.8, 1.1, 12.5, 1.2, 5.3, 0.4, 0.3, 0.1, 4.4, 4.4, 7.6, 0.9, 0.9, 0.9, 0.9, 1.6, 1.2, 1.1, 1.2, 1.2, 1.3, 1.2, 19.8, 10.9, 7.2, 1.9, 1.5, 1.9]
CPU freq. (MHz): 3142
Load avg. in last 1, 5, 15 mins (%): [9.2, 5.1, 4.5]
Disk usage (%): 12.6
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 377.4
Available memory (GB): 218.2
Used memory (GB): 154.9

================================
Printing GPU config...
================================
Num GPUs: 8
Has CUDA: True
CUDA version: 10.1
cuDNN enabled: True
cuDNN version: 7603
Current device: 0
Library compiled for CUDA architectures: ['sm_37', 'sm_50', 'sm_60', 'sm_61', 'sm_70', 'sm_75', 'compute_37']
GPU 0 Name: Tesla V100-SXM2-16GB
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 80
GPU 0 Total memory (GB): 15.8
GPU 0 CUDA capability (maj.min): 7.0
GPU 1 Name: Tesla V100-SXM2-16GB
GPU 1 Is integrated: False
GPU 1 Is multi GPU board: False
GPU 1 Multi processor count: 80
GPU 1 Total memory (GB): 15.8
GPU 1 CUDA capability (maj.min): 7.0
GPU 2 Name: Tesla V100-SXM2-16GB
GPU 2 Is integrated: False
GPU 2 Is multi GPU board: False
GPU 2 Multi processor count: 80
GPU 2 Total memory (GB): 15.8
GPU 2 CUDA capability (maj.min): 7.0
GPU 3 Name: Tesla V100-SXM2-16GB
GPU 3 Is integrated: False
GPU 3 Is multi GPU board: False
GPU 3 Multi processor count: 80
GPU 3 Total memory (GB): 15.8
GPU 3 CUDA capability (maj.min): 7.0
GPU 4 Name: Tesla V100-SXM2-16GB
GPU 4 Is integrated: False
GPU 4 Is multi GPU board: False
GPU 4 Multi processor count: 80
GPU 4 Total memory (GB): 15.8
GPU 4 CUDA capability (maj.min): 7.0
GPU 5 Name: Tesla V100-SXM2-16GB
GPU 5 Is integrated: False
GPU 5 Is multi GPU board: False
GPU 5 Multi processor count: 80
GPU 5 Total memory (GB): 15.8
GPU 5 CUDA capability (maj.min): 7.0
GPU 6 Name: Tesla V100-SXM2-16GB
GPU 6 Is integrated: False
GPU 6 Is multi GPU board: False
GPU 6 Multi processor count: 80
GPU 6 Total memory (GB): 15.8
GPU 6 CUDA capability (maj.min): 7.0
GPU 7 Name: Tesla V100-SXM2-16GB
GPU 7 Is integrated: False
GPU 7 Is multi GPU board: False
GPU 7 Multi processor count: 80
GPU 7 Total memory (GB): 15.8
GPU 7 CUDA capability (maj.min): 7.0

[wyli]

normally the GradCAM is used in the context of classification models... now it's unclear to me how we adapt it to the unet segmentation, perhaps we should focus on the encoder downsampling block only?

[joho84]

Hi @wyli, hi @rijobro,
thanks for your help so far!
As far as I understood @rijobro managed to get the GradCAM running for the dummy U-Net example above. Or did I misinterpret that?
Do you have another hint for me what I could try? I still get the same memory allocation error.
Thank you!

[rijobro]

The code successfully executed on my MacOS, but I can't get it to successfully run anywhere else (Google colab, remote linux both on CPU and GPU, etc.). As @wyli mentioned, I'm not entirely sure what the expected output of the grad cam might look like for a segmentation task.

[joho84]

Hi @rijobro @wyli
sorry for the long delay!
I was hoping to achieve something similar to this: https://arxiv.org/abs/2002.11434
Any chance this will be implemented in monai in the future? Or could you maybe point me in the right direction to adopt this in the existing monai code?
Thank you very much!