Transform Landmark with Image/Segmentation

[kleingeo]

I have images, segmentations and landmarks associated with the segmentations that I want to augment for training purposes. I am currently using the RandAffineGrid to get the grid and resample the images as necessary, but the transformation of the landmark is not working.

rand_affine_grid_transform = m_trans.RandAffineGrid(
    translate_range=(20, 20, 20),
    rotate_range=(np.pi / 36, np.pi / 36, np.pi / 4),
    scale_range=(0.15, 0.15, 0.15))

rand_affine_grid = rand_affine_grid_transform(spatial_size=size)

resample_trans_img = Resample(as_tensor_output=False, padding_mode='zeros')
resample_trans_seg = Resample(as_tensor_output=False, mode='nearest', padding_mode='zeros')
img_resample = resample_trans_img(image, grid=rand_affine_grid)
seg_resample = resample_trans_seg(seg, grid=rand_affine_grid)

# landmark is a np.array([x, y, z, 1]) in voxel space
landmarks_phys = np.matmul(metadata['affine'], landmark)

trans_affine = rand_affine_grid_transform.affine.numpy()

transform_landmark_phys = np.matmul(trans_affine, landmarks_phys)

transform_landmark_vox = np.matmul(np.linalg.inv(metadata['affine']), transform_landmark_phys)

[Nic-Ma]

Hi @kleingeo ,

Thanks for your interest and experiments with MONAI.
May I know what the error is that you that "not working"?

Hi @wyli ,

Does your AffineGrid PR #2969 can cover this case?

Thanks in advance.

[kleingeo]

May I know what the error is that you that "not working"?

When I save the images and opening them (image and segmentation) in ITK-Snap, the landmarks coordinates do not line-up.