End-to-End Text Generation from 3D Image Volumes

[AceMcAwesome77]

Hi, I think it would be great to have a model framework that uses transformers for generation of text using 3D image volume data as the input. There have been a number of projects recently synthesizing clinical report text from 2D XR. Expanding this to 3D CT volumes would be an important step on the path to automated generation of radiology reports for all modalities. Are there any components of the MONAI framework that could be adapted or combined with transformer-based NLP outputs to create this kind of model? Thanks!

[AceMcAwesome77]

Hi, just following up on this at the end of the year. Have there been any additions to the MONAI framework regarding end-to-end text generation from 3D image input? Thanks!