HIT LAB 진행 현황 및 결과

3/3/2022, 9:53:00 AM
🔗 github



Implementation the DeepIGeoS Paper
:page_facing_up: DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation (2018)
additional page : Notion page (in Korean :kr:)


Please check environments and requirements before you start. If required, we recommend you to either upgrade versions or install them for smooth running.

︎ Environments

Ubuntu 16.04Python 3.7.11

︎ Requirements

dotmap GeodisTK opencv-python tensorboard torch torchio torchvision tqdm PyQt5
Plain Text


Download the BraTS 2021 dataset form web BraTS 2021 using load_datasets.sh.
$ bash load_datasets.sh
Plain Text


︎ P-Net

$ python train_pnet.py -c configs/config_pnet.json
Plain Text

︎ R-Net

$ python train_rnet.py -c configs/config_rnet.json
Plain Text

︎ Tensorboard

$ tensorboard --logdir experiments/logs/
Plain Text


︎ Simple QT Application

To operate DeepIGeos with simple mouse click interaction, We create a QT based application. You can just run DeepIGeoS with the main code titled 'main_deepigeos.py' as shown below.
$ python main_deepigeos.py
Plain Text

:dna: Results

︎ with the Simulated Interactions

After the simulated user interaction, the result follows rules below :
Simulations were generated on three slices with the largest mis-segments in each axes; sagittal, coronal, and axial.
To find mis-segmented regions, the automatic segmentations by P-net are compared with the ground truth.
Then the user interactions on the each mis-segmented region are simulated by n pixels, which is randomly sampled, in the region. (Suppose the size of one connected under-segmented or over-segmentedregion is Nm, we set n for that region to 0 if Nm < 30 and [Nm/100] otherwise)
Yellow border : Ground Truth Mask
Green area : P-Net Prediction Mask
Red area : R-Net Refinement Mask

︎ with the User Interaction

Results after the required user interactions are below :
Click LOAD IMG button to load image
Click P-NET button to operate automatic segmentation
Check the automatic segmentation results
Point where to refine results by mouse click
Circle : Under-segmented region
Square : Over-segmented region
Click R-NET button to operate refinement
Check refined segmentation results

Background of DeepIGeoS

For the implementation of the DeepIGeoS paper, all steps, we understood and performed are described in the following subsections.

︎ Abstract

Consider a deep CNN-based interactive framework for the 2D and 3D medical image segmentation
Present a new way to combine user interactions with CNNs based on geodesic distance maps
Propose a resolution-preserving CNN structure which leads to a more detailed segmentation result compared with traditional CNNs with resolution loss
Extend the current RNN-based CRFs for segmentation so that the back-propagatable CRFs can use user interactions as hard constraints and all the parameters of potential functions can be trained in an end-to-end way.

︎ Architecture

Two stage pipeline : P-Net(obtains automatically initial segmentation) + R-Net(refines initial segmentation w/ small # of user interactions that we encode as geodesic distance maps)
P-Net : use CNN to obtain an initial automatic segmentation
R-Net: refine the segmentation by taking as input the original image, the initial segmentation and geodesic distance maps based on foreground/background user interactions.

︎ CRF-Net

The CRF-Net(f) is connected to P-Net and also the CRF-Net(fu) is connected to R-Net.
CRF-Net(f): extend CRF based on RNN so that the pairwise potentials can be freeform functions.
CRF-Net(fu): integrate user interactions in our CRF-Net(f) in the interactive refinement context.
But here CRF-Nets are not implemented for simplicity

︎ Geodesic Distance Maps

The interactions with the same label are converted into a distance map.
The euclidean distance treats each direction equally and it does not take the image context into account.
In contrast, the geodesic distance helps to better differentiate neighboring pixels with different appearances, and improves label consistency in homogeneous regions.

︎ BraTS Dataset

We only consider T2-FLAIR(panel C) images in the BraTS 2021 and segment the whole tumor.
The BraTS dataset describes a retrospective collection of brain tumor mpMRI scans acquired from multiple different institutions under standard clinical conditions, but with different equipment and imaging protocols, resulting in a vastly heterogeneous image quality reflecting diverse clinical practice across different institutions. Inclusion criteria comprised pathologically confirmed diagnosis and available MGMT promoter methylation status. These data have been updated, since BraTS 2020, increasing the total number of cases from 660 to 2,000.