Feb 18 reading records: MRF& brain segmentation

http://deanhan.com/2018/04/22/MRF/
This is one person’s website, and he made a very clear explantation about MRF. Besides, the application is about image denoising which image is black or white.

• book–deep learning for medical image analysis----chapter 12

Convolutional Neural Networks for Robust and Real-Time 2-D/3-D Registration
读后感：
感觉和读deep learning 那本书很像，其实内容没有很细的东西，只是如果遇到了你感兴趣的东西，你把paper download下来再去仔细阅读，这样才会有长进，单独看，也没有什么基础，不会有很多进益，这样子。

打算暂时舍弃这本书，先读paper，读一些paper，了解了一些东西之后再来看书。

• reading papers

1. Deep Neural Networks for Anatomical Brain Segmentation.

http://openaccess.thecvf.com/content_cvpr_workshops_2015/W01/papers/Brebisson_Deep_Neural_Networks_2015_CVPR_paper.pdf

This paper proposed a deep artificial neural network for the automated segmentation of the entire brain.

automated segmentation of the entire brain: classifying all its voxels into their corresponding protocol region.

Our object: we consider segmentation of the whole brain (cortical and sub-cortical areas) into a large number N of anatomical regions, where N is defined by the segmentation protocol.

Previous methods:

Multi-atlas based method, which highly reply on the registration step, and in which the atlas are nonlinearly registered to the query image.

limits:

1. rely on the assumption that brains are similar enough to be accurately mapped from one to another. (if there is a neurodegenerative disorder?
2. regions whose boundaries are clearly identifiable by a contrast in intensity but arbitrarily defined by the segmentation protocol are likely to be less accurately registered.

Our method

CNN, for medical imaging segmentation problems.

Our method tackle the segmentation of the whole 3D brain and introduce multi-scale input features to enforce the spatial consistency of the segmentation.

Architecture of the network

1. Input features

• our aim: classify each voxel into its corresponding anatomical region.
• each voxel be described by an input vector, which is our input of our neural network.
• two types of inputs were incorporated in this work in order to ensure both local precision and global spatial consistency

1.1 Features to ensure local precision

which is ensured by following features:

• a 3D patch of size a$\times$ a$\times$a centred on the voxel is used to capture local information at a high level of detail.

• three 2D orthogonal patches of size b$\times$b (each extracted from the sagittal, coronal and transverse planes respectively), also centred on the voxel, are added with the purpose of capturing a slightly broader but still local context around the voxel of interest.

• these three 2D patches can be seen as a trade-off between 3D and 2D:
  three 2D patches: they capture 3D information but require a significantly smaller amount of memory for storage than 3D patch.

1.2 Features to ensure global spatial consistency

• anatomical regions consistently preserve the same relative positions in all the subjects.

• Including global information is therefore likely to yield additional improvements

An obvious strategy would be simply increase the size of 2D or 3D patches introduced earlier so as to span larger portions of the image and cover more distant anatomy. —but this will generate very high-dimensional inputs requiring large memory.

• Downscale a large 2D orthogonal patches.

Instead, we extract large 2D orthogonal patches that we downscale by a factor s, in neural network, this operator is equivalent to a s$\times$s mean-pooling with stride s.
So the downscaled patch still captures as large portions of the MRI as the original patches but with lower resolution.

The patch of image we select indicates the voxel intensities, which will be regarded as input of neural network and then extract the features and finally get the segmentation.

In addition to the voxel intensities, the coordinates of each voxel in 3D space are also expected to be very informative for anatomical segmentation purpose.

• Using relative distances for each voxel to each one of the N centroids as additional inputs of the network

The distances to centroids are also invariant to brain scaling upon scaling centroids and the image coordinates by the average distance between two centroids D,

This enforce the average distance between two centroids to be the same for all the brains.

2. Deep Neural Network

• convolutional layers

• max-pooling layers
• fully connected layers
• Activation Functions

weighting sharing

3. Training Algorithm

4. Estimation of the Centroid

when we consider a new brain, we can not compute directly the centroid. Thus we uses two neural networks in sequence.

our conclusion

We proposed two types of input features and a corresponding architecture to precisely delineate the boundaries of the regions while ensuring global spatial consistency. Due to current memory constraints on single GPU cards, we opted for a multi-scale system with different sizes of intensity patches. The three orthogonal patches significantly outperformed the individual 2D or 3D patches, proving that they are an excellent trade-off to capture 3D information with considerably less memory than a dense 3D patch.

We introduced two other input vectors ensuring the global spatial consistency of the segmentation.

• First, we showed that the network can learn surprisingly well the relative positions of the regions with large raw downscaled 2D patches of voxel intensities.
• Second, we showed that distances to centroids, despite their imprecision, provide robust inputs to efficiently capture the location of the voxel in the brain.

Their robustness is due to their redundancy and their independence to translations and rotations.

In our approach, global consistency was therefore enforced by the inputs of the model without having to resort to any complicated post-processing, such as conditional random fields, which are commonly used in the literature.

We also observed that distances to centroids and downscaled patches contain redundant information and one may consider only one of these two sets of features and obtain almost the same performance.

On the one hand, downscaled patches have the advantage of using the raw averaged intensities, while distances to centroids require a preprocessing step to approximate the centroids by using two networks.

On the other hand, downscaled patches take a significantly larger amount of memory than the 134 distances to centroids.