肝脏分割 Detection-aided liver lesion segmentation using deep learning

A fully automatic technique for segmenting the liver and localizing its unhealthy
tissues is a convenient tool in order to diagnose hepatic diseases and assess the
response to the according treatments. In this work we propose a method to segment
the liver and its lesions from Computed Tomography (CT) scans using
Convolutional Neural Networks (CNNs), that have proven good results in a variety of
computer vision tasks, including medical imaging.
The network that segments the lesions consists of a cascaded architecture, which first focuses on the region of the liver in order to segment the lesions on it.  
Moreover, we train a detector to localize the lesions, and mask the results of the segmentation network with the  positive detections. The segmentation architecture is based on DRIU [8], a Fully Convolutional Network (FCN) with side outputs that work on feature maps of different resolutions, to finally benefit from the multi-scale information learned by different stages of the network. The main contribution of this work is the use of a detector to localize the lesions, which we show to be beneficial to remove false positives triggered by the segmentation network. Source code and models are available at https://imatge-upc.github.io/liverseg-2017-nipsws/.

一种用于分割肝脏并将其不健康定位的全自动技术组织是诊断肝脏疾病和评估肝脏疾病的便利工具回应相应的治疗。在这项工作中，我们提出了一种分割方法计算机断层扫描（CT）扫描使用的肝脏及其病变卷积神经网络（CNN），已经证明在各种各样的计算机视觉任务表现出了好的结果，包括医学成像。细分病变的网络由一个级联结构组成，该结构首先聚焦于肝脏区域，以分割病变。此外，我们训练检测器以定位病变，并用正检测掩盖分割网络的结果。分割架构基于DRIU [8]，一个完全卷积网络（FCN），带有侧面输出，能够处理不同分辨率的特征地图，最终受益于网络不同阶段学到的多尺度信息。这项工作的主要贡献是使用检测器来定位病变，我们证明这对于消除由分割网络触发的误报有利。源代码和模型可在https://imatge-upc.github.io/liverseg-2017-nipsws/获得

Segmenting the liver and its lesions on medical images helps oncologists to accurately diagnose liver cancer, as well as to assess the treatment response of patients. Typically, doctors rely on manual segmentation techniques in order to interpret the Contrast Tomography (CT) and Magnetic Resonance Imaging (MRI) images. Automatic tools that are not as subjective and time-consuming have been
widely studied in the recent years. Liver lesion segmentation is a challenging task due to the low contrast between liver, lesions, and also nearby organs. Other additional difficulties are the lesion size variability and the noise in CT scans. Building a robust system that is able to beat these difficulties is still an open problem. Recently, methods based on deep Convolutional Neural Networks (CNNs)
have demonstrated to be robust to these challenges, and have achieved the state of the art at this task [3, 5, 1].
In this paper we adapt DRIU [8] for the task of segmenting both the liver and its lesions from CT scans. DRIU is a Fully Convolutional Network (FCN) that has side outputs with supervision at different convolutional stages. This architecture has proven to be successful for the medical task of segmenting the blood vessels and optical disk of eye fundus images, as well as for video object segmentation [2]
in generic videos. The core of our network for lesion and liver segmentation consists
in using the strength of a segmentation network plus a detection network to localize the lesions. For training all the networks we used the Liver Tumor Segmentation (LiTS) dataset, which is composed of 131 CT scans for training and 70 for testing

在医学图像上分割肝脏及其病变有助于肿瘤学家准确诊断肝癌，并评估患者的治疗反应。通常，医生依靠手动分割技术来解释对比层析成像（CT）和磁共振成像（MRI）图像。自动工具不是主观和耗时的
在近几年被广泛研究。由于肝脏，病灶和附近器官之间的低对比度，肝脏病灶分割是一项具有挑战性的任务。其他更多的困难是CT扫描中的病灶大小变异性和噪声。建立一个能够克服这些困难的强大系统仍然是一个悬而未决的问题。最近，基于深度卷积神经网络（CNN）的方法，
已经证明对这些挑战是有力的，并且已经在这项任务中取得了最先进的成果[3,5,1]。
在本文中，我们调整DRIU [8]来完成从CT扫描中分割肝脏及其病变的任务。 DRIU是一个完全卷积网络（FCN），它具有在不同卷积阶段进行监督的侧面输出。这种架构已被证明是成功的分割眼底血管和光盘图像的医学任务，以及视频对象分割[2]

在通用视频中。我们的网络用于病变和肝脏分割的核心在于在使用分割网络的强度加上检测网络来定位病变。为了训练所有网络，我们使用了肝肿瘤分割（LiTS）数据集，该数据集由131个用于训练的CT扫描和70个用于测试的组成

2 Detection-aided liver and its lesions segmentation using deep learning

Our pipeline is illustrated in Figure 1. It is a cascaded architecture, which first segments the liver to focus on the region of interest in order to segment the lesion. In this section, we first present the baseline segmentation architecture, and then the different features implemented to adapt it to the liver and lesion segmentation task.

我们的管道如图1所示。它是一个级联的架构，它首先将肝脏分段以关注感兴趣的区域，以分割病灶。 在本节中，我们首先介绍基线分割体系结构，然后介绍实现的不同功能以使其适应肝脏和病灶分割任务。

Figure 1: Architecture for the detection-aided liver and its lesions segmentation. The first stage consists in segmenting the liver. Once we have the liver prediction, we place a 3D bounding box around the liver, and the different slices cropped by this bounding box are segmented by the lesion segmentation network. The detection network operates on patches sampled around the liver. Finally,we only keep the positive detections that agree with the segmentation of the lesion.

图1：检测辅助肝脏及其病变分割的体系结构。 第一阶段包括分割肝脏。 一旦我们进行了肝脏预测，我们在肝脏周围放置一个三维边界框，由这个边界框剪切的不同切片由病灶分割网络分割。 检测网络运行在肝脏周围采样的斑点上。 最后，我们只保留与病灶分割一致的正面检测结果。

2.1 Segmentation network

The lesion and liver segmentation networks are both based on DRIU [8], an architecture for retinal image segmentation that segments the blood vessels and optic disc on fundus images. The architecture uses VGG-16 [10] as the base network, removing the last fully connected layers,
so that the network consists of convolutional, ReLU, and max-pooling layers. The base network is pre-trained on Imagenet [9] and consists of a set of convolutional stages, each of them working at the same feature map resolution, separated by the pooling layers. As the network goes deeper, the information is  coarser and the learned features are more related to semantics. On the other hand, at the shallower
feature maps that work at a higher resolution, filters capture more local information. To take advantage of the information learned at feature maps that work at different resolutions, DRIU uses several side outputs with supervision. A side output is a set of convolutional layers that are connected at the end of a specific convolutional stage from the base network. Each of these side outputs specializes on
different types of features, depending on the resolution at the connection point. The feature maps produced by each side output are resized and linearly combined to output the final result.

病灶和肝脏分割网络均基于DRIU [8]，这是一种视网膜图像分割体系结构，用于分割眼底图像上的血管和视盘。该架构使用VGG-16 [10]作为基础网络，去除最后全连接层，因此网络由卷积，ReLU和最大池层组成。基础网络在Imagenet上进行了预先训练[9]，由一组卷积阶段组成，每个卷积阶段以相同的特征映射分辨率工作，并由池层分隔。随着网络越来越深，信息越来越粗糙，学到的功能更多地与语义相关。另一方面，在较浅的地方以更高分辨率工作的功能图，过滤器捕获更多本地信息。为了利用在不同分辨率下工作的功能图上学到的信息，DRIU使用多个带有监控的侧面输出。侧面输出是一组卷积层，它们连接在来自基础网络的特定卷积阶段的末尾。每个侧面输出都专注于不同类型的功能，取决于连接点的分辨率。每个边输出产生的特征映射被调整大小并线性组合以输出最终结果。

2.1.1 Loss objective
Regarding the loss objective for training the segmentation network, we worked with a weighted
version of the Binary Cross Entropy (BCE) loss due to the imbalance between positive and negative
lesion pixels in the database. The weighted BCE is defined as:

关于训练分割网络的损失目标，我们使用了由于数据库中的正面和负面病变像素之间的不平衡而导致的二进制交叉熵（BCE）损失的加权版本。 加权BCE定义为：

For training each segmentation network we compute a weighting term for the foreground and another for the background class. The weighting term for the foreground class is computed by summing all the pixels belonging to the foreground divided by total number of pixels. It is important to notice that only the slices that contain the foreground class are considered, as proposed in [4]. The same is

done for the background class. We normalize both terms, and we obtain the weights w and 1 − w to balance the Binary Cross Entropy.

为了训练每个分割网络，我们计算前景的权重项和后台类的权重项。 通过将属于前景的所有像素除以像素总数来计算前景类别的加权项。 需要注意的是，只有包含前景类的切片才被考虑，如[4]中提出的那样。背景类也一样。 我们对两个项进行归一化，并且我们获得权重w和1-w以平衡二元交叉熵。

肝脏的分割使我们能够对感兴趣的区域进行剪切以分割病灶。 尽管如此，因为我们知道病变总是在肝脏内部，所以我们可以进一步从肝脏分割中受益，决定不通过那些被预测为非肝脏的像素向后传播梯度。 这种策略的好处是双重的：（i）网络只是从像素学习实际上可以属于目标类别，并且（ii）正像素和负像素更平衡，因为负像素的数量显着减少。 因此，损失目标的权重项仅考虑属于肝脏类的像素。

2.2

我们观察到，我们的分割网络缺乏捕捉肝脏健康状况的全局视图的能力，这有助于查看是否存在病变，并且因此在许多图像中引发了一些误报。 与分割网络相比，病变检测器获得更全面的肝脏组织健康状况信息，分割网络的最终输出以像素为单位，并且不需要对整个肝脏区域做出全局决策。 我们将使用检测器来定位病变并保留检测器和分割网络都认为不健康的像素。 图2说明了检测器如何执行的一些示例。

为了训练检测器，我们将边界框放在肝脏区域，然后将它们标记为正面或负面。 为了放置边界框的条件是它与肝脏重叠至少25％。 我们使用50×50像素的窗口，考虑如果框内至少有50个像素的病灶，则进行积极的观察。 步幅为50个像素，并且15个像素的边距被添加到窗口的每一侧以提供额外的上下文，使得每个窗口最终尺寸为80×80。

我们使用的模型是在ImageNet上预训练的ResNet-50 [6]，去除其最后一个分类层并用单个神经元代替它以检测健康/不健康的肝脏组织。 我们使用64个同等平衡的正面和负面修补程序的批量大小。 我们使用翻转和旋转将数据增加8倍。

3

作为最终的后处理步骤，我们添加一个3D完全连接的条件随机场。 3D-CRF是应用于结构化预测的统计建模技术。 考虑到所有输入，CRF模拟输出预测的条件分布。 给定最终标签，给出由分段网络输出的软预测作为最大后验推断
在密集的CRF中。 该模型考虑了空间相干性以及输入体积强度值方面的外观。 3D-CRF是完全连接的，所以它在图像中的所有像素对上建立成对电势，使相似像素之间的标签一致性最大化。 我们使用[3]的实现，它使用[7]的3D-CRF公式。

4 结论

在这项工作中，我们提出了一种使用两个级联分割网络和一个病变检测器来分割肝脏及其病变的算法。我们已经研究了如何利用所提供数据的特征，使用网络输入处的体积信息，并利用肝脏分割预测来分割病变。
我们对肝脏病变进行分割的方法只是从肝脏像素中学习，这表明将算法学习的样本限制在相关样本或困难样本中，有利于解决问题。这种策略对于使用“注意”机制是熟悉的，因为有一个选择的位置（肝脏）从中学习，这改善了学习表示
病变。这种策略的另一个优点是，将学习限制在某些样本中可以利用正像素和负像素之间的不平衡。
这项工作最重要的结论是细分网络可以实现的精细定位和探测器学习的一般化之间的权衡。由于分割网络的输出是按像素方式的，因此它倾向于触发假正像素，因为没有对更多全局决策施加限制。另一方面，检测器决定是否有完整的补丁

健康与否，对病变的确切形状没有限制。使用这两种技术分析输入图像可以为LiTS质询数据库提供更好的整体结果。我们认为这对医学图像分割流水线来说是一个有趣的方向，通常处理非常小的结构，这是一种有助于定位目标区域的检测辅助分割流水线。