具有不确定性感知注意机制的肺结节分割和不确定区域预测| 文献速递-深度学习结合医疗影像疾病诊断与病灶分割

Title

题目

Lung Nodule Segmentation and UncertainRegion Prediction With an Uncertainty-Aware Attention Mechanism

具有不确定性感知注意机制的肺结节分割和不确定区域预测

01

文献速递介绍

肺结节分割在肺癌计算机辅助诊断（CAD）系统中至关重要，提供了关键信息，如结节大小、形状和其他重要医学特征。然而，在深度学习方法的通用训练和测试范式中，每个结节图像数据仅有一个由一名放射科医生勾画的注释掩模。因此，网络每次只能提供结节区域的单一预测。然而，在临床实践中，不同的放射科医生可能会针对肺结节提供各种分割注释，这是由于他们不同的培训和临床经验所致。因此，基于单一注释的传统方法无法反映临床经验的多样性，限制了深度学习方法的应用。

解决影像科医生之间注释变化问题的一个直接方法是为每个肺结节图像结合多个注释。然而，这会带来另一个问题：多个注释不可避免地会带来不确定性和冲突，因为放射科医生可能会以不同方式对同一区域进行注释。为了克服这个问题，Kohl等人在2018年提出了概率U-Net，利用条件变分自动编码器将多个分割变体编码为低维潜在空间。通过从该空间中采样，网络可以影响相应的分割图。基于这项研究，胡等人提出将地面真实不确定性与概率U-Net相结合，可以提高预测不确定性估计、样本准确性和样本多样性。这些方法依赖于潜在空间和在该空间中的随机样本。因此，这些方法只能通过多次预测提供不确定区域。

Abstract

摘要

Radiologists possess diverse training andclinical experiences, leading to variations in the segmentation annotations of lung nodules and resulting insegmentation uncertainty. Conventional methods typically*select a single annotation as the learning target or attemptto learn a latent space comprising multiple annotations.**However, these approaches fail to leverage the valuableinformation inherent in the consensus and disagreementsamong the multiple annotations. In this paper, we propose an Uncertainty-Aware Attention Mechanism (UAAM)that utilizes consensus and disagreements among multipleannotations to facilitate better segmentation. To this end,we introduce the Multi-Confidence Mask (MCM), which com bines a Low-Confidence (LC) Mask and a High-Confidence(HC) Mask. The LC mask indicates regions with low segmen tation confidence, where radiologists may have differentsegmentation choices. Following UAAM, we further designan Uncertainty-Guide Multi-Confidence Segmentation Network (UGMCS-Net), which contains three modules: a Feature Extracting Module that captures a general feature ofa lung nodule, an Uncertainty-Aware Module that producesthree features for the annotations’ union, intersection, andannotation set, and an Intersection-Union ConstrainingModule that uses distances between the three features tobalance the predictions of final segmentation and MCM.To comprehensively demonstrate the performance of ourmethod, we propose a Complex-Nodule Validation on LIDCIDRI, which tests UGMCS-Net’s segmentation performanceon lung nodules that are difficult to segment using common methods. Experimental results demonstrate that ourmethod can significantly improve the segmentation performance on nodules that are difficult to segment usingconventional methods.

影像科医生具有不同的培训和临床经验，导致肺结节分割注释的差异，从而产生分割的不确定性。传统方法通常选择单个注释作为学习目标，或者尝试学习包含多个注释的潜在空间。然而，这些方法未能充分利用多个注释之间的共识和分歧中蕴含的宝贵信息。在本文中，我们提出了一种不确定性感知注意机制（UAAM），利用多个注释之间的共识和分歧来促进更好的分割。为此，我们引入了多置信度蒙版（MCM），其中包括低置信度（LC）蒙版和高置信度（HC）蒙版。LC蒙版指示具有低分割置信度的区域，放射科医生可能具有不同的分割选择。在UAAM之后，我们进一步设计了一个不确定性引导的多置信度分割网络（UGMCS-Net），其中包含三个模块：一个特征提取模块，捕获肺结节的通用特征，一个不确定性感知模块，产生注释的并集、交集和注释集的三个特征，以及一个交集-并集约束模块，使用三个特征之间的距离来平衡最终分割和MCM的预测。为了全面展示我们方法的性能，我们提出了在LIDC IDRI上进行复杂结节验证，测试UGMCS-Net在使用常规方法难以分割的肺结节上的分割性能。实验结果表明，我们的方法可以显著改善对使用传统方法难以分割的结节的分割性能。

Method

方法

NetworkIn Figure 3, we present the architecture of the UncertaintyGuided Multi-Confidence Segmentation Network (UGMCSNet). The network takes the lung nodule CT image as inputs,and produces two outputs: a predicted Multi-Confidence Mask(MCM) and a final segmentation X**S. The MCM combinespredicted union ∪(X) and intersection ∩(X). The learningtargets of the network are the annotation set GT , as well astheir Union Mask ∪(**GT ) and Intersection Mask ∩(GT ). Theinput images and their corresponding masks have dimensionsof 50 × 50 pixels, obtained through cropping from the LIDCIDRI dataset with official annotations. Before being fed intothe network, the input images and masks are resized todimensions of 3 × 64 × 64 pixels.

在图3中，我们展示了不确定性引导的多置信度分割网络（UGMCS-Net）的架构。该网络以肺结节CT图像作为输入，并产生两个输出：预测的多置信度掩码（MCM）和最终分割XS。MCM结合了预测的并集∪(X)*和交集∩(X)。网络的学习目标是注释集GT，以及它们的并集掩码∪(GT )和交集掩码∩(GT )。输入图像及其相应的掩码的尺寸为50×50像素，通过从LIDC IDRI数据集中裁剪获得，带有官方注释。在输入网络之前，输入图像和掩码被调整大小为3×64×64像素。

Conclusion

结论

This paper introduces the Uncertainty-Aware AttentionMechanism (UAAM), which leverages the consensus ordisagreements among multiple annotations to improve segmentation and identify regions with low segmentation confidence. UAAM captures features from the Multi-ConfidenceMask (MCM), a combination of a Low-Confidence (LC)Mask and a High-Confidence (HC) Mask. Based on UAAM,we further design an Uncertainty-Guide Segmentation Network (UGMCS-Net), which contains a Feature ExtractingModule*, an Uncertainty-Aware Module, and an IntersectionUnion Constraining Module*. These modules together learnvaluable information from the consensus or disagreementsamong multiple annotations, providing regions with high andlow segmentation confidences, and a segmentation result thatcan balance all possibilities. Besides the traditional validationmethod, we propose a Complex-Nodule Validation on LIDCIDRI, which tests UGMCS-Net’s segmentation performanceon the lung nodules that are difficult to segment by U-Net.Experimental results demonstrate that our method can significantly improve the segmentation performance on nodules withpoor segmentation by U-Net.

本文介绍了一种不确定性感知注意机制（UAAM），利用多个注释之间的共识或分歧来改善分割，并识别具有低分割置信度的区域。UAAM从多置信度掩码（MCM）中捕获特征，该掩码是低置信度（LC）蒙版和高置信度（HC）蒙版的组合。基于UAAM，我们进一步设计了一个不确定性引导的分割网络（UGMCS-Net），其中包含一个特征提取模块、一个不确定性感知模块和一个交集-并集约束模块。这些模块共同从多个注释之间的共识或分歧中学习有价值的信息，提供具有高和低分割置信度的区域，以及可以平衡所有可能性的分割结果。除了传统的验证方法外，我们提出了在LIDC IDRI上进行的复杂结节验证，该验证测试UGMCS-Net在U-Net难以分割的肺结节上的分割性能。实验结果表明，我们的方法可以显著改善U-Net对于分割困难结节的分割性能。

Figure

图

Fig. 1. (A). Uncertainty caused by multiple annotations. HC is highconfidence mask, which is the intersection of the annotation set. LC isLow-Confidence mask, which is the difference between the annotationset’s union and intersection. Multi-Confidence mask is the combinationof LC and HC. (B). Hounsfield unit kernel estimations in HC and LC ofLIDC-IDRI.

图1. (A). 多个注释引起的不确定性。HC代表高置信度蒙版，即注释集的交集。LC代表低置信度蒙版，即注释集的并集与交集之间的差异。多置信度蒙版是LC和HC的组合。(B). LIDC-IDRI中HC和LC的Hounsfield单位核估计。

Fig. 2. Overview of Uncertainty-aware attention mechanism. The difference between union and intersection masks is the low-confidence mask,which guides the learning of low-certainty features. The intersectionmask is the high-confidence mask, which guides the learning of highcertainty features. The annotation set guides the learning of a plausiblesegmentation, which is a balance between all annotations.

图2. 不确定性感知注意机制 概述。并集掩码与交集掩码之间的差异是低置信度掩码，指导低确定性特征的学习。交集掩码是高置信度掩码，指导高确定性特征的学习。注释集指导了一个合理分割的学习，这是所有注释之间的平衡。

Fig. 3. Overview of uncertainty-guided multi-confidence segmentation network. This network contains three modules: (1) Feature extracting module,(2) Uncertainty-aware module, and (3) Intersection-union constraining module.

图3不确定性引导的多置信度分割网络概述。该网络包含三个模块：(1) 特征提取模块，(2) 不确定性感知模块，以及 (3) 交集-并集约束模块。

Fig. 4. Intersection-union constraining module. It contains three featureaware attention blocks, and each block has specific feature learningpreferences.

图4. 交集-并集约束模块。它包含三个特征感知注意力块，每个块具有特定的特征学习偏好。

Fig. 5. Multiple annotation fusion loss.

图5. 多注释融合损失。

Fig. 6. Segmentation results of U-Net, Attention U-Net, R2U-Net,Channel U-Net, Nested U-Net, UGS-Net, and UGMCS-Net. The redboxes corresponding to the Input column indicate the features thatshould be noted or the error-prone locations of the nodules duringsegmentation. The red boxes in the UGMCS-Net column indicate thesegmentation detail of UGMCS-Net at these locations. The green boxesindicate the inadequacies in the suboptimal segmentation result. The lastcolumn is the diameter of the corresponding nodule in mm.

图6. U-Net、Attention U-Net、R2U-Net、Channel U-Net、Nested U-Net、UGS-Net 和 UGMCS-Net 的分割结果。与 输入 列相对应的红色框表示在分割过程中应注意的特征或结节的易错位置。UGMCS-Net 列中的红色框表示 UGMCS-Net 在这些位置的分割细节。绿色框表示次优分割结果中的不足之处。最后一列是相应结节的直径，以毫米为单位。

Fig. 7. The predicted intersection ∩(X), predicted union ∪(X), final segmentation XS, and MCM are generated by the UGMCS-Net. Colors in MCMare used for better visualization, red for ∩(X) and blue for ∪(X). In addition, final segmentation is represented in the MCM and marked with greento facilitate comparison. Red boxes indicate areas or features of nodules that are not easily distinguishable. The last column is the diameter of thecorresponding nodule in mm.

图7. UGMCS-Net 生成了预测的交集 ∩(X)，预测的并集 ∪(X)，最终分割 XS 和 MCM。在MCM中，红色代表 ∩(X)，蓝色代表 ∪(X)，用于更好地可视化。此外，最终分割也呈现在MCM中，并用绿色标记以便比较。红色框表示结节的区域或特征不易区分。最后一列是相应结节的直径，以毫米为单位。

Fig. 8. Comparison of HU value kernel density estimation of real HC,LC, and predicted HC and LC.

图8. 实际高置信度（HC）、低置信度（LC）和预测的高置信度（HC）和低置信度（LC）的HU值核密度估计的比较。

Fig. 9. Visualization of different convolutional layers. The Result in each case represents the final prediction of the network. M3, M2, and M1respectively represent the visual feature maps of the third from the bottom, the second, and the first convolution layer under different networkconfigurations. Red arrows indicate areas of nodules that need attention. d represents the diameter of the corresponding nodule in mm.

图9.不同卷积层的可视化。每种情况下的 结果 表示网络的最终预测。M3、M2 和 M1 分别表示在不同网络配置下，从底部数起的第三、第二和第一卷积层的视觉特征图。红色箭头表示需要注意的结节区域。d 表示相应结节的直径，以毫米为单位。

Fig. 10. Failure cases demonstration. The number below each segmen tation mask is its DSC score. The unit of DSC is the percentage.

图10. 失败案例展示。每个分割掩码下方的数字是其 DSC 分数。DSC 的单位是百分比。

Fig. 11. The feature visualizations of R ′ LC and R ′ HC

图11. R ′ LC 和 R ′ HC 的特征可视化

Fig. 12. Complex-nodule validation. This validation tests UGMCS-Net’s segmentation performance on the lung nodules that are difficult to segmentby U-Net by three levels. The last two masks of each CT image are the segmentation results of UGMCS-Net and U-Net respectively. The segmentationresults of U-Net are shown in black, and the UGMCS-Net are shown in red. All indicators are expressed in percentages.

图12. 复杂结节验证。此验证通过三个级别测试UGMCS-Net在U-Net难以分割的肺结节上的分割性能。每张CT图像的最后两个掩码分别是UGMCS-Net和U-Net的分割结果。U-Net的分割结果显示为黑色，UGMCS-Net显示为红色。所有指标均以百分比表示。

Table

表

TABLE I performance comparison between our ugmcs-net and ten networks based on the u-net structure on the lidc-idridataset. ugs-net represents a preliminary version of this work in a . all indicators are expressed in percentages

表格 I：基于LIDC-IDRI数据集的UGMCS-Net和十个基于U-Net结构的网络性能比较。UGS-Net代表本工作的初步版本。所有指标均以百分比表示。

TABLE II statistical analysis of u-net, attentionu-net, and ugmcs-net

表II U-Net、Attention U-Net 和 UGMCS-Net 的统计分析

TABLE III performance comparison between u-net, attention u-net, ugs-net, and ugmcs-net on label1 and label2, which are presented as subscript 1 and 2

表格 IIIU-Net、Attention U-Net、UGS-Net 和 UGMCS-Net 在 Label1 和 Label2 上的性能比较，它们分别表示为下标 1 和 2

TABLE IV ablation study for modules. all indicators are expressed in percentages

表格 IV模块消融研究。所有指标均以百分比表示

TABLE V  the parameter quantity and computation of the model under different variants

表格 V 不同变体下模型的参数数量和计算量

TABLE VI  performance comparison between different backbone

表格 VI 不同骨干网络之间的性能比较

TABLE VII  performance comparison between different filter settings

表格 VII 不同滤波器设置之间的性能比较

TABLE VIII performance comparison between different α settings

表格 VIII 不同 α 设置之间的性能比较

TABLE IX analysis of segmentation performances in complex-nodule validation. ugmcs-8a + 8b, ugmcs-iucm, and ugmcs-netaverage dsc and iou are followed by the difference (green number) from the corresponding metric of attention u-net. all indicators are expressed in percentages

表IX 复杂结节验证中分割性能的分析。UGMCS-8a + 8b、UGMCS-IUCM 和 UGMCS-Net的平均DSC和IOU，后跟着与Attention U-Net相应指标的差异（绿色数字）。所有指标均以百分比表示。