MIST：用于组织病理学亚型预测的多实例选择性Transformer|文献速递--基于深度学习的医学影像病灶分割

Title

题目

MIST: Multi-instance selective transformer for histopathological subtype prediction

MIST：用于组织病理学亚型预测的多实例选择性Transformer

01

文献速递介绍

组织病理学亚型预测在癌症疾病的诊断和治疗中具有重要的临床意义。组织病理学亚型预测旨在识别全视野图像（WSI）中与病理组织相关的不同子类别（图1(1)），例如，正常黏膜、碎屑、病理性良性组织、淋巴细胞和侵袭性癌（Han等，2022）。组织病理学亚型预测通常依赖于临床决策，以制定最佳治疗方案（Han等，2017）。通过了解组织病理学亚型，病理学家可以及早控制肿瘤细胞的转移，并根据多种癌症的特殊临床表现和预后结果制定有效的治疗方案（Han等，2017），如乳腺癌和结直肠癌。此外，组织病理学亚型的预测为肿瘤微环境分析提供了新的见解，并对临床终点产生了重大影响（Gurcan等，2009；Kather等，2019，2018，2017）。

Abatract

摘要

Accurate histopathological subtype prediction is clinically significant for cancer diagnosis and tumor microenvironment analysis. However, achieving accurate histopathological subtype prediction is a challengingtask due to (1) instance-level discrimination of histopathological images, (2) low inter-class and large intraclass variances among histopathological images in their shape and chromatin texture, and (3) heterogeneousfeature distribution over different images. In this paper, we formulate subtype prediction as fine-grainedrepresentation learning and propose a novel multi-instance selective transformer (MIST) framework, effectivelyachieving accurate histopathological subtype prediction. The proposed MIST designs an effective selective selfattention mechanism with multi-instance learning (MIL) and vision transformer (ViT) to adaptive identifyinformative instances for fine-grained representation. Innovatively, the MIST entrusts each instance withdifferent contributions to the bag representation based on its interactions with instances and bags. Specifically,a SiT module with selective multi-head self-attention (S-MSA) is well-designed to identify the representativeinstances by modeling the instance-to-instance interactions. On the contrary, a MIFD module with theinformation bottleneck is proposed to learn the discriminative fine-grained representation for histopathologicalimages by modeling instance-to-bag interactions with the selected instances. Substantial experiments on fiveclinical benchmarks demonstrate that the MIST achieves accurate histopathological subtype prediction andobtains state-of-the-art performance with an accuracy of 0.936. The MIST shows great potential to handlefine-grained medical image analysis, such as histopathological subtype prediction in clinical applications.

准确的组织病理学亚型预测对于癌症诊断和肿瘤微环境分析具有重要的临床意义。然而，由于以下几个挑战，实现准确的组织病理学亚型预测是一项艰巨的任务：(1) 组织病理学图像的实例级别区分，(2) 组织病理学图像在形状和染色质纹理上的类间差异小而类内差异大，以及(3) 不同图像之间异质特征分布。在本文中，我们将亚型预测表述为细粒度表征学习，并提出了一种新颖的多实例选择性Transformer（MIST）框架，有效实现了准确的组织病理学亚型预测。所提出的MIST设计了一种结合多实例学习（MIL）和视觉Transformer（ViT）的有效选择性自注意机制，自适应地识别出细粒度表征的有用实例。创新性地，MIST根据实例与袋之间的相互作用赋予每个实例在袋表示中的不同贡献。具体而言，MIST中的SiT模块设计了选择性多头自注意力机制（S-MSA），通过建模实例与实例之间的相互作用来识别具有代表性的实例。相反，MIFD模块通过信息瓶颈策略，基于与所选实例的实例与袋之间的相互作用，学习组织病理图像的判别性细粒度表征。在五个临床基准上的大量实验表明，MIST实现了准确的组织病理学亚型预测，并以0.936的准确率获得了最先进的性能。MIST显示了在处理细粒度医学图像分析（如临床应用中的组织病理学亚型预测）方面的巨大潜力。

Method

方法

The proposed MIST (Fig. 3) formulates histopathological subtypeprediction as fine-grained representation learning and achieveshistopathological subtype prediction by constructing a multiple-stagevision transformer coupled with multi-instance learning. The newlydesigned MIST learns the instance-level fine-grained features by adaptive entrusting each instance with different contributions for thehistopathological image representation. Therefore, the MIST has threetightly connected components:

 (1) selective instance transformer(SiT) with selective self-attention mechanism (S-MSA) to adaptivelyidentify the representative instances from the bag for discriminative representation of the histopatho