文献速递|基于深度学习的癌症生物标志物发现多模态数据融合_multimodal data fusion for cancer biomarker discov-CSDN博客

本文链接：https://blog.csdn.net/xx_nm98/article/details/143694690

【一】文章信息

文献题目：Multimodal data fusion for cancer biomarker discovery with deep learning

【二】核心亮点

1. By integrating data from different modes such as molecular data, digital pathology and radiography, patient populations can be better stratified and personalized care can be provided. (通过整合分子数据、数字病理和放射影像等不同模态的数据，可以更好地分层患者群体并提供个性化护理。)

By standardizing the data set, the study improves the interpretability and consistency of the data, thereby facilitating wider research and clinical applications. (该研究通过标准化数据集，提高了数据的可解释性和一致性，从而促进更广泛的研究和临床应用。)

【三】摘要

Technological advances now make it possible to study a patient from multiple angles with high-dimensional, high-throughput multi-scale biomedical data. In oncology, massive amounts of data are being generated ranging from molecular, histopathology, radiology to clinical records. The introduction of deep learning has significantly advanced the analysis of biomedical data. However, most approaches focus on single data modalities leading to slow progress in methods to integrate complementary data types. Development of effective multimodal fusion approaches is becoming increasingly important as a single modality might not be consistent and sufficient to capture the heterogeneity of complex diseases to tailor medical care and improve personalised medicine. Many initiatives now focus on integrating these disparate modalities to unravel the biological processes involved in multifactorial diseases such as cancer. However, many obstacles remain, including lack of usable data as well as methods for clinical validation and interpretation. Here, we cover these current challenges and reflect on opportunities through deep learning to tackle data sparsity and scarcity, multimodal interpretability, and standardisation of datasets.

如今，技术进步使得利用高维、高通量、多尺度生物医学数据从多角度研究患者成为可能。在肿瘤学领域，从分子、组织病理学、放射学到临床记录，正在产生大量的数据。深度学习的引入极大地推进了生物医学数据的分析。然而，大多数方法侧重于单一数据模式，导致集成互补数据类型的方法进展缓慢。发展有效的多模式融合方法正变得越来越重要，因为单一模式可能不一致，也不足以捕捉复杂疾病的异质性，从而调整医疗保健和改善个性化医疗。现在，许多举措的重点是整合这些不同的模式，以揭示涉及多因素疾病（如癌症）的生物学过程。然而，仍然存在许多障碍，包括缺乏可用的数据以及临床验证和解释的方法。在这里，我们涵盖了这些当前的挑战，并反思了通过深度学习解决数据稀疏性和稀缺性、多模态可解释性和数据集标准化的机会。

【四】图表

Fig. 1: Generation and processing of routinely collected biomedical modalities in oncology.

Prior to data fusion, different steps are needed to go from the raw data to workable data representations for each modality, e.g. EHRs, molecular data and medical images.

图1:肿瘤学中常规收集的生物医学模式的生成和处理。

在数据融合之前，从原始数据到每种形式的可用数据表示需要不同的步骤，例如EHRs、分子数据和医学图像。

Fig. 2: Overview of different fusion strategies for multimodal data.

a) Raw data is processed into workable formats. b) For each modality features are extracted using dedicated encoder algorithms. c) Early fusion. d) Intermediate fusion. e) Late fusion.

图2:多模态数据的不同融合策略的概述。

a)原始数据被处理成可工作的格式。b)对于每个模态，使用专用的编码器算法提取特征。c)早期融合。d)中间融合。e)晚期融合。

Fig. 3: Examples of model interpretability methods for histopathology and gene expression.

Histopathology: a) Examples of informative tiles for predicting the presence of TP53 mutations from histopathology images in prostate cancer (unpublished data). b) Visualisation of regions within tiles most relevant to the prediction, derived via GradCAM151. c) Individual cells within informative tiles are segmented and classified by HoverNet155. For a fine-grained interpretation of relevant cells (black annotations), pertinent cells within the tile are encircled by calculating the contours from regions highlighted by Grad-CAM. Gene Expression: d) Examples of SHAP visualisation152 of hypothetical gene importance according to unimodal model (top) and joint multimodal model (bottom) for cancer survival prediction. e) Example of pathway importance visualisation based on the respective gene SHAP-values in unimodal (top) versus joint multimodal (bottom) models with respect to cancer survival prediction.

图3：组织病理学和基因表达的模型可解释性方法示例。

组织病理学：a)从前列腺癌组织病理学图像中预测TP53突变存在的信息片示例（未发表的数据）。b)通过GradCAM151导出的与预测最相关的tile内区域的可视化。c)使用HoverNet155对信息tile内的单个细胞进行分割和分类。对于相关细胞的细粒度解释（黑色注释），通过计算由Grad-CAM突出显示的区域的轮廓，将tile内的相关细胞包围起来。基因表达：d)根据单峰模型（上）和联合多峰模型（下）预测癌症生存的假设基因重要性的SHAP可视化示例152。e)基于单峰（上）与联合多峰（下）模型中各自基因shap值的途径重要性可视化示例，与癌症生存预测有关。

如何学习大模型 AI ？

由于新岗位的生产效率，要优于被取代岗位的生产效率，所以实际上整个社会的生产效率是提升的。

但是具体到个人，只能说是：

“最先掌握AI的人，将会比较晚掌握AI的人有竞争优势”。

这句话，放在计算机、互联网、移动互联网的开局时期，都是一样的道理。

我在一线互联网企业工作十余年里，指导过不少同行后辈。帮助很多人得到了学习和成长。

我意识到有很多经验和知识值得分享给大家，也可以通过我们的能力和经验解答大家在人工智能学习中的很多困惑，所以在工作繁忙的情况下还是坚持各种整理和分享。但苦于知识传播途径有限，很多互联网行业朋友无法获得正确的资料得到学习提升，故此将并将重要的AI大模型资料包括AI大模型入门学习思维导图、精品AI大模型学习书籍手册、视频教程、实战学习等录播视频免费分享出来。

在这里插入图片描述