【论文阅读】PCNet: Prior Category Network for CT Universal Segmentation Model 详细解读-CSDN博客

Abstract 摘要 ——简介创新点、效果
Introduction 引言——背景、意义、发展历程、提出问题
Related Work 相关工作——重点看缺点
Methods——论文主要方法，网络模型介绍
Experiment——步骤及结果分析，实验训练测试，数据集，评价指标
Discussion——结论和未来研究方向

接下来，正式开始看文章，这些是对于一个文章来说比较重要的（摘要、模型结构图、方法、实验、代码）需要着重看

论文介绍

题目：PCNet: Prior Category Network for CT Universal Segmentation Model

期刊：TMI

日期：2024

论文链接：PCNet: Prior Category Network for CT Universal Segmentation Model | IEEE Journals & Magazine | IEEE Xplore

代码链接：GitHub - YixinChen-AI/PCNet

文章概述

文章提出了：先验类别网络 PCNet 和 基于 CLIP 的类别提示

先验类别网络 (PCNet)，通过利用不同类别解剖结构之间的先验知识来提高分割性能
可以处理各种医疗分割任务的单一模型（包括不同的分割目标，如器官、血管和骨骼）
PCNet由三部分组成：先验类别提示 (PCP)、层次类别系统 (HCS)、层次类别损失 (HCL)
CLIP提供了一个统一的标签空间
CLIP中的文本编码器将自由文本转换为嵌入，这些嵌入可以作为指导分割模型输出的提示

数据集

TotalSeg数据集：1204张图像，104个独特的解剖结构
训练：1081 个，验证：57 个，测试： 65 个
所有图像都重新采样到1.5 × 1.5 × 1.5mm3各向同性分辨率
CT扫描裁剪为96 × 96 × 96的输入

训练参数

优化策略：Nesterov动量为0.99的SGD优化器，权值衰减为3e-5
批大小为8，每个epoch包含1000次迭代
初始学习率为 0.01，并遵循聚学习率策略衰减：(1−epoch/1000)0.9
硬件：80GB VRAM 的 NVIDIA A800 GPU

模型评价

评价指标： Dice Similarity Coefficient (DSC) ：分割性能指标。分数越高，分割越准确
评价数据：12个下游数据集，它们中可用的所有注释数据都用于测试模型的可迁移性

从这张表我们观察，蓝框到绿框指标的提升，说明了集成文本的重要性，绿到红说明了合并先验类别知识的重要性

对比实验与实验结果

将 PCNet 与其他具有相同主干但配置不同的模型进行比较

例如，UNet 作为主干时，将 UNet + PCNet 和 CDUM UNet 进行了统计比较

结果如下图所示，

这张图表示了在不同的数据集，不同的模型下的Dice得分（DSC），其中Dice得分后有***的，代表是我们的方法

只看彩色框住的部分，可以得出：

同样是 TotalSeg all ，与 CDUM UNet 相比，PC 的 UNet 从 83.96 增加到 87.23，可见提升

消融实验

通过增加和删除PCP、HCS 、HCL、注意力机制或以随机方式创建 100 个新类别，来证明文中提出的三个模块的重要性
同时，为了证明文本提示的重要性，还通过改变文字描述等方式，设计了消融研究，确保先验知识的准确性在PCNet框架中至关重要
同时，评估了各种 CLIP 主干，将clip与 ViT-B [50]、ViT-L [50]、ResNet101 [51] 和 ResNext50×64 [52]主干相结合，最优配置是PCNet-SwinUNETR 与 CLIP-ResNet101

此处以第一点为例，即说明三个模块的重要性

红色框：文本特征与先验知识在 PCP 中的重要性
蓝色框：“Small、Base 和 Large”中实现了最高的平均 DSC，合并更多层次类别的重要性
黄色框：HCL 可以成功地引导模型捕获 HCS 中的先验知识

结论

PCNet 框架来改进通用模型在医学 CT 图像上的分割，特别是对于处理具有大量器官/组织的复杂任务

三个核心组件（PCP、HCS 和 HCL）利用 CLIP 和临床医生提示来有效地建立解剖关系

将我们的方法与一系列分割模型相结合可以显着提高它们的性能（即插即用）

基于 PCNet 的模型在多个下游数据集上也表现很好，且无需进行微调

Methods方法

这里，把文章的方法拿出来单独讲解

概述

想要具体了解一个文章的方法，看图和代码是最直观的，我们先看图

图片左上角的“categories statistics”分类统计中，一共137个类别，其中Basic Categories，Specific Organs，Anatomical Structures，Functional Systems就是HSC中区分的四个层，顺着箭头方向，接下来到了PCP模块，在这里将上一步学到的知识变成文本嵌入，通过text encoder处理。
与此同时，图片中间的CT image，通过标准化处理，变成了96*96*96的大小，再通过特征提取器，提取成为特征f，通过全局平均池化层处理， f ' 与text encoder后的结果融合，一同进入多层感知部分
继续上一步的操作，此时生成了参数θ，经过卷积，注意力机制（使用PCP图作为注意力图），然后将得到的特征传递到分割器，生成最终预测。最终，黄色区域HCL

这三个模块的主要作用如下：

PCP 蓝色区域：

将先前的医学知识集成到类别提示中，为 PCP 图创建嵌入，并通过注意力机制与图像特征相结合。

HCS 橙色区域：

分层扩展这些类别，形成一个与基本类别相关的 HCS 图。

HCL 黄色区域：

通过HCS图，引导参数梯度有效地捕获这种分层信息，即明确引导模型捕获类别之间的先验关系

PCP（Prior Category Prompt）

概括

目的：识别特定的器官并提供有关其解剖结构和与其他类别的关系的附加信息

文本分支：为每个器官生成 CLIP 嵌入。通过clip，学习医学术语，识别器官结构，将学到的文本嵌入模型中（标签嵌入）

图像分割：包含两部分，提取器和分割器

处理后，包含137 个类别（文本提示），主要从如下角度来考虑：

文字描述的清晰度，长度
描述器官的相对位置和相邻结构

流程

1. 提取特征：

$f = E(x)$

x：数据集的 CT 扫描，E：分割主干模型的提取器，特征图 f

2. 全局平均池化：

特征图 f ：D×W×H×Z 变为全局特征 f ' ：D×1×1×1， D：特征数量

3. text encoder：

ei ：CLIP 的预训练文本编码器生成的第 i 个类别的 PCP 的 CLIP 嵌入

4. 多层感知机：

生成参数θi ：D×1×1×1 （137个θ，同137个类别），此过程将文本语义和图像信息融合

5. 卷积：

利用 θ 的卷积操作来控制视觉特征 f ，f '' ：137×W ×H×S

6. PCP 图：

PCP 图 G ：137×137，第j列和第i行的元素：第i和第j类的提示嵌入之间的余弦相似度，指导模型捕获不同类别之间的相似性

7. 注意力：

特征 f '' 与 PCP 图 G 相结合，G 用作注意力图，得到 p ：模型的预测

HCS（Hierarchy Category System）

概括

利用层间关系分割，那么是如何分层的，一共四层

具体结构：左肾、右肾

特定器官：心脏、肺...

解剖结构：血管、腹部器官、骨骼...

功能结构：呼吸系统、心血管系统...

在结合 HCS 后，模型必须解释 Dice 和 BCE 损失计算中的扩展标签。因此，分层类别用于 LBCE 和 LDice。更新后的损失函数为

可以通过层级，来学习共同特征

HCL（Hierarchy Category Loss）

这一块的出现，主要是增强一下之前两个模块的效果。注意到了不同层之间的影响

目的：用 HCL 明确引导模型捕获类别之间的先验关系

qj 第 j 个层级类别的预测结果

pi 该层级下各基本类别的预测

代码解读

这篇文章的代码作者开源的较少，所以我就全部放在下面了，如果真的想要实现图像分割，还需要将代码补全，如下应该只是即插即用的部分。

首先是PCNet.py

定义了一个名为PCNet的PyTorch神经网络模型，用于3D医学图像分割任务。它使用了不同的骨干网络（backbone）结构来提取特征，然后进行分割

import numpy as np
import torch
import monai
import torch.nn as nn
from .STUNet import STUNet

class PCNet(nn.Module):

# img_size：输入图像的尺寸。in_channels：输入图像的通道数。out_channels：输出的通道数（通常是目标器官的类别数）。backbone：指定要使用的骨干网络结构，默认是swinunetr。organ_embedding：用于器官嵌入的编码矩阵。feat_concat：控制是否将特征图进行拼接
    def __init__(self, img_size, in_channels, out_channels, backbone = 'swinunetr', organ_embedding = None,feat_concat = False,):
        # encoding: rand_embedding or word_embedding
        super().__init__()
        self.backbone_name = backbone
        self.feat_concat = feat_concat

# 根据传入的backbone参数，选择不同的模型结构。可选的模型包括SwinUNETR、SegResNet、UNet、VNet、STUNet等。不同的模型有不同的超参数（如feature_size、drop_rate等），这些超参数控制模型的层数、特征大小和正则化强度等。
        if backbone == 'swinunetr_small':
            self.backbone = monai.networks.nets.SwinUNETR(
                img_size=img_size,
                in_channels=in_channels,
                out_channels=organ_embedding.shape[0],
                feature_size=24,
                drop_rate=0,
                attn_drop_rate=0,
                dropout_path_rate=0,
            )
        elif backbone == 'swinunetr_base':
            self.backbone = monai.networks.nets.SwinUNETR(
                img_size=img_size,
                in_channels=in_channels,
                out_channels=organ_embedding.shape[0],
                feature_size=48,
                drop_rate=0.1,
                attn_drop_rate=0.1,
                dropout_path_rate=0.1,
            )
        elif backbone == 'swinunetr_large':
            self.backbone = monai.networks.nets.SwinUNETR(
                img_size=img_size,
                in_channels=in_channels,
                out_channels=organ_embedding.shape[0],
                feature_size=96,
                drop_rate=0,
                attn_drop_rate=0,
                dropout_path_rate=0,
            )
        elif backbone == 'segresnet':
            self.backbone = monai.networks.nets.SegResNet(
                blocks_down=[1,2,2,4],
                blocks_up=[1,1,1],
                init_filters=24,
                in_channels=in_channels,
                out_channels=organ_embedding.shape[0],
                dropout_prob=0.2,
            )
        elif backbone == 'unet':
            self.backbone = monai.networks.nets.UNet(
                spatial_dims=3,
                in_channels=in_channels,
                out_channels=organ_embedding.shape[0],
                channels=(16,32,64,128,256),
                strides=(2,2,2,2),
                num_res_units=2,
                norm=monai.networks.layers.Norm.BATCH,
            )
        elif backbone == 'unet_large':
            self.backbone = monai.networks.nets.UNet(
                spatial_dims=3,
                in_channels=in_channels,
                out_channels=organ_embedding.shape[0],
                channels=(32,64,128,256,512),
                strides=(2,2,2,2),
                num_res_units=2,
                norm=monai.networks.layers.Norm.BATCH,
            )
        elif backbone == 'unetpp':
            self.backbone = monai.networks.nets.BasicUNetPlusPlus(
                spatial_dims=3,
                in_channels=in_channels,
                out_channels=organ_embedding.shape[0],
                features=(32,32,64,128,256,32),
                dropout=0,
                upsample="deconv"
            )
        elif backbone == "vnet":
            self.backbone = monai.networks.nets.VNet(
                spatial_dims=3,
                in_channels=in_channels,
                out_channels=organ_embedding.shape[0],
            )
        elif backbone == "STUNet_small":
            model = STUNet(in_channels, 105, depth=[1,1,1,1,1,1], dims=[16, 32, 64, 128, 256, 256],
                    pool_op_kernel_sizes = ((2,2,2),(2,2,2),(2,2,2),(2,2,2),(2,2,2)),
               conv_kernel_sizes = ((3,3,3),(3,3,3),(3,3,3),(3,3,3),(3,3,3),(3,3,3)))
            model.load_state_dict(torch.load("./STUNet/small_ep4k.model")["state_dict"])
            model.seg_outputs[0] = nn.Conv3d(256,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[1] = nn.Conv3d(128,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[2] = nn.Conv3d(64,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[3] = nn.Conv3d(32,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[4] = nn.Conv3d(16,organ_embedding.shape[0],kernel_size=1,stride=1)
            self.backbone = model
        elif backbone == "STUNet_base":
            model = STUNet(1, 105, depth=[1,1,1,1,1,1], dims=[32, 64, 128, 256, 512, 512],
                    pool_op_kernel_sizes = ((2,2,2),(2,2,2),(2,2,2),(2,2,2),(2,2,2)),
               conv_kernel_sizes = ((3,3,3),(3,3,3),(3,3,3),(3,3,3),(3,3,3),(3,3,3)))
            model.load_state_dict(torch.load("./models/STUNet/base_ep4k.model")["state_dict"])
            model.seg_outputs[0] = nn.Conv3d(512,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[1] = nn.Conv3d(256,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[2] = nn.Conv3d(128,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[3] = nn.Conv3d(64,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[4] = nn.Conv3d(32,organ_embedding.shape[0],kernel_size=1,stride=1)
            self.backbone = model

        elif backbone == "STUNet_large":
            model = STUNet(1, 105, depth=[2,2,2,2,2,2], dims=[64, 128, 256, 512, 1024, 1024],
                    pool_op_kernel_sizes = ((2,2,2),(2,2,2),(2,2,2),(2,2,2),(2,2,2)),
               conv_kernel_sizes = ((3,3,3),(3,3,3),(3,3,3),(3,3,3),(3,3,3),(3,3,3)))
            # model.load_state_dict(torch.load("./models/STUNet/large_ep4k.model")["state_dict"])
            model.seg_outputs[0] = nn.Conv3d(1024,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[1] = nn.Conv3d(512,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[2] = nn.Conv3d(256,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[3] = nn.Conv3d(128,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[4] = nn.Conv3d(64,organ_embedding.shape[0],kernel_size=1,stride=1)
            self.backbone = model
            
        elif backbone == "STUNet_huge":
            model = STUNet(1,105, depth=[3,3,3,3,3,3], dims=[96, 192, 384, 768, 1536, 1536],
                    pool_op_kernel_sizes = ((2,2,2),(2,2,2),(2,2,2),(2,2,2),(2,2,2)),
               conv_kernel_sizes = ((3,3,3),(3,3,3),(3,3,3),(3,3,3),(3,3,3),(3,3,3)))
            model.load_state_dict(torch.load("./models/STUNet/huge_ep4k.model")["state_dict"])
            model.seg_outputs[0] = nn.Conv3d(1536,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[1] = nn.Conv3d(768,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[2] = nn.Conv3d(384,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[3] = nn.Conv3d(192,organ_embedding.shape[0],kernel_size=1,stride=1)
            model.seg_outputs[4] = nn.Conv3d(96,organ_embedding.shape[0],kernel_size=1,stride=1)
            self.backbone = model
            
        else:
            raise Exception('{} backbone is not implemented in curretn version'.format(backbone))


# 将organ_embedding矩阵注册为模型的缓冲区变量（不会作为可训练参数）。
计算器官嵌入矩阵的相似性（通过内积）以生成prompt_graph。
        self.register_buffer("organ_embedding",organ_embedding)
        mat = torch.matmul(organ_embedding,organ_embedding.T)
        self.register_buffer("prompt_graph",mat)
        


# text_encoder：一个简单的两层全连接网络，用于将特征映射到器官嵌入空间。
segmentors：3D卷积层，用于最终的分割输出。
gap：全局平均池化层，将特征图的尺寸缩小到固定的8x8x8
        #self.organ_embedding = organ_embedding
        self.text_encoder = nn.Sequential(
            nn.Linear(512+512,256),
            nn.LeakyReLU(),
            nn.Dropout(0.2),
            nn.Linear(256,organ_embedding.shape[0])
        )
        self.segmentors = nn.Conv3d(organ_embedding.shape[0] * 2,organ_embedding.shape[0],kernel_size=1,bias=True)
        self.gap = nn.AdaptiveAvgPool3d(output_size=(8,8,8))

# 输入x_in经过骨干网络提取特征。
使用gap进行全局平均池化并计算器官嵌入特征。
将特征与嵌入向量拼接，生成权重并应用于3D卷积。
最终将拼接的特征输入到分割器得到分割输出。
    def forward(self, x_in):
        x_in = self.backbone(x_in)
        x_feat = self.gap(x_in)
        b = x_in.shape[0]
        x_feat = x_feat.view(b,self.organ_embedding.shape[0],-1)
        x_feat = torch.mean(x_feat,dim=0)
        weight = self.text_encoder(
            torch.cat([x_feat,self.organ_embedding],dim=1)).unsqueeze(-1).unsqueeze(-1).unsqueeze(-1)
        x_out = torch.nn.functional.conv3d(x_in,weight)
        x_out = x_out.permute(0,2,3,4,1)
        x_out = torch.matmul(x_out,self.prompt_graph).permute(0,4,1,2,3)
        x = torch.concat([x_out,x_in],dim=1)
        return self.segmentors(x)

HCS.py

这段代码主要定义了一些与医学图像分割任务相关的器官、组织和系统的映射关系。这些映射关系可以用于将医学图像中的标记（通常是数字标签）与人体解剖学结构（如器官或组织）进行关联。

import SimpleITK as sitk
import torch


mapping_visual={0:"background",1: 'adrenal_gland_left', 2: 'adrenal_gland_right', 3: 'aorta', 4: 'autochthon_left', 5: 'autochthon_right', 6: 'brain', 7: 'clavicula_left', 8: 'clavicula_right', 9: 'colon', 10: 'duodenum', 11: 'esophagus', 12: 'face', 13: 'femur_left', 14: 'femur_right', 15: 'gallbladder', 16: 'gluteus_maximus_left', 17: 'gluteus_maximus_right', 18: 'gluteus_medius_left', 19: 'gluteus_medius_right', 20: 'gluteus_minimus_left', 21: 'gluteus_minimus_right', 22: 'heart_atrium_left', 23: 'heart_atrium_right', 24: 'heart_myocardium', 25: 'heart_ventricle_left', 26: 'heart_ventricle_right', 27: 'hip_left', 28: 'hip_right', 29: 'humerus_left', 30: 'humerus_right', 31: 'iliac_artery_left', 32: 'iliac_artery_right', 33: 'iliac_vena_left', 34: 'iliac_vena_right', 35: 'iliopsoas_left', 36: 'iliopsoas_right', 37: 'inferior_vena_cava', 38: 'kidney_left', 39: 'kidney_right', 40: 'liver', 41: 'lung_lower_lobe_left', 42: 'lung_lower_lobe_right', 43: 'lung_middle_lobe_right', 44: 'lung_upper_lobe_left', 45: 'lung_upper_lobe_right', 46: 'pancreas', 47: 'portal_vein_and_splenic_vein', 48: 'pulmonary_artery', 49: 'rib_left_1', 50: 'rib_left_2', 51: 'rib_left_3', 52: 'rib_left_4', 53: 'rib_left_5', 54: 'rib_left_6', 55: 'rib_left_7', 56: 'rib_left_8', 57: 'rib_left_9', 58: 'rib_left_10', 59: 'rib_left_11', 60: 'rib_left_12', 61: 'rib_right_1', 62: 'rib_right_2', 63: 'rib_right_3', 64: 'rib_right_4', 65: 'rib_right_5', 66: 'rib_right_6', 67: 'rib_right_7', 68: 'rib_right_8', 69: 'rib_right_9', 70: 'rib_right_10', 71: 'rib_right_11', 72: 'rib_right_12', 73: 'sacrum', 74: 'scapula_left', 75: 'scapula_right', 76: 'small_bowel', 77: 'spleen', 78: 'stomach', 79: 'trachea', 80: 'urinary_bladder', 81: 'vertebrae_C1', 82: 'vertebrae_C2', 83: 'vertebrae_C3', 84: 'vertebrae_C4', 85: 'vertebrae_C5', 86: 'vertebrae_C6', 87: 'vertebrae_C7', 88: 'vertebrae_L1', 89: 'vertebrae_L2', 90: 'vertebrae_L3', 91: 'vertebrae_L4', 92: 'vertebrae_L5', 93: 'vertebrae_T1', 94: 'vertebrae_T2', 95: 'vertebrae_T3', 96: 'vertebrae_T4', 97: 'vertebrae_T5', 98: 'vertebrae_T6', 99: 'vertebrae_T7', 100: 'vertebrae_T8', 101: 'vertebrae_T9', 102: 'vertebrae_T10', 103: 'vertebrae_T11', 104: 'vertebrae_T12'}

hierarchy_mapping = {
    "adrenal_glands":[1,2],
    "autochthon muscles":[4,5],
    "clavicles":[7,8],
    "femurs":[13,14],
    "gluteus muscles":[16,17,18,19,20,21],
    "hips":[27,28],
    "humerus":[29,30],
    "iliac arteries":[31,32],
    "iliac veins":[33,34],
    "iliopsoas muscles":[35,36],
    "kidneys":[38,39],
    "lungs":[41,42,43,44,45],
    "scapulas":[74,75],
    "vertebrae_C":[81,82,83,84,85,86,87],
    "vertebrae_T":[93,94,95,96,97,98,99,100,101,102,103,104],
    "vertebrae_L":[88,89,90,91,92],
    "heart":[22,23,24,25,26],
    
    "brain and face":[6,12], # this could be useful for neurocranial studies
    "vessels":[31,32,48,33,34,37,47,3],
    "abdominal organs":[40,15,38,39,46,77,78,76,9,80],
    "thoracic organs":[22,23,24,25,26,41,42,43,44,45,11],
    "organs":[1,2,6,9,10,11,15,38,39,40,41,42,43,44,45,46,76,77,78,79,80,12],
    "vertebraes":[81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104],
    "muscles":[4,5,16,17,18,19,20,21,35,36],
    "ribs":[49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72],
    "bones":[7,8,13,14,27,28,29,30,73,74,75,81,82,83,84,85,86,87,88,
             89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,49,50,51,52,53,54,
             55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72],
    
    "cardiovascular system":[3,22,23,24,25,26,31,32,33,34,37,47,48],
    "gastrointestinal tract":[9,10,11,76,78], # 
    "respiratory system":[41,42,43,44,45,79], # 
    "urinary system":[38,39,80],
    "digestive system":[11,78,76,9,15,46],
    "musculoskeletal system":[4,5,16,17,18,19,20,21,35,36,7,8,13,14,27,28,29,
                              30,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,
                              64,65,66,67,68,69,70,71,72,73,74,75,81,82,83,84,85,86,
                              87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104],

}

BTCV_cervix_mapping = {0:0,1:80,2:0,3:0,4:76}
msd_spleen_mapping = {0:0,1:77}
segthor_mapping = {0:0,1:1,2:121,3:79,4:3}



abdomen1k_mapping = None

hierachy = None

这段代码主要定义了一些与医学图像分割任务相关的器官、组织和系统的映射关系。这些映射关系可以用于将医学

mapping_visual 字典

mapping_visual 是一个字典，用于将标记（整数标签）映射到对应的人体解剖学结构名称。例如，标签 1 映射到 adrenal_gland_left（左肾上腺），标签 6 映射到 brain（大脑）。
这个字典定义了 0 到 104 的标签，涵盖了各种人体器官、骨骼、肌肉等。

hierarchy_mapping 字典

hierarchy_mapping 是一个更高级的映射字典，用于将一组相关的解剖学结构组织到一个更大的类别中。例如，adrenal_glands 对应的值 [1,2] 表示左肾上腺和右肾上腺这两个标签。
这个字典帮助将不同的标签组织到一个更大的类别中，例如：
organs 包含所有的器官标签。
bones 包含所有的骨骼标签。
cardiovascular system 包含心血管系统的相关标签。

BTCV_cervix_mapping, msd_spleen_mapping, segthor_mapping 字典

这些字典是特定任务的标签映射。例如，BTCV_cervix_mapping 是 BTCV 数据集的标签映射，msd_spleen_mapping 是 MSD 数据集的标签映射，segthor_mapping 是 SegTHOR 数据集的标签映射。
这些映射通常用于将特定数据集的标签映射到更通用的标签，例如在多任务学习或迁移学习中。

abdomen1k_mapping 和 hierachy

这两个变量在代码中被定义为 None，可能在未来会被赋值或初始化。这意味着目前没有定义或赋值。

总结

这段代码为医学图像分割任务提供了多个标签映射字典，可以用于将标记的数据转换为易于理解的人体结构名称，并组织不同的标签到更高级的类别中。
这些映射字典对于多任务学习、迁移学习或处理不同数据集的标签一致性非常有用。

prompts.py

imICR_prompt={
    "background": "The area of the computerized tomography that does not contain any of the other categories. It is typically the space surrounding the body or the parts of the body that are not of interest in the current analysis.",
    'adrenal_gland_left':'The left adrenal gland is a small, triangular organ located on top of the left kidney.', 
    'adrenal_gland_right':'The right adrenal gland is a small, triangular organ located on top of the right kidney.', 
    'aorta':' The main and largest artery in the human body, originating from the left ventricle of the heart and extending down to the abdomen, where it splits into two smaller arteries.',
    'autochthon_left':'The autochthon muscles on the left are deep muscles of the back', 
    'autochthon_right':'The autochthon muscles on the right are deep muscles of the back', 
    'brain':'The brain is located in the head.', 
    'clavicula_left':'The left clavicle is doubly curved long bone that forms part of the shoulder girdle, near the rib 1', 
    'clavicula_right':'The right clavicle is doubly curved long bone that forms part of the shoulder girdle, near the rib 1', 
    'colon':'The colon is located in the abdomen.', 
     'duodenum':'The duodenum is the first section of the small intestine, located near the stomach. ', 
     'esophagus':'The esophagus is a muscular tube connecting the throat to the stomach.  ', 
     'face':'The face is the front part of the head.', 
     'femur_left':'The left femur ( left thigh bone) is the longest and strongest bones in the body.', 
     'femur_right':'The right femur ( right thigh bone) is the longest and strongest bones in the body.', 
     'gallbladder':'The gallbladder is a pear-shaped sac behind the lower portion of the liver.', 
     'gluteus_maximus_left':'The left Gluteus Maximus is the largest muscle in the left buttock and adjacent to the gluteus medius and gluteus minimus muscles.', 
     'gluteus_maximus_right':'The right Gluteus Maximus is the largest muscle in the right buttock and adjacent to the gluteus medius and gluteus minimus muscles.', 
     'gluteus_medius_left':'The left Gluteus Medius is located near the outer surface of the pelvis and adjacent to the gluteus maximus and gluteus minimus.', 
     'gluteus_medius_right':'The right Gluteus Medius is located near the outer surface of the pelvis and adjacent to the gluteus maximus and gluteus minimus.', 
     'gluteus_minimus_left':'The left Gluteus Minimus is the smallest of the gluteal muscle, located beneath the gluteus medius.', 
     'gluteus_minimus_right':'The right Gluteus Minimus is the smallest of the gluteal muscle, located beneath the gluteus medius.', 
     'heart_atrium_left':'The left heart atrium is a upper chamber of the heart.', 
     'heart_atrium_right':'The right heart atrium is a upper chamber of the heart.', 
     'heart_myocardium':'Myocardium is the muscular wall of the heart.', 
     'heart_ventricle_left':'The left heart ventricle is a lower chamber of the heart.', 
     'heart_ventricle_right':'The right heart ventricle is a lower chamber of the heart.', 
     'hip_left':'The left hip connect the left thigh bone (femur) to the pelvis.', 
     'hip_right':'The left hip connect the right thigh bone (femur) to the pelvis.', 
     'humerus_left':'The left humerus is the long bone in the upper left arm from the shoulder to the elbow, connecting the scapula and the lower arm bones.', 
     'humerus_right':'The right humerus is the long bone in the upper right arm from the shoulder to the elbow, connecting the scapula and the lower arm bones.', 
     'iliac_artery_left':'The left iliac artery is a major blood vessel in the pelvic region that supplies blood to the left lower limb.', 
     'iliac_artery_right':'The right iliac artery is a major blood vessel in the pelvic region that supplies blood to the right lower limb.', 
     'iliac_vena_left':'The left iliac vein a large blood vessel connecting from the left lower limb back to the heart', 
     'iliac_vena_right':'The right iliac vein a large blood vessel connecting from the right lower limb back to the heart', 
     'iliopsoas_left':'The left iliopsoas is a large muscle in the left lower body near the spinal column.', 
     'iliopsoas_right':'The right iliopsoas is a large muscle in the right lower body near the spinal column.', 
     'inferior_vena_cava':'The inferior vena cava is the largest vein in the body from the lower half of the body back to the heart.', 
     'kidney_left':'The left kidney is located slightly higher than the right kidney due to the position of the liver.', 
     'kidney_right':'The right kidney is located slightly lower than the left kidney due to the position of the liver.', 
     'liver':'The liver is the largest internal organ, located in the upper right part of the abdomen, beneath the diaphragm and on top of the stomach, right kidney and intestines. ', 
     'lung_lower_lobe_left':'The left lower lung lobe is the bottom section of the left lung.', 
     'lung_lower_lobe_right':'The right lower lung lobe is the largest section of the right lung.', 
     'lung_middle_lobe_right':'The right middel lung lobe is a small lobe of the right lung.',
     'lung_upper_lobe_left':'The left upper lung lobe is the top section of the left lung.', 
     'lung_upper_lobe_right':'The right upper lung lobe is a small lobe of the right lung.', 
     'pancreas':'The pancrease is an organ located in the abdomen, behind the stomach.',
     'portal_vein_and_splenic_vein':'The portal vein and splenic vein is from the gastrointestinal tract and spleen to liver.',
     'pulmonary_artery':'The pulmonary artery is from the right ventricle of the heart to the lungs.', 
     'rib_left_1':'The left first rib located near the left clavicle, from the first thoracic vertebrae.',
     'rib_left_10':'The left tenth rib, from the tenth thoracic vertebrae.', 
     'rib_left_11':'The left eleventh rib, from the eleventh thoracic vertebrae.',
     'rib_left_12':'The left twelfth rib is the lowest rib, from the twelfth thoracic vertebrae. ', 
     'rib_left_2':'The left second rib, from the second thoracic vertebrae.', 
     'rib_left_3':'The left third rib, from the third thoracic vertebrae.', 
     'rib_left_4':'The left fourth rib, from the fourth thoracic vertebrae.', 
     'rib_left_5':'The left fifth rib, from the fifth thoracic vertebrae.', 
     'rib_left_6':'The left sixth rib, from the sixth thoracic vertebrae.', 
     'rib_left_7':'The left seventh rib, from the seventh thoracic vertebrae.', 
     'rib_left_8':'The left eighth rib, from the eighth thoracic vertebrae.', 
     'rib_left_9':'The left ninth rib, from the ninth thoracic vertebrae.', 
     'rib_right_1':'The right first rib located near the right clavicle, from the first thoracic vertebrae.', 
     'rib_right_10':'The right tenth rib, from the tenth thoracic vertebrae.',
     'rib_right_11':'The right eleventh rib, from the eleventh thoracic vertebrae.', 
     'rib_right_12':'The right twelfth rib is the lowest rib, from the twelfth thoracic vertebrae.', 
     'rib_right_2':'The right second rib, from the second thoracic vertebrae.', 
     'rib_right_3':'The right third rib, from the third thoracic vertebrae.', 
     'rib_right_4':'The right fourth rib, from the fourth thoracic vertebrae.', 
     'rib_right_5':'The right fifth rib, from the fifth thoracic vertebrae.', 
     'rib_right_6':'The right sixth rib, from the sixth thoracic vertebrae.', 
     'rib_right_7':'The right seventh rib, from the seventh thoracic vertebrae.', 
     'rib_right_8':'The right eighth rib,from the eighth thoracic vertebrae.', 
     'rib_right_9':'The right ninth rib, from the ninth thoracic vertebrae.', 
     'sacrum':'The sacrum is a large, triangular bone at the base of the spine.', 
     'scapula_left':'The left scapula (shoulder blade), is a flat, triangular bone located on the upper left side of the back, connecting the left humerus and the left clavicle.', 
     'scapula_right':'The right scapula (shoulder blade), is a flat, triangular bone located on the upper right side of the back, connecting the right humerus and the right clavicle.', 
     'small_bowel':'The small bowel is a long, coiled organ in the digestive system, located between the stomach and the large intestine. ', 
     'spleen':'The spleen is an organ located in the upper left part of the abdomen, under the rib cage.', 
     'stomach':'The stomach, a pouch-like organ located in the upper part of the abdominal cavity, connects the esophagus with the small intestine, which is a muscular organ located on the left size of the upper abdomen, near with liver, spleen and pancreas.', 
     'trachea':'The trachea is a tube connect to lungs.', 
     'urinary_bladder':'The urinary bladder is a hollow muscular organ, located in the lower abdomen and connect to kidneys.', 
     'vertebrae_C1':'The first cervical vertebra.', 
     'vertebrae_C2':'The second cervical vertebra.', 
     'vertebrae_C3':'The third cervical vertebra.', 
     'vertebrae_C4':'The fourth cervical vertebra.', 
     'vertebrae_C5':'The fifth cervical vertebra.', 
     'vertebrae_C6':'The sixth cervical vertebra.',
     'vertebrae_C7':'The seventh cervical vertebra.', 
     'vertebrae_L1':'The first lumbar vertebra.', 
     'vertebrae_L2':'The second lumbar vertebra.', 
     'vertebrae_L3':'The third lumbar vertebra.', 
     'vertebrae_L4':'The fouth lumbar vertebra.', 
     'vertebrae_L5':'The fifth lumbar vertebra.', 
     'vertebrae_T1':'The first thoracic vertebra, near the left first rib and right first rib.', 
     'vertebrae_T10':'The tenth thoracic vertebra, near the left tenth rib and right tenth rib.', 
     'vertebrae_T11':'The eleventh thoracic vertebra, near the left eleventh rib and right eleventh rib.', 
     'vertebrae_T12':'The twelfth thoracic vertebra, near the left twelfth rib and right twelfth rib.', 
     'vertebrae_T2':'The second thoracic vertebra, near the left second rib and right second rib.', 
     'vertebrae_T3':'The third thoracic vertebra, near the left third rib and right third rib.', 
     'vertebrae_T4':'The fourth thoracic vertebra, near the left fourth rib and right fourth rib.', 
     'vertebrae_T5':'The fifth thoracic vertebra, near the left fifth rib and right fifth rib.', 
     'vertebrae_T6':'The sixth thoracic vertebra, near the left sixth rib and right sixth rib.', 
     'vertebrae_T7':'The seventh thoracic vertebra, near the left seventh rib and right seventh rib.', 
     'vertebrae_T8':'The eighth thoracic vertebra, near the left eighth rib and right eighth rib.', 
     'vertebrae_T9':'The ninth thoracic vertebra, near the left ninth rib and right ninth rib.',

     'adrenal_glands':'Adrenal Glands are a pair of triangular organs located on top of each kidney.',
     'autochthon muscles':'Autochthon Muscles are the deep muscles of the back',
     'clavicles':'Clavicles connect the upper limbs to the trunk.',
     'femurs':'Femurs (thigh bones) are the longest and strongest bones in the body.',
     'gluteus muscles':'Gluteus muscles are the muscles of the buttocks.',
     'hips':'The hip joints are near with the femurs and pelvic bones.',
     'humerus':'The humerus are the upper arm bone.',
     'iliac arteries':'The iliac arteries are the major arteries in the pelvis.',
     'iliac veins':'The iliac veins are the major veins in the pelvis.',
     'iliopsoas muscles':'The iliopsoas muscles are the deep hip flexor muscles.',
     'kidneys':'The kidnes are the bean-shaped organs, located in the lower back.',
     'lungs':'The lungs are the primary organs in the thoracic.',
     'scapulas':'The scapulas connect the clavicle and humerus in the arms.',
     'vertebrae_C':'The cervical vertebrae located in the neck region',
     'vertebrae_T':'The thoracic vertebrae located in the chest region',
     'vertebrae_L':'The lumbar vertebrae located in the lower back',
     'heart':'The central organ of the cardiovascular system, located in the thoracic cavity.',

     'brain and face':'The brain and face are near head, at the top of the body.',
     'vessels':'The vessles are the network of blood vessels (arteries, veins, capillaries)',
     'abdominal organs':'Organs located in the abdominal cavity',
     'thoracic organs':'Organs located in the thoracic cavity',
     'organs':'all organs in the body',
     'vertebraes':'The entire vertebral column',
     'muscles':'all muscles in the body, interact with bones and vessels',
     'ribs':'All ribs in the body, interact with the thoracic vertebrae and muscles.',
     'bones':'The skeletal framework of the body, interact with all body tissues.',

     'cardiovascular system':'cardiovascular system is responsible for circulating blood throughout the body.',
     'gastrointestinal tract':'gastrointestinal tract is responsible for digestion and absorption of nutrients.',
     'respiratory system':'Respiratory system is responsible for gas exchange.',
     'urinary system':'urinary system is responsible for excretion of waste products',
     'digestive system':'digestive system is responsible for breaking down food, absorbing nutrients and removing waste.',
     'musculoskeletal system':'musculoskeletal system contain muscles and bones, responsible for body movement, support and protection.',
}