背景
1.数据处理
0)Pyradiomics
【影像组学pyradiomics教程】(二) 简单的使用-调用类方法
【影像组学pyradiomics教程】(三) 使用配置文件进行特征提取
1)重采样
def resampleSize(sitkImage, newsize):
# 重采样函数
euler3d = sitk.Euler3DTransform()
xsize, ysize, zsize = sitkImage.GetSize()
xspacing, yspacing, zspacing = sitkImage.GetSpacing()
new_spacing_x = xspacing / (newsize[0] / float(xsize))
new_spacing_y = yspacing / (newsize[0] / float(ysize))
new_spacing_z = zspacing / (newsize[0] / float(zsize))
origin = sitkImage.GetOrigin()
direction = sitkImage.GetDirection()
# 根据新的spacing 计算新的size
newspace = (new_spacing_x, new_spacing_y, new_spacing_z)
sitkImage = sitk.Resample(sitkImage, newsize, euler3d, sitk.sitkNearestNeighbor, origin, newspace, direction)
shapeImage = sitkImage.GetSize()
print(f'shape of resampledImage: {shapeImage}')
return sitkImage
基于数组做线性插值的scipy.ndimage.interpolation.zoom
from scipy.ndimage import zoom
from numpy import *
import SimpleITK as sitk
import numpy as np
import torch
img_pth = 'heart_004.nii.gz'
img = sitk.ReadImage(img_pth)
array3D = sitk.GetArrayFromImage(img)
array3D = array3D.transpose(2, 1, 0)
x, y, z = array3D.shape
# one-hot
# 设置类别的数量
num_classes = 4
one_hot = torch.ones((num_classes, 112, 112, 112))
for i in range(num_classes):
tmp = torch.ones(array3D.shape)
tmp[array3D != i] = 0
# zoom + round
zoom_tmp = zoom(tmp, 0.5, order=1)
# # 四舍五入
# newArray3D = np.around(newArray3D)
# # 向下取整
# newArray3D = np.floor(newArray3D)
# 向上取整
round_tmp = np.ceil(zoom_tmp)
round_tmp = torch.from_numpy(round_tmp)
one_hot[i] = round_tmp
print(one_hot.shape)
# label
label = torch.zeros((112, 112, 112))
for i in range(num_classes):
label[one_hot[i] == 1] = i
# label = torch.topk(one_hot, 1)[1].squeeze()
print(label.shape)
newArray3D = label.numpy()
print(newArray3D.shape)
newArray3D = newArray3D.transpose(2, 1, 0)
new_img = sitk.GetImageFromArray(newArray3D)
sitk.WriteImage(new_img, '04_112.nii.gz')
2)Padding
# padding
# 先判断是否是偶数长度,不是则先单向填充
if imgCrop.shape[0] % 2 == 1:
imgCrop = np.pad(imgCrop, [(1, 0), (0, 0), (0, 0)],
mode='constant', constant_values=0)
if imgCrop.shape[1] % 2 == 1:
imgCrop = np.pad(imgCrop, [(0, 0), (1, 0), (0, 0)],
mode='constant', constant_values=0)
if imgCrop.shape[2] % 2 == 1:
imgCrop = np.pad(imgCrop, [(0, 0), (0, 0), (1, 0)],
mode='constant', constant_values=0)
# 再分别从z,x,y轴正负方向+1填充0
while imgCrop.shape[0] < paddingSize[2]:
imgCrop = np.pad(imgCrop, [(1, 1), (0, 0), (0, 0)],
mode='constant', constant_values=0)
while imgCrop.shape[1] < paddingSize[1]:
imgCrop = np.pad(imgCrop, [(0, 0), (1, 1), (0, 0)],
mode='constant', constant_values=0)
while imgCrop.shape[2] < paddingSize[0]:
imgCrop = np.pad(imgCrop, [(0, 0), (0, 0), (1, 1)],
mode='constant', constant_values=0)
3)扩充数据集
心脏相关
3)人工标注
1. 3D Slicer
4)叠加标签
5)独热 <--> 顺序编码
2. 3D分割网络
1)2D --> 3D
上采样函数:
nn.ConvTranspose2d和nn.Upsample的区别
转置卷积(ConvTranspose):有参数可以训练,上采样力度最大的,所以有些时候的结果看起来会不太真实。任务:GAN,分割,超分。
Upsample:没有参数可训练,往往会在其后接一个conv进行学习,采取给定策略进行上采样。任务:超分,目标检测。
2)Transformer
- UNETR
paper: UNETR: Transformers for 3D Medical Image Segmentation
code: research-contributions/UNETR
analysis: [深度学习论文笔记]UNETR
- nnFormer
paper:nnFormer: Interleaved Transformer for Volumetric Segmentation
code:nnFormer
nnFormer (Not-aNother transFORMER): 基于交叉Transformer结构的3D医疗影像分割网络
3)nnUNet详细教程
论文解读- nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation(附实现教程)
4)时序信息:3D --> 4D
(1)借鉴:视频目标分割(Video Object Segmentaion)
-
综述:视频目标分割总结
- 基于运动(传播)的方法
- 基于检测(在线学习)的方法
- 基于匹配的方法
-
常用方法介绍
- RANet:基于传播 + 匹配
paper: RANet: Ranking Attention Network for Fast Video Object Segmentation
code: Storife / RANet
analysis: [ICCV19论文笔记]Ranking Attention-适用于视频分割的全新Attention机制
- STM:基于匹配的Memory
paper: Video Object Segmentation using Space-Time Memory Networks
code: seoungwugoh/STM
analysis: 《Video Object Segmentation using Space-Time Memory Networks》论文阅读笔记
- TransVOS
paper: TransVOS: Video Object Segmentation with Transformers
code: sallymmx / TransVOS
analysis: 听说你还在设计memory?简单高效的视频目标分割网络TransVOS来啦
(2)借鉴:目标追踪(Visual Object Tracking)
-
综述:主流网络模型之目标跟踪
1.基于相关滤波(Correlation Filter)
2.基于卷积神经网络(CNN)
-
网络模型:
目标追踪:FCNT、GOTURN、SiamFC、SiamRPN、SiamRPN++
-
医学影像上的应用
(3)借鉴:域适应网络
(4)借鉴:半监督分割
(5)借鉴:4D Segmentation
Segmentation of 4D images via space-time neural networks、
(6)借鉴:对比学习
3.3D分类网络
4.域迁移(CT2MRI)
5.训练
1)损失函数
Loss functions for image segmentation
2)优化器
3)训练/测试集分布不一致
6.推断&评估
常用的医学图像分割评价指标
7.Tricks
减少过拟合的方法总结
8.结果整理
Feature map热力图可视化方法
计算机视觉可解释性——CAM热力图的研读与复现
Grad-CAM简介
Grad-cam实现
三维
Matplotlib 多图显示 重叠显示
三维激活图展示(见下)
9.改进
注意力模块
注意力的理解心得
加入先验知识
综述:如何给模型加入先验知识