01 Nodule Segmentation （Identification of lung cancer lesions）

 Basic knowledge of segmentation

Training process of segmentation model

1. raw data
2. Data processing: data cleaning, data loading, data conversion
3. Cutting images: finding problematic pixels in full pixel space
4. Candidate grouping: grouping problematic points
5. Candidate classification: Classify images of nodules to determine if they are tumors
6. Malignant judgment: further determine whether the image judged as a tumor is malignant or not

分割模型训练过程

1. 原始数据
2. 数据处理：数据清洗，数据加载，数据转化
3. 切割图像：在全像素空间中寻找有问题的像素
4. 候选分组：对有问题的点位进行分组
5. 候选分类：对结节的图像进行分类，确定是否是肿瘤
6. 恶性判断：对判断为肿瘤的图像进一步判断是否为恶性肿瘤

Nodule segmentation process

1. Use the open-source segmentation model U-Net. There is no need to write your own model. Open source models are the main way to handle business problems in work.

2. Adapt the model to the data of our project. Due to the fact that U-Net was originally prepared for 2D images, but our project data is 3D data, some modifications are needed:

• Update the model. Integrating the U-Net model into our code can run well
• Modify the dataset. Build a usable dataset for the model, and the segmentation model needs to output a segmented image, not just the classification results
• Modify the training cycle. Using a new loss function to adapt the image results output by the model

3. Observation results.

结节分割流程

1. 使用开源的分割模型U-Net。不需要自己编写模型，开源模型是工作中处理业务问题的主要方式。

2. 调整模型适配我们项目的数据。由于U-Net原本是为二维图像准备的，但我们的项目数据是三维数据，所以需要一些修改：

• 更新模型。将U-Net模型融合到我们的代码里，能跑起来。
• 修改数据集。为模型构建一套可以使用的数据集，分割模型需要输出被分割的一块图像，并不仅仅是分类结果。
• 修改训练循环。使用新的损失函数来适配模型输出的图像结果

3. 观察结果

Definition of Object Segmentation

Object segmentation refers to the process of subdividing a digital image into multiple image subregions (the sets of pixels), and the features of the same subregions show significant differences.

Semantic segmentation

assigning a category label to each pixel in an image.

Instance segmentation

Compared to semantic segmentation, it is not only necessary to distinguish pixels of different categories, but also to distinguish different individuals of the same category.

图像分割的定义: 图像分割是指将数字图像细分为多个图像子区域（像素的集合）的过程，并且同一个子区域的特征呈现较为明显的差异。

语义分割：把图像中的每一个像素赋予一个类别标签

实例分割：相对于语义分割来说，不仅要区分不同类别的像素，还需要对同一类不同个体进行区分。

---

Introduction to U-Net Model

The main structure of Unet network includes three parts: encoder, decoder, and bottleneck layer.

(1) Encoder section: includes three program blocks. Each program block includes a 3 * 3 convolution (using the Relu activation function) and a 2 * 2 pooling layer in steps of 2. After processing each program block, the feature map gradually shrinks.

(2) Bottleneck layer section: Contains two 3 * 3 convolutional layers.

(3) Decoder section: Symmetric to the encoder section, including three program blocks, each of which includes a 2 * 2 upsampling operation with a step of 2, followed by feature mapping concatenation with the encoding section, and finally through two 3 * 3 convolutions (using the Relu activation function).

The final output layer contains a 1 * 1 convolutional layer, followed by a sigmoid activation function, to generate a two-dimensional fault probability image.

Unet网络的主要结构包括了编码器、解码器、瓶颈层三个部分。

1. 编码器部分：包括了三个程序块。每个程序块都包括3*3的卷积（使用Relu激活函数），步幅为2的2*2的池化层。每个程序块处理后，特征图逐步缩小。
2. 瓶颈层部分：包含了两个3*3的卷积层。
3. 解码器部分：与编码器部分对称，也包括三个程序块，每个程序块包括步幅为2的2*2的上采样操作，然后与编码部分进行特征映射级联（concatenate），最后通过两个3*3的卷积（使用Relu激活函数）。

最终输出层包含一个1*1的卷积层，其后紧跟sigmoid激活函数，生成二维断层概率图像。