[paper]基于混合网络的WSI自动分割

A hybrid network for automatic hepatocellular carcinoma segmentation in H&E-stained whole slide images

Year: 2021, Author: Wang X, Fang Y, Yang S et al.

Journal Name: Medical Image（H-INDEX = 135）

Link: 论文地址（Sci-Hub）

0. Summary

This paper uses three-branch network architecture to complete segmentation and classification of the WSI images, extensive experiments on the 2019 PAIP Challenge HCC datast and two pathological datasets from CRAG and UHCMC&CWRU Dtaset, get the superior results and good rank in the 2019 Grand Challenge with the method.

1. Research Objective

A fully automated and reliable HCC segmentation system that can siginificantly reduce the burden of pathologists.

2. Background and Problems

• Background

  • Liver cancer is the fourth leading cause of cancer-related death worldwide, accurate segmentation of H&E WSI images has become an indispensable tool for liver cancer surgery evaluation.
  • In recent years, deep learning techniques have exhibited their power in representation learning, and they have also been widely applied in pathological image analysis tasks, but there are lots of obvious defects on the segmentation and classification.

• Problems

  • The appearance of HCC lesions varies greatly.

  • The process of tradition manual segmentation is tedious and time-consuming.

  • The accuracy of tradition method is difficult to ensure.

3. Methods

• Three-branch Network Architecture

  • **The Shared Encoder ** SE-ResNeXt-101, efficient image feature extraction.

• **Three Decoding Branches **
  

  • Br _{tumor_seg}

    The branch of pixel-wise HCC segmentation, focuses on the precise segmentation of tumor regions of tumor-positive patches (ignored the tumor-negative patches and all normal samples), unable to learn full variations of normal tissues.

  • Br _{whole_seg}

    The branch of pixel-wise HCC segmentation, including both the tumor and normal samples, learn the representative features from all the available training data.

  • Br _cls

    The branch of peach-level HCC binary classification, determines input patch has a tumor or not.

• Loss Function

  • L_{tumor_seg}

    The loss function of the Br _{tumor_seg} , deals tumorous patches only, uses a combination of a Dice_loss and a binary cross-entropy loss(bce_loss).
    

    • p_i – the probability of piexl i in the given patch being tumorous.
    • q_i ∈ {0, 1} – the corresponding ground-truth label.
    • ε – a small positive number used for improving numerical stabilities.
    • bce_loss – penalizes the classification error of each pixel.
    • Dice_loss – minimizes the discrepancies between the predicted tumor segmentation and the ground-truth.

  • L_{whole_seg}

    The loss function of the Br _{whole_seg} , both the tumor and normal patches contribute to the training.

    

    • m_j – the probability of pixel j in the given patch being tumorous.
    • n_j ∈ {0, 1} – the corresponding ground-truth label.

  • L_cls – The lossfunction of the Br _cls.
    

    • s_k – the probability of the _k-th patch beijing categorized as having a tumor.
    • t_k ∈ {0, 1} – the ground-truth label of the given patch.
      • t_k = 1 – means there is at least one pixel of the patch belonging to a tumor region.
      • t_k = 0 – means the patch contains only normal tissues.

  • L_final – The total loss function after entire hybird network has jointly trained.

    

    • w_{tumor_seg} , w_{whole_seg} , w_cls – hyperparameters taht control the relative weight of each loss function.
    • w_{tumor_seg} = 1.0, w_{whole_seg} = 2.0, w_cls = 0.2 in this experiment.

4. Experiment Setup and Result

• Datasets

  • 2019 PAIP Challenge
    • 50 training, 10 validation, 40 testing.
    • Training and validation sets are all resected liver specimens, testing set contains both resected and biopsy specimens.
  • CRAG Dataset
    • 173 training, 40 testing.
    • Each image is qpproximately 1512 X 1516 pixels in size.
  • UHCMC&CWRU Dataset
    • 77 training, 33 testing.
    • Each images is approximately 3500 X 2500 pixels in size.

• Experimental Setup

  • Used Otsu method to remove the nonissue regions of the WSI images.
  • Used sliding window scheme with no overlap to crop each WSI into patches of size 256 X 256 pixels.
  • Used standard real-time data augmentation methods (horizontal flipping, vertical flipping, random rescaling, random cropping, random rotation) to make the model invariant to geometric pertubations.
  • Used RandomHSV to randomly change the hue, saturation and value of images in the hue-saturation-value(HSV) color space, to make the model robust to color perturbations.

• Evaluation

  • 2019 PAIP Challenge – used the Jaccard index (Jaccard index < 0.65 -> prediction score = 0).
  • CRAG Dataset – used the Dice score to evaluate model performance.
  • UHCMC&CWRU Dataset – used the Dice score to evaluate model performance.

• Other Settings

  • The size of mini-batch for model training is 32.
  • Used the Adam optimizer as the optimization method.
  • Set the initial learning rate is 0.0003, reduced by a factor of 10 at the 10th and 20th epochs(total 30 training epochs).

• Experimental Process

  • Used sliding window fashion to each testing WSI image to obtain the final segmentation result.
  • First, divided into nonoverlapping patches of size 256 X 256 pixels each.
  • Then, applied the model to each patch separately to obtain its binary tumor segmentation mask according the equation.

  

  • Finally, pieced together the masks from all the patches to obtain the full segmentation result for the original WSI image.

• Results

5. Conclusion

• Prepose a novel multi-task learning architecture with three task-specific branches for pixel-wise HCC segmentation of large H&E-stained WSIs.
• Use ensemble learning strategy that combine two complementary base models to further boost the segmentation accuracy.
• Through extensive experiments on the HCC segmentation dataset and two pathological datasts, demonstrate the superior segmentation results, verifies the effctiveness and robutness of the method.

6. Notes

References

• ResNeXt-101 – Xie, S., Girshick, R., Dollar, P., Tu, Z., He, K., 2017. Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1492–1500.
• scSEM – Roy, A. G., Navab, N., Wachinger, C., 2018. Concurrent spatial and channel squeeze & excitationin fully convolutional networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 421–429.