复现 DemystifyLocalViT Segmentation

3D_DLW

已于 2023-10-05 10:18:17 修改

阅读量1.1k

点赞数 1

分类专栏： # PyTorch 深度学习文章标签：语义分割代码复现

于 2022-04-18 17:29:26 首次发布

本文链接：https://blog.csdn.net/DLW__/article/details/124201467

版权

深度学习同时被 2 个专栏收录

33 篇文章 1 订阅

订阅专栏

PyTorch

14 篇文章 0 订阅

订阅专栏

论文: https://arxiv.org/abs/2106.04263
代码: https://github.com/Atten4Vis/DemystifyLocalViT

前言

本文记录复现 DemystifyLocalViT 在图像语义分割中的应用过程。
作者将该模型语义分割的代码公开在 https://github.com/Atten4Vis/DemystifyLocalViT/tree/master/downstreams/segmentation

复现环境

Ubuntu 18.4
CUDA 11.2
Conda+Python 3.7.11
mmcv 1.4.8
mmsegmentation 0.23.0

环境说明
作者的 DemystifyLocalViT 中的 Segmentation 部分是使用 mmSegmentation 框架实现的。
并且使用的mmSeg框架版本为0.11.0, 这要求mmcv版本在1.1.4~1.3.9之间。
可能由于CUDA版本问题(其他项目依赖, 不能改), 我mmcv-full==1.3.9版本死活装不上去, 因此使用当前(20220416)最新版本的mmcv1.4.8 + mmSeg0.23.0, 然后将作者的代码迁移过来。

环境构建

参考 mmSeg官方文档 (中文)

先使用Conda创建环境, 安装pytorch和mmcv, mmsegmentation

conda create -n mmlab python=3.7.11 -y
conda activate mmlab

conda install pytorch=1.11.0 torchvision cudatoolkit=11.3 -c pytorch
pip install mmcv-full==1.4.8 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.11.0/index.html

# 安装mmsegmentation
git clone https://github.com/open-mmlab/mmsegmentation.git
cd mmsegmentation
pip install -e .
# 链接数据集地址
ln -s <your_dataset_path> data
cd ..

安装完mmSeg之后建议先根据官网教程验证安装。

配置项目

## 克隆DWNet的储存库, 并复制些有用的文件到我们最新版的mmseg中
#git clone https://github.com/Atten4Vis/DemystifyLocalViT.git
## 1 将配置文件复制过来
#cp -r DemystifyLocalViT/downstreams/segmentation/configs/dwnet mmsegmentation/configs
cd mmsegmentation

# 1.下载配置文件 upernet_dynamic_dwnet_tiny_patch4_window7_512x512_160k_ade20k.py
# 注: 这里配置文件名中的window7是指的配置中的窗口大小, 不是运行环境
cd configs
mkdir dwnet
cd dwnet
wget https://raw.githubusercontent.com/Atten4Vis/DemystifyLocalViT/master/downstreams/segmentation/configs/dwnet/upernet_dynamic_dwnet_tiny_patch4_window7_512x512_160k_ade20k.py
cd ../..

# 2.1.下载模型 dwnet.py
cd mmseg/models/backbones
wget https://raw.githubusercontent.com/Atten4Vis/DemystifyLocalViT/master/downstreams/segmentation/mmseg/models/backbones/dwnet.py
# 2.2.在 mmsegmentation/mmseg/models/backbones/__init__.py中完成DWNet的注册。
...
cd ../../..

# 3.下载模型权重文件 upernet_dynamic_dwnet_tiny.pth
mkdir models
cd models
mkdir dwnet
cd dwnet
wget https://github.com/Atten4Vis/DemystifyLocalViT/releases/download/prerelease/upernet_dynamic_dwnet_tiny.pth
cd ../..

# 4.下载数据集
cd data
mkdir ade
cd ade
wget http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip
http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip
unzip ADEChallengeData2016.zip
cd ../..

# 5.执行训练
python tools/train.py configs/dwnet/upernet_dynamic_dwnet_tiny_patch4_window7_512x512_160k_ade20k.py

# 6.执行测试
python tools/test.py configs/dwnet/upernet_dynamic_dwnet_tiny_patch4_window7_512x512_160k_ade20k.py models/dwnet/upernet_dynamic_dwnet_tiny.pth --eval mIoU

模型架构图

以输入128x128大小图像的dynamic_dwnet_tiny模型为例
在这里插入图片描述

测试结果

> python tools/test.py configs/dwnet/upernet_dynamic_dwnet_tiny_patch4_window7_512x512_160k_ade20k.py models/dwnet/upernet_dynamic_dwnet_tiny.pth --eval mIoU

2022-04-18 14:38:07,999 - mmseg - INFO - Multi-processing start method is `None`
2022-04-18 14:38:07,999 - mmseg - INFO - OpenCV num_threads is `<built-in function getNumThreads>
2022-04-18 14:38:08,083 - mmseg - INFO - Loaded 2000 images
/home/dl/mmsegmentation/mmseg/models/losses/cross_entropy_loss.py:226: UserWarning: Default ``avg_non_ignore`` is False, if you would like to ignore the certain label and average loss over non-ignore labels, which is the same with PyTorch official cross_entropy, set ``avg_non_ignore=True``.
  'Default ``avg_non_ignore`` is False, if you would like to '
load checkpoint from local path: models/dwnet/upernet_dynamic_dwnet_tiny.pth
/home/dl/mmsegmentation/tools/test.py:249: UserWarning: SyncBN is only supported with DDP. To be compatible with DP, we convert SyncBN to BN. Please use dist_train.sh which can avoid this error.
  'SyncBN is only supported with DDP. To be compatible with DP, '
[>>>>>>>>>>>>>>>>>>>>>>>>>>>] 2000/2000, 3.0 task/s, elapsed: 665s, ETA:     0sper class results:

+---------------------+-------+-------+
|        Class        |  IoU  |  Acc  |
+---------------------+-------+-------+
|         wall        | 74.51 | 86.95 |
|       building      | 81.71 | 91.64 |
|         sky         | 93.84 | 97.21 |
|        floor        | 78.77 | 90.32 |
|         tree        | 71.87 | 87.01 |
|       ceiling       | 81.25 | 90.56 |
|         road        | 81.97 | 89.64 |
|         bed         | 87.05 | 95.01 |
|      windowpane     | 59.56 | 76.34 |
|        grass        | 65.73 | 83.84 |
|       cabinet       | 57.41 | 69.92 |
|       sidewalk      | 62.65 | 78.48 |
|        person       | 77.81 | 90.52 |
|        earth        | 32.16 | 45.79 |
|         door        | 42.19 | 55.54 |
|        table        | 54.19 | 71.68 |
|       mountain      | 53.22 | 72.77 |
|        plant        | 51.14 | 60.59 |
|       curtain       |  67.9 | 81.51 |
|        chair        | 53.13 | 68.15 |
|         car         | 82.34 | 91.19 |
|        water        | 46.18 | 59.28 |
|       painting      |  67.1 | 86.53 |
|         sofa        | 60.57 | 80.31 |
|        shelf        | 41.55 | 56.79 |
|        house        | 50.66 | 67.21 |
|         sea         |  42.3 | 63.55 |
|        mirror       |  59.5 | 66.68 |
|         rug         | 50.02 | 58.25 |
|        field        | 29.92 | 45.88 |
|       armchair      | 37.08 | 51.43 |
|         seat        | 57.46 |  80.3 |
|        fence        | 36.06 | 50.07 |
|         desk        | 45.57 | 67.58 |
|         rock        | 34.54 | 51.15 |
|       wardrobe      | 43.42 | 62.41 |
|         lamp        | 58.71 | 71.32 |
|       bathtub       | 70.73 | 78.86 |
|       railing       | 32.47 | 45.06 |
|       cushion       |  51.9 | 66.33 |
|         base        | 25.58 | 33.95 |
|         box         |  20.9 | 27.54 |
|        column       |  38.1 | 49.18 |
|      signboard      |  35.5 | 48.11 |
|   chest of drawers  | 39.24 | 54.82 |
|       counter       | 30.97 | 35.94 |
|         sand        | 28.66 |  55.3 |
|         sink        | 68.58 | 76.58 |
|      skyscraper     | 57.75 | 74.37 |
|      fireplace      | 72.86 | 86.27 |
|     refrigerator    | 70.09 | 84.08 |
|      grandstand     | 39.06 | 61.05 |
|         path        | 19.23 |  26.7 |
|        stairs       | 28.09 | 35.65 |
|        runway       | 65.81 | 87.47 |
|         case        | 44.03 |  58.9 |
|      pool table     | 90.51 | 94.46 |
|        pillow       | 50.98 | 60.66 |
|     screen door     | 59.52 | 73.15 |
|       stairway      | 30.88 | 35.85 |
|        river        | 10.58 | 23.88 |
|        bridge       | 62.17 | 74.15 |
|       bookcase      | 36.94 | 53.49 |
|        blind        | 40.64 | 44.84 |
|     coffee table    | 53.48 | 77.98 |
|        toilet       | 82.43 | 89.29 |
|        flower       | 41.28 | 57.67 |
|         book        |  42.8 | 65.98 |
|         hill        | 10.69 |  14.6 |
|        bench        | 38.36 | 49.08 |
|      countertop     | 54.45 | 71.18 |
|        stove        | 75.66 | 83.81 |
|         palm        | 49.91 |  70.4 |
|    kitchen island   | 33.69 | 59.09 |
|       computer      | 66.19 | 78.67 |
|     swivel chair    | 44.92 |  60.9 |
|         boat        | 68.29 | 78.67 |
|         bar         | 27.72 | 33.96 |
|    arcade machine   | 44.97 | 47.79 |
|        hovel        | 31.79 | 35.37 |
|         bus         | 81.86 | 89.83 |
|        towel        | 56.75 | 65.18 |
|        light        | 52.41 | 59.77 |
|        truck        | 28.21 | 35.42 |
|        tower        | 27.77 |  35.0 |
|      chandelier     | 66.25 | 78.44 |
|        awning       | 22.87 |  27.1 |
|     streetlight     | 23.24 | 28.77 |
|        booth        | 49.58 | 52.85 |
| television receiver | 65.01 | 75.92 |
|       airplane      | 57.94 | 65.04 |
|      dirt track     |  7.45 |  16.1 |
|       apparel       | 31.15 |  47.0 |
|         pole        | 20.06 | 25.56 |
|         land        |  4.21 |  7.02 |
|      bannister      | 10.05 | 14.81 |
|      escalator      | 23.01 | 26.02 |
|       ottoman       | 42.57 |  56.1 |
|        bottle       | 32.97 | 48.27 |
|        buffet       | 46.11 |  54.5 |
|        poster       | 28.57 | 38.07 |
|        stage        | 15.52 | 24.34 |
|         van         | 41.29 | 60.72 |
|         ship        | 45.55 | 47.46 |
|       fountain      | 19.45 | 20.15 |
|    conveyer belt    | 69.46 | 90.24 |
|        canopy       | 11.18 | 14.57 |
|        washer       | 62.97 | 67.66 |
|      plaything      | 15.63 | 25.52 |
|    swimming pool    | 38.25 | 68.51 |
|        stool        |  37.8 | 49.99 |
|        barrel       | 40.56 | 66.37 |
|        basket       | 27.09 | 36.52 |
|      waterfall      | 54.61 | 63.26 |
|         tent        | 84.77 | 97.99 |
|         bag         | 10.94 | 13.71 |
|       minibike      | 51.07 | 58.24 |
|        cradle       | 79.27 | 93.83 |
|         oven        | 43.34 | 62.98 |
|         ball        | 39.68 | 45.41 |
|         food        | 52.07 | 62.52 |
|         step        |  9.94 | 11.85 |
|         tank        | 55.94 | 59.83 |
|      trade name     | 29.85 | 37.78 |
|      microwave      |  74.1 | 81.89 |
|         pot         | 38.97 | 48.09 |
|        animal       | 56.87 | 60.89 |
|       bicycle       | 54.09 | 72.34 |
|         lake        | 48.25 | 55.53 |
|      dishwasher     | 55.37 |  71.9 |
|        screen       | 67.28 | 84.68 |
|       blanket       | 12.37 | 14.05 |
|      sculpture      | 49.82 | 60.43 |
|         hood        | 55.38 | 61.45 |
|        sconce       | 40.37 | 50.29 |
|         vase        | 32.64 | 46.22 |
|    traffic light    | 29.17 | 47.19 |
|         tray        |  3.72 |  7.58 |
|        ashcan       |  35.5 | 44.42 |
|         fan         | 55.26 | 68.46 |
|         pier        | 49.24 | 66.17 |
|      crt screen     |  2.66 |  7.45 |
|        plate        | 51.61 | 64.61 |
|       monitor       |  5.19 |  6.29 |
|    bulletin board   |  41.7 |  50.1 |
|        shower       |  1.09 |  4.05 |
|       radiator      | 51.56 | 58.36 |
|        glass        |  10.9 | 12.11 |
|        clock        | 30.79 | 37.01 |
|         flag        |  31.0 | 36.27 |
+---------------------+-------+-------+
Summary:

+-------+-------+-------+
|  aAcc |  mIoU |  mAcc |
+-------+-------+-------+
| 81.15 | 45.72 | 57.06 |
+-------+-------+-------+

Process finished with exit code 0