基于PaddleSeg的钢筋长度超限监控

已于 2023-03-21 21:13:56 修改
阅读量238 收藏 1
点赞数
分类专栏： 算法 人工智能 文章标签： 计算机视觉 人工智能 深度学习
于 2023-03-14 23:17:13 首次发布
原文链接：https://aistudio.baidu.com/aistudio/projectdetail/5712507?forkThirdPart=1
版权

★★★ 本文源自AlStudio社区精品项目，【点击此处】查看更多精品内容 >>>

基于PaddleSeg的钢筋长度超限监控

1. 项目介绍

钢铁厂在生产钢筋的过程中，会存在部分钢筋长度超限的问题，如果不进行处理，容易造成机械臂损伤。因此，需要通过质检流程，筛选出存在长度超限问题的钢筋批次，并进行预警。传统的处理方式是人工核查，一方面增加了人工成本，拉低了生产效率，另一方面也要求工人师傅对业务比较熟练，能够准确地判断钢筋长度是否超限，存在一定的误判率。在AI时代，利用深度学习技术，可以实现端到端全自动的钢筋长度超限监控，从而降低人工成本，提高生产效率。整体技术方案可以归纳为如下步骤：

  1. 在钢筋一侧安装摄像头，拍摄图像；
  2. 利用图像分割技术提取钢筋掩膜；
  3. 根据摄像头位置和角度确定长度界限；
  4. 最后根据该长度界限和钢筋分割范围的几何关系判断本批次钢筋是否超限。

动画演示分割过程

1.1 痛点问题

钢筋长度超限监控问题，传统质检方式需要专人介入，自动化程度低，且存在一定的误判率，影响钢筋生产的效率和质量。该业务场景背景复杂、钢筋数量、形状、超限程度不一，导致利用模型进行图像分割难度较大。而钢筋超限会导致机器损伤，对于超限情况需要及时响应，避免损伤发生，因此对模型精度与速度要求非常高。

1.2 解决方案

钢筋超限监控问题可以转换为图像分割后的几何判断问题，而为了实现图像分割，我们使用提供了全流程分割方案的PaddleSeg开发套件，只需简单地修改配置文件，就可以进行模型训练，获得高精度的分割效果。进一步地，我们挑选使用精度和速度平衡的PP-LiteSeg，实现高精度的同时，满足工业部署的要求。

1.3 模型简介

轻量级语义分割模型 PP-LiteSeg，包含以下几个关键性模块改进：

  • FLD(Flexible and Lightweight Decoder)，是一种灵活且轻量级的解码器，用于降低传统解码器模块的计算负载；
  • UAFM(Unified Attention Fusion Module)，是一个统一注意力融合模块，采用空域与通道注意力生成用于特征融合的权值；
  • SPPM(Simple Pyramid Pooling Module)，是一个简单的金字塔池化模块，以极低的计算消耗进行全局上下文信息聚合。

相比其他方案，PP-LiteSeg取得了精度与速度之间的平衡，其结构如下：

2. 安装环境

  • Python >= 3.6
  • PaddlePaddle >= 2.1
  • PaddleSeg
# 将 PaddleSeg git clone 到本地
%cd /home/aistudio/work
# !git clone https://github.com/PaddlePaddle/PaddleSeg
# 由于github国内访问较慢，因此，也可以使用gitee
# !git clone https://gitee.com/PaddlePaddle/PaddleSeg
# checkout到v2.6
# !git checkout release/2.6
!git clone --branch release/2.6 --depth 1 https://gitee.com/PaddlePaddle/PaddleSeg.git  # 设置depth为1，可以免去下载无用的历史版本记录，加快clone速度

# 切换到PaddleSeg目录
%cd /home/aistudio/work/PaddleSeg
# 安装项目所需的依赖
# !pip install -U pip wheel
!pip install -r requirements.txt

3. 处理数据

# 查看当前挂载的数据集目录，并解压。
%cd /home/aistudio/data/ 
!tar xf data196420/dataset.tar.xz

# 切换到工作目录
%cd /home/aistudio/work/PaddleSeg

/home/aistudio/work/PaddleSeg

可借助PaddleSeg中tools目录下的脚本labelme2seg.py，将labelme格式标注转换成paddleseg支持的格式。

# 运行`labelme2seg.py`
!python /home/aistudio/work/PaddleSeg/tools/labelme2seg.py ~/data/dataset

使用PaddleSeg提供的脚本（split_dataset_list.py）将数据集划分为训练集、验证集和测试集。

!python tools/split_dataset_list.py ~/data/dataset . annotations --split 0.7 0.15 0.15

4. 模型训练

此处我们选择轻量级语义分割模型PP-LiteSeg模型，对钢筋进行分割。具体介绍可参考PP-LiteSeg的README说明文件。为了在自定义数据集上使用PP-LiteSeg模型，需要对PaddleSeg提供的默认配置文件（PaddleSeg/configs/pp_liteseg/pp_liteseg_stdc1_cityscapes_1024x512_scale0.5_160k.yml）进行轻微修改。

如下所示，添加自定义数据集路径、类别数等信息。

batch_size: 4  # total: 4*4
iters: 2000

optimizer:
  type: sgd
  momentum: 0.9
  weight_decay: 5.0e-4

lr_scheduler:
  type: PolynomialDecay
  end_lr: 0
  power: 0.9
  warmup_iters: 100
  warmup_start_lr: 1.0e-5
  learning_rate: 0.005

loss:
  types:
    - type: OhemCrossEntropyLoss
      min_kept: 130000   # batch_size * 1024 * 512 // 16
    - type: OhemCrossEntropyLoss
      min_kept: 130000
    - type: OhemCrossEntropyLoss
      min_kept: 130000
  coef: [1, 1, 1]

train_dataset:
  type: Dataset
  dataset_root: /home/aistudio/data/dataset
  train_path: /home/aistudio/data/dataset/train.txt
  num_classes: 2
  transforms:
    - type: ResizeStepScaling
      min_scale_factor: 0.125
      max_scale_factor: 1.5
      scale_step_size: 0.125
    - type: RandomPaddingCrop
      crop_size: [1024, 512]
    - type: RandomHorizontalFlip
    - type: RandomDistort
      brightness_range: 0.5
      contrast_range: 0.5
      saturation_range: 0.5
    - type: Normalize
  mode: train

val_dataset:
  type: Dataset
  dataset_root: /home/aistudio/data/dataset
  val_path: /home/aistudio/data/dataset/val.txt
  num_classes: 2
  transforms:
    - type: Normalize
  mode: val

test_config:
  aug_eval: True
  scales: 0.5

model:
  type: PPLiteSeg
  backbone:
    type: STDC1
    pretrained: https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
  arm_out_chs: [32, 64, 128]
  seg_head_inter_chs: [32, 64, 64]


!python3 train.py --config /home/aistudio/work/pp_liteseg_stdc1.yml \
    --use_vdl \
    --save_dir output/mask_iron \
    --save_interval 500 \
    --log_iters 100 \
    --num_workers 8 \
    --do_eval \
    --keep_checkpoint_max 10

/home/aistudio/work/PaddleSeg/paddleseg/models/losses/rmi_loss.py:78: DeprecationWarning: invalid escape sequence \i
  """
2023-03-06 11:12:52 [INFO]	
------------Environment Information-------------
platform: Linux-4.15.0-140-generic-x86_64-with-debian-stretch-sid
Python: 3.7.4 (default, Aug 13 2019, 20:35:49) [GCC 7.3.0]
Paddle compiled with cuda: True
NVCC: Build cuda_11.2.r11.2/compiler.29618528_0
cudnn: 8.2
GPUs used: 1
CUDA_VISIBLE_DEVICES: None
GPU: ['GPU 0: Tesla V100-SXM2-32GB']
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~16.04) 7.5.0
PaddleSeg: 2.6.0
PaddlePaddle: 2.4.0
OpenCV: 4.6.0
------------------------------------------------
2023-03-06 11:12:53 [INFO]	
---------------Config Information---------------
batch_size: 4
iters: 2000
loss:
  coef:
  - 1
  - 1
  - 1
  types:
  - ignore_index: 255
    min_kept: 130000
    type: OhemCrossEntropyLoss
  - ignore_index: 255
    min_kept: 130000
    type: OhemCrossEntropyLoss
  - ignore_index: 255
    min_kept: 130000
    type: OhemCrossEntropyLoss
lr_scheduler:
  end_lr: 0
  learning_rate: 0.005
  power: 0.9
  type: PolynomialDecay
  warmup_iters: 100
  warmup_start_lr: 1.0e-05
model:
  arm_out_chs:
  - 32
  - 64
  - 128
  backbone:
    pretrained: https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
    type: STDC1
  seg_head_inter_chs:
  - 32
  - 64
  - 64
  type: PPLiteSeg
optimizer:
  momentum: 0.9
  type: sgd
  weight_decay: 0.0005
test_config:
  aug_eval: true
  scales: 0.5
train_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: train
  num_classes: 2
  train_path: /home/aistudio/data/dataset/train.txt
  transforms:
  - max_scale_factor: 1.5
    min_scale_factor: 0.125
    scale_step_size: 0.125
    type: ResizeStepScaling
  - crop_size:
    - 1024
    - 512
    type: RandomPaddingCrop
  - type: RandomHorizontalFlip
  - brightness_range: 0.5
    contrast_range: 0.5
    saturation_range: 0.5
    type: RandomDistort
  - type: Normalize
  type: Dataset
val_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: val
  num_classes: 2
  transforms:
  - type: Normalize
  type: Dataset
  val_path: /home/aistudio/data/dataset/val.txt
------------------------------------------------
W0306 11:12:53.040948  2444 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0306 11:12:53.041000  2444 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
2023-03-06 11:12:54 [INFO]	Loading pretrained model from https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
Connecting to https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
Downloading PP_STDCNet1.tar.gz
[==================================================] 100.00%
Uncompress PP_STDCNet1.tar.gz
[==================================================] 100.00%
2023-03-06 11:13:00 [INFO]	There are 145/145 variables loaded into STDCNet.
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/nn/layer/norm.py:712: UserWarning: When training, we now always track global mean and variance.
  "When training, we now always track global mean and variance."
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/math_op_patch.py:277: UserWarning: The dtype of left and right variables are not the same, left dtype is paddle.float32, but right dtype is paddle.int64, the right dtype will convert to paddle.float32
  .format(lhs_dtype, rhs_dtype, lhs_dtype))
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:13:25 [INFO]	[TRAIN] epoch: 3, iter: 100/2000, loss: 2.1926, lr: 0.004950, batch_cost: 0.2491, reader_cost: 0.06910, ips: 16.0550 samples/sec | ETA 00:07:53
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:13:49 [INFO]	[TRAIN] epoch: 6, iter: 200/2000, loss: 1.8833, lr: 0.004765, batch_cost: 0.2382, reader_cost: 0.08182, ips: 16.7906 samples/sec | ETA 00:07:08
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:14:11 [INFO]	[TRAIN] epoch: 9, iter: 300/2000, loss: 1.7562, lr: 0.004526, batch_cost: 0.2207, reader_cost: 0.06289, ips: 18.1201 samples/sec | ETA 00:06:15
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:14:34 [INFO]	[TRAIN] epoch: 11, iter: 400/2000, loss: 1.5520, lr: 0.004286, batch_cost: 0.2253, reader_cost: 0.03780, ips: 17.7563 samples/sec | ETA 00:06:00
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:14:58 [INFO]	[TRAIN] epoch: 14, iter: 500/2000, loss: 1.6580, lr: 0.004044, batch_cost: 0.2409, reader_cost: 0.06312, ips: 16.6059 samples/sec | ETA 00:06:01
2023-03-06 11:14:58 [INFO]	Start evaluating (total_samples: 32, total_iters: 32)...
 3/32 [=>............................] - ETA: 14s - batch_cost: 0.5002 - reader cost: 0.4586Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
32/32 [==============================] - 7s 231ms/step - batch_cost: 0.2309 - reader cost: 0.2074
2023-03-06 11:15:05 [INFO]	[EVAL] #Images: 32 mIoU: 0.9828 Acc: 0.9936 Kappa: 0.9826 Dice: 0.9913
2023-03-06 11:15:05 [INFO]	[EVAL] Class IoU: 
[0.9915 0.9742]
2023-03-06 11:15:05 [INFO]	[EVAL] Class Precision: 
[0.9957 0.987 ]
2023-03-06 11:15:05 [INFO]	[EVAL] Class Recall: 
[0.9958 0.9868]
2023-03-06 11:15:05 [INFO]	[EVAL] The model with the best validation mIoU (0.9828) was saved at iter 500.
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:15:27 [INFO]	[TRAIN] epoch: 17, iter: 600/2000, loss: 1.1448, lr: 0.003801, batch_cost: 0.2184, reader_cost: 0.05941, ips: 18.3150 samples/sec | ETA 00:05:05
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:15:49 [INFO]	[TRAIN] epoch: 19, iter: 700/2000, loss: 1.0914, lr: 0.003556, batch_cost: 0.2136, reader_cost: 0.04413, ips: 18.7289 samples/sec | ETA 00:04:37
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:16:13 [INFO]	[TRAIN] epoch: 22, iter: 800/2000, loss: 1.3022, lr: 0.003309, batch_cost: 0.2464, reader_cost: 0.06541, ips: 16.2332 samples/sec | ETA 00:04:55
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:16:36 [INFO]	[TRAIN] epoch: 25, iter: 900/2000, loss: 1.0730, lr: 0.003060, batch_cost: 0.2270, reader_cost: 0.06602, ips: 17.6230 samples/sec | ETA 00:04:09
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:16:59 [INFO]	[TRAIN] epoch: 28, iter: 1000/2000, loss: 1.3344, lr: 0.002809, batch_cost: 0.2290, reader_cost: 0.05981, ips: 17.4704 samples/sec | ETA 00:03:48
2023-03-06 11:16:59 [INFO]	Start evaluating (total_samples: 32, total_iters: 32)...
 3/32 [=>............................] - ETA: 18s - batch_cost: 0.6236 - reader cost: 0.5855Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
32/32 [==============================] - 8s 243ms/step - batch_cost: 0.2428 - reader cost: 0.2198
2023-03-06 11:17:07 [INFO]	[EVAL] #Images: 32 mIoU: 0.9767 Acc: 0.9912 Kappa: 0.9763 Dice: 0.9882
2023-03-06 11:17:07 [INFO]	[EVAL] Class IoU: 
[0.9883 0.965 ]
2023-03-06 11:17:07 [INFO]	[EVAL] Class Precision: 
[0.9969 0.9739]
2023-03-06 11:17:07 [INFO]	[EVAL] Class Recall: 
[0.9913 0.9906]
2023-03-06 11:17:07 [INFO]	[EVAL] The model with the best validation mIoU (0.9828) was saved at iter 500.
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:17:27 [INFO]	[TRAIN] epoch: 30, iter: 1100/2000, loss: 1.1206, lr: 0.002555, batch_cost: 0.2058, reader_cost: 0.04158, ips: 19.4394 samples/sec | ETA 00:03:05
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:17:51 [INFO]	[TRAIN] epoch: 33, iter: 1200/2000, loss: 0.9999, lr: 0.002298, batch_cost: 0.2337, reader_cost: 0.06408, ips: 17.1125 samples/sec | ETA 00:03:06
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:18:14 [INFO]	[TRAIN] epoch: 36, iter: 1300/2000, loss: 0.8688, lr: 0.002038, batch_cost: 0.2311, reader_cost: 0.06141, ips: 17.3074 samples/sec | ETA 00:02:41
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:18:35 [INFO]	[TRAIN] epoch: 38, iter: 1400/2000, loss: 0.8655, lr: 0.001775, batch_cost: 0.2123, reader_cost: 0.04386, ips: 18.8383 samples/sec | ETA 00:02:07
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:18:59 [INFO]	[TRAIN] epoch: 41, iter: 1500/2000, loss: 0.8779, lr: 0.001506, batch_cost: 0.2428, reader_cost: 0.07047, ips: 16.4764 samples/sec | ETA 00:02:01
2023-03-06 11:18:59 [INFO]	Start evaluating (total_samples: 32, total_iters: 32)...
 3/32 [=>............................] - ETA: 17s - batch_cost: 0.6126 - reader cost: 0.5622Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
32/32 [==============================] - 8s 243ms/step - batch_cost: 0.2428 - reader cost: 0.2171
2023-03-06 11:19:07 [INFO]	[EVAL] #Images: 32 mIoU: 0.9858 Acc: 0.9947 Kappa: 0.9857 Dice: 0.9928
2023-03-06 11:19:07 [INFO]	[EVAL] Class IoU: 
[0.993  0.9787]
2023-03-06 11:19:07 [INFO]	[EVAL] Class Precision: 
[0.9969 0.9878]
2023-03-06 11:19:07 [INFO]	[EVAL] Class Recall: 
[0.996  0.9906]
2023-03-06 11:19:08 [INFO]	[EVAL] The model with the best validation mIoU (0.9858) was saved at iter 1500.
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:19:31 [INFO]	[TRAIN] epoch: 44, iter: 1600/2000, loss: 0.7487, lr: 0.001233, batch_cost: 0.2284, reader_cost: 0.06946, ips: 17.5115 samples/sec | ETA 00:01:31
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:19:52 [INFO]	[TRAIN] epoch: 46, iter: 1700/2000, loss: 0.7271, lr: 0.000952, batch_cost: 0.2108, reader_cost: 0.04096, ips: 18.9710 samples/sec | ETA 00:01:03
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:20:16 [INFO]	[TRAIN] epoch: 49, iter: 1800/2000, loss: 0.7356, lr: 0.000662, batch_cost: 0.2409, reader_cost: 0.06832, ips: 16.6025 samples/sec | ETA 00:00:48
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:20:39 [INFO]	[TRAIN] epoch: 52, iter: 1900/2000, loss: 0.6789, lr: 0.000356, batch_cost: 0.2301, reader_cost: 0.05793, ips: 17.3814 samples/sec | ETA 00:00:23
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
2023-03-06 11:21:02 [INFO]	[TRAIN] epoch: 55, iter: 2000/2000, loss: 0.6566, lr: 0.000006, batch_cost: 0.2289, reader_cost: 0.06389, ips: 17.4774 samples/sec | ETA 00:00:00
2023-03-06 11:21:02 [INFO]	Start evaluating (total_samples: 32, total_iters: 32)...
 3/32 [=>............................] - ETA: 16s - batch_cost: 0.5517 - reader cost: 0.5198Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
32/32 [==============================] - 8s 238ms/step - batch_cost: 0.2374 - reader cost: 0.2141
2023-03-06 11:21:09 [INFO]	[EVAL] #Images: 32 mIoU: 0.9847 Acc: 0.9943 Kappa: 0.9846 Dice: 0.9923
2023-03-06 11:21:09 [INFO]	[EVAL] Class IoU: 
[0.9924 0.9771]
2023-03-06 11:21:09 [INFO]	[EVAL] Class Precision: 
[0.9967 0.9869]
2023-03-06 11:21:09 [INFO]	[EVAL] Class Recall: 
[0.9957 0.9899]
2023-03-06 11:21:10 [INFO]	[EVAL] The model with the best validation mIoU (0.9858) was saved at iter 1500.
<class 'paddle.nn.layer.conv.Conv2D'>'s flops has been counted
<class 'paddle.nn.layer.norm.BatchNorm2D'>'s flops has been counted
<class 'paddle.nn.layer.activation.ReLU'>'s flops has been counted
<class 'paddle.nn.layer.pooling.AvgPool2D'>'s flops has been counted
<class 'paddle.nn.layer.pooling.AdaptiveAvgPool2D'>'s flops has been counted
Total Flops: 11465470848     Total Params: 8039122

4.1 训练避坑指南

  1. one_hot_kernel 相关的报错，需要注意类别是否正确设置，需要考虑背景类，即num_classes: 实际类 + 1。
Error: /paddle/paddle/phi/kernels/gpu/one_hot_kernel.cu:38 Assertion `p_in_data[idx] >= 0 && p_in_data[idx] < depth` failed. Illegal index value, Input(input) value should be greater than or equal to 0, and less than depth [1], but received [1].
Error: /paddle/paddle/phi/kernels/gpu/one_hot_kernel.cu:38 Assertion `p_in_data[idx] >= 0 && p_in_data[idx] < depth` failed. Illegal index value, Input(input) value should be greater than or equal to 0, and less than depth [1], but received [1].
Error: /paddle/paddle/phi/kernels/gpu/one_hot_kernel.cu:38 Assertion `p_in_data[idx] >= 0 && p_in_data[idx] < depth` failed. Illegal index value, Input(input) value should be greater than or equal to 0, and less than depth [1], but received [1].
  1. one_hot_kernel 另一种报错
Input(input) value should be greater than or equal to 0, and less than depth [5], but received [-1].
Error: /paddle/paddle/phi/kernels/gpu/one_hot_kernel.cu:38 Assertion `p_in_data[idx] >= 0 && p_in_data[idx] < depth` failed. Illegal index value, Input(input) value should be greater than or equal to 0, and less than depth [5], but received [-1].
Error: /paddle/paddle/phi/kernels/gpu/one_hot_kernel.cu:38 Assertion `p_in_data[idx] >= 0 && p_in_data[idx] < depth` failed. Illegal index value, Input(input) value should be greater than or equal to 0, and less than depth [5], but received [-1].
Error: /paddle/paddle/phi/kernels/gpu/one_hot_kernel.cu:38 Assertion `p_in_data[idx] >= 0 && p_in_data[idx] < depth` failed. Illegal index value,

此时需要注意mask中标签是否超过[0, num_classes + 1]的范围。

4.2 数据模型可视化

使用AI Studio自带的基于VisualDL的可视化功能，可以很方便地实现数据可视化。

数据可视化设置

点击启动VisualDL服务，即可在AI Studio上启动VisualDL服务，并开始追踪训练进展情况。

5. 模型评估

!python val.py \
       --config /home/aistudio/work/pp_liteseg_stdc1.yml \
       --model_path output/mask_iron/best_model/model.pdparams

2023-03-06 11:22:01 [INFO]	
---------------Config Information---------------
batch_size: 4
iters: 2000
loss:
  coef:
  - 1
  - 1
  - 1
  types:
  - min_kept: 130000
    type: OhemCrossEntropyLoss
  - min_kept: 130000
    type: OhemCrossEntropyLoss
  - min_kept: 130000
    type: OhemCrossEntropyLoss
lr_scheduler:
  end_lr: 0
  learning_rate: 0.005
  power: 0.9
  type: PolynomialDecay
  warmup_iters: 100
  warmup_start_lr: 1.0e-05
model:
  arm_out_chs:
  - 32
  - 64
  - 128
  backbone:
    pretrained: https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
    type: STDC1
  seg_head_inter_chs:
  - 32
  - 64
  - 64
  type: PPLiteSeg
optimizer:
  momentum: 0.9
  type: sgd
  weight_decay: 0.0005
test_config:
  aug_eval: true
  scales: 0.5
train_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: train
  num_classes: 2
  train_path: /home/aistudio/data/dataset/train.txt
  transforms:
  - max_scale_factor: 1.5
    min_scale_factor: 0.125
    scale_step_size: 0.125
    type: ResizeStepScaling
  - crop_size:
    - 1024
    - 512
    type: RandomPaddingCrop
  - type: RandomHorizontalFlip
  - brightness_range: 0.5
    contrast_range: 0.5
    saturation_range: 0.5
    type: RandomDistort
  - type: Normalize
  type: Dataset
val_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: val
  num_classes: 2
  transforms:
  - type: Normalize
  type: Dataset
  val_path: /home/aistudio/data/dataset/val.txt
------------------------------------------------
W0306 11:22:01.376241  7367 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0306 11:22:01.376297  7367 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
2023-03-06 11:22:02 [INFO]	Loading pretrained model from https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
2023-03-06 11:22:03 [INFO]	There are 145/145 variables loaded into STDCNet.
2023-03-06 11:22:03 [INFO]	Loading pretrained model from output/mask_iron/best_model/model.pdparams
2023-03-06 11:22:03 [INFO]	There are 247/247 variables loaded into PPLiteSeg.
2023-03-06 11:22:03 [INFO]	Loaded trained params of model successfully
2023-03-06 11:22:03 [INFO]	Start evaluating (total_samples: 32, total_iters: 32)...
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
32/32 [==============================] - 6s 196ms/step - batch_cost: 0.1953 - reader cost: 0.1395
2023-03-06 11:22:09 [INFO]	[EVAL] #Images: 32 mIoU: 0.9858 Acc: 0.9947 Kappa: 0.9857 Dice: 0.9928
2023-03-06 11:22:09 [INFO]	[EVAL] Class IoU: 
[0.993  0.9787]
2023-03-06 11:22:09 [INFO]	[EVAL] Class Precision: 
[0.9969 0.9878]
2023-03-06 11:22:09 [INFO]	[EVAL] Class Recall: 
[0.996  0.9906]

由评估输出可见，模型性能 mIoU: 0.9858 Acc: 0.9947。能够满足实际工业场景需求。

# 也可进行多尺度翻转评估
!python val.py \
       --config /home/aistudio/work/pp_liteseg_stdc1.yml \
       --model_path output/mask_iron/best_model/model.pdparams \
       --aug_eval \
       --scales 0.75 1.0 1.25 \
       --flip_horizontal

2023-03-06 11:22:43 [INFO]	
---------------Config Information---------------
batch_size: 4
iters: 2000
loss:
  coef:
  - 1
  - 1
  - 1
  types:
  - min_kept: 130000
    type: OhemCrossEntropyLoss
  - min_kept: 130000
    type: OhemCrossEntropyLoss
  - min_kept: 130000
    type: OhemCrossEntropyLoss
lr_scheduler:
  end_lr: 0
  learning_rate: 0.005
  power: 0.9
  type: PolynomialDecay
  warmup_iters: 100
  warmup_start_lr: 1.0e-05
model:
  arm_out_chs:
  - 32
  - 64
  - 128
  backbone:
    pretrained: https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
    type: STDC1
  seg_head_inter_chs:
  - 32
  - 64
  - 64
  type: PPLiteSeg
optimizer:
  momentum: 0.9
  type: sgd
  weight_decay: 0.0005
test_config:
  aug_eval: true
  scales: 0.5
train_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: train
  num_classes: 2
  train_path: /home/aistudio/data/dataset/train.txt
  transforms:
  - max_scale_factor: 1.5
    min_scale_factor: 0.125
    scale_step_size: 0.125
    type: ResizeStepScaling
  - crop_size:
    - 1024
    - 512
    type: RandomPaddingCrop
  - type: RandomHorizontalFlip
  - brightness_range: 0.5
    contrast_range: 0.5
    saturation_range: 0.5
    type: RandomDistort
  - type: Normalize
  type: Dataset
val_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: val
  num_classes: 2
  transforms:
  - type: Normalize
  type: Dataset
  val_path: /home/aistudio/data/dataset/val.txt
------------------------------------------------
W0306 11:22:43.975018  7513 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0306 11:22:43.975075  7513 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
2023-03-06 11:22:45 [INFO]	Loading pretrained model from https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
2023-03-06 11:22:45 [INFO]	There are 145/145 variables loaded into STDCNet.
2023-03-06 11:22:45 [INFO]	Loading pretrained model from output/mask_iron/best_model/model.pdparams
2023-03-06 11:22:45 [INFO]	There are 247/247 variables loaded into PPLiteSeg.
2023-03-06 11:22:45 [INFO]	Loaded trained params of model successfully
2023-03-06 11:22:45 [INFO]	Start evaluating (total_samples: 32, total_iters: 32)...
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
32/32 [==============================] - 6s 199ms/step - batch_cost: 0.1990 - reader cost: 0.0707
2023-03-06 11:22:52 [INFO]	[EVAL] #Images: 32 mIoU: 0.9877 Acc: 0.9954 Kappa: 0.9876 Dice: 0.9938
2023-03-06 11:22:52 [INFO]	[EVAL] Class IoU: 
[0.9939 0.9815]
2023-03-06 11:22:52 [INFO]	[EVAL] Class Precision: 
[0.997  0.9904]
2023-03-06 11:22:52 [INFO]	[EVAL] Class Recall: 
[0.9969 0.9909]

# 还可进行滑窗评估
!python val.py \
       --config /home/aistudio/work/pp_liteseg_stdc1.yml \
       --model_path output/mask_iron/best_model/model.pdparams \
       --is_slide \
       --crop_size 256 256 \
       --stride 128 128

2023-03-06 11:23:01 [INFO]	
---------------Config Information---------------
batch_size: 4
iters: 2000
loss:
  coef:
  - 1
  - 1
  - 1
  types:
  - min_kept: 130000
    type: OhemCrossEntropyLoss
  - min_kept: 130000
    type: OhemCrossEntropyLoss
  - min_kept: 130000
    type: OhemCrossEntropyLoss
lr_scheduler:
  end_lr: 0
  learning_rate: 0.005
  power: 0.9
  type: PolynomialDecay
  warmup_iters: 100
  warmup_start_lr: 1.0e-05
model:
  arm_out_chs:
  - 32
  - 64
  - 128
  backbone:
    pretrained: https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
    type: STDC1
  seg_head_inter_chs:
  - 32
  - 64
  - 64
  type: PPLiteSeg
optimizer:
  momentum: 0.9
  type: sgd
  weight_decay: 0.0005
test_config:
  aug_eval: true
  scales: 0.5
train_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: train
  num_classes: 2
  train_path: /home/aistudio/data/dataset/train.txt
  transforms:
  - max_scale_factor: 1.5
    min_scale_factor: 0.125
    scale_step_size: 0.125
    type: ResizeStepScaling
  - crop_size:
    - 1024
    - 512
    type: RandomPaddingCrop
  - type: RandomHorizontalFlip
  - brightness_range: 0.5
    contrast_range: 0.5
    saturation_range: 0.5
    type: RandomDistort
  - type: Normalize
  type: Dataset
val_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: val
  num_classes: 2
  transforms:
  - type: Normalize
  type: Dataset
  val_path: /home/aistudio/data/dataset/val.txt
------------------------------------------------
W0306 11:23:01.181236  7584 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0306 11:23:01.181291  7584 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
2023-03-06 11:23:02 [INFO]	Loading pretrained model from https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
2023-03-06 11:23:02 [INFO]	There are 145/145 variables loaded into STDCNet.
2023-03-06 11:23:02 [INFO]	Loading pretrained model from output/mask_iron/best_model/model.pdparams
2023-03-06 11:23:02 [INFO]	There are 247/247 variables loaded into PPLiteSeg.
2023-03-06 11:23:02 [INFO]	Loaded trained params of model successfully
2023-03-06 11:23:02 [INFO]	Start evaluating (total_samples: 32, total_iters: 32)...
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
32/32 [==============================] - 17s 533ms/step - batch_cost: 0.5324 - reader cost: 0.0147
2023-03-06 11:23:19 [INFO]	[EVAL] #Images: 32 mIoU: 0.9862 Acc: 0.9948 Kappa: 0.9861 Dice: 0.9930
2023-03-06 11:23:19 [INFO]	[EVAL] Class IoU: 
[0.9932 0.9792]
2023-03-06 11:23:19 [INFO]	[EVAL] Class Precision: 
[0.9975 0.9868]
2023-03-06 11:23:19 [INFO]	[EVAL] Class Recall: 
[0.9957 0.9922]

6. 模型预测

使用predict.py可用来查看具体样本的切割样本效果。

!python predict.py \
       --config /home/aistudio/work/pp_liteseg_stdc1.yml \
       --model_path output/mask_iron/best_model/model.pdparams \
       --image_path /home/aistudio/data/dataset/ec539f77-7061-4106-9914-8d66f450234d.jpg \
       --save_dir output/result
# !python predict.py \
#        --config /home/aistudio/work/pp_liteseg.yml \
#        --model_path output/mask_iron/best_model/model.pdparams \
#        --image_path /home/aistudio/data/dataset/20220705-153804.jpg \
#        --save_dir output/result

2023-03-06 11:25:10 [INFO]	
---------------Config Information---------------
batch_size: 4
iters: 2000
loss:
  coef:
  - 1
  - 1
  - 1
  types:
  - min_kept: 130000
    type: OhemCrossEntropyLoss
  - min_kept: 130000
    type: OhemCrossEntropyLoss
  - min_kept: 130000
    type: OhemCrossEntropyLoss
lr_scheduler:
  end_lr: 0
  learning_rate: 0.005
  power: 0.9
  type: PolynomialDecay
  warmup_iters: 100
  warmup_start_lr: 1.0e-05
model:
  arm_out_chs:
  - 32
  - 64
  - 128
  backbone:
    pretrained: https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
    type: STDC1
  seg_head_inter_chs:
  - 32
  - 64
  - 64
  type: PPLiteSeg
optimizer:
  momentum: 0.9
  type: sgd
  weight_decay: 0.0005
test_config:
  aug_eval: true
  scales: 0.5
train_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: train
  num_classes: 2
  train_path: /home/aistudio/data/dataset/train.txt
  transforms:
  - max_scale_factor: 1.5
    min_scale_factor: 0.125
    scale_step_size: 0.125
    type: ResizeStepScaling
  - crop_size:
    - 1024
    - 512
    type: RandomPaddingCrop
  - type: RandomHorizontalFlip
  - brightness_range: 0.5
    contrast_range: 0.5
    saturation_range: 0.5
    type: RandomDistort
  - type: Normalize
  type: Dataset
val_dataset:
  dataset_root: /home/aistudio/data/dataset
  mode: val
  num_classes: 2
  transforms:
  - type: Normalize
  type: Dataset
  val_path: /home/aistudio/data/dataset/val.txt
------------------------------------------------
W0306 11:25:10.929594  7985 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0306 11:25:10.929652  7985 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
2023-03-06 11:25:12 [INFO]	Loading pretrained model from https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
2023-03-06 11:25:12 [INFO]	There are 145/145 variables loaded into STDCNet.
2023-03-06 11:25:12 [INFO]	Number of predict images = 1
2023-03-06 11:25:12 [INFO]	Loading pretrained model from output/mask_iron/best_model/model.pdparams
2023-03-06 11:25:12 [INFO]	There are 247/247 variables loaded into PPLiteSeg.
2023-03-06 11:25:12 [INFO]	Start to predict...
1/1 [==============================] - 1s 1s/step

预测的结果如下所示

import matplotlib.pyplot as plt
import cv2
im = cv2.imread("/home/aistudio/work/PaddleSeg/output/result/pseudo_color_prediction/ec539f77-7061-4106-9914-8d66f450234d.png")
# cv2.imshow("result", im)
plt.imshow(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))
plt.figure()
im = cv2.imread("/home/aistudio/work/PaddleSeg/output/result/added_prediction/ec539f77-7061-4106-9914-8d66f450234d.jpg")
plt.imshow(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))

<matplotlib.image.AxesImage at 0x7f4aad9cf050>

请添加图片描述

请添加图片描述

接下来，利用预测的结果，并采用最大联通域处理后，判断钢筋是否超限，(此脚本的路径为 work/postprocessing.py)

import cv2

def largestcomponent(img_path, threshold=None):
    """
    Filter the input image_path with threshold, only component that have area larger than threshold will be kept.

    Arg:
    img_path: path to a binary img
    threshold: connected componet with area larger than this value will be kept

    """
    binary = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)
    binary[binary == binary.min()] = 0
    binary[binary == binary.max()] = 255
    assert (
        binary.max() == 255 and binary.min() == 0
    ), "The input need to be a binary image, but the maxval in image is {} and the minval in image is {}".format(
        binary.max(), binary.min()
    )

    if threshold is None:
        threshold = binary.shape[0] * binary.shape[1] * 0.01
    contours, hierarchy = cv2.findContours(
        binary, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE
    )

    for k in range(len(contours)):
        if cv2.contourArea(contours[k]) < threshold:
            cv2.fillPoly(binary, [contours[k]], 0)

    cv2.imwrite(img_path.split(".")[0] + "_postprocessed.png", binary)

# 此处，我们可以对比最大联通域处理前后的差别，可以发现滤除了小的联通区域
prediction = "/home/aistudio/work/PaddleSeg/output/result/pseudo_color_prediction/ec539f77-7061-4106-9914-8d66f450234d.png"
# prediction = "/home/aistudio/work/PaddleSeg/output/result/pseudo_color_prediction/20220705-153804.png"
largestcomponent(prediction)
before_image = cv2.imread(prediction)
after_image = cv2.imread(prediction.replace(".png", "_postprocessed.png"))
plt.subplot(1, 2, 1)
plt.imshow(cv2.cvtColor(before_image, cv2.COLOR_BGR2RGB))
plt.subplot(1, 2, 2)
plt.imshow(cv2.cvtColor(after_image, cv2.COLOR_BGR2RGB))
plt.show()

请添加图片描述

# 判断钢筋是否超限
def excesslimit(image_path, direction="right", position=0.6):
    """
    Automatically tells if the steel bar excess manually set position.

    Arg:
    img_path: path to a binary img
    direction: which part of the img is the focused area for detecting bar excession.
    position: the ratio of the position of the line to the width of the image.

    Return:
    excess: whether there is steel wheel excess the limit line.
    excess_potion: the portion of the excess steel bar to the whole bar.
    """

    excess_portion = 0.0

    binary = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
    binary[binary == binary.min()] = 0
    binary[binary == binary.max()] = 255
    assert (
        binary.max() == 255 and binary.min() == 0
    ), "The input need to be a binary image, but the maxval in image is {} and the minval in image is {}".format(
        binary.max(), binary.min()
    )
    assert (
        direction == "left" or direction == "right"
    ), "The direction indicates the side of image that iron excess, it should be 'right' or 'left', but we got {}".format(
        direction
    )
    assert (
        position > 0 and position < 1
    ), "The position indicates the relative position to set the line, it should bigger than 0 and smaller than 1, but we got {}".format(
        position
    )

    img_pos = int(binary.shape[1] * position)

    if direction == "right":
        if binary[:, img_pos:].sum() > 0:
            excess_portion = binary[:, img_pos:].sum() / binary.sum()
        binary[:, img_pos : img_pos + 3] = 255

    else:
        if binary[:, :img_pos].sum() > 0:
            excess_portion = binary[:, :img_pos].sum() / binary.sum()
        binary[:, img_pos - 3 : img_pos] = 255

    print(
        "The iron is {}excessed in {}, and the excess portion is {}".format(
            ["", "not "][excess_portion == 0], image_path, excess_portion
        )
    )

    # cv2.imwrite(image_path.split(".")[0] + "_fullpostprocessed.png", binary)
    cv2.imwrite(image_path.replace("_postprocessed.png", "_fullpostprocessed.png"), binary)

    return excess_portion > 0, excess_portion


# 对预测结果批量判断是否超限
import os
import glob

output_dir = "/home/aistudio/work/PaddleSeg/output"
pseudo_color_result = os.path.join(output_dir, 'result/pseudo_color_prediction')
os.system(f"rm {pseudo_color_result}/*_*postprocessed.*")
for img_path in glob.glob(os.path.join(pseudo_color_result, "*.png")):
    largestcomponent(img_path)
    postproc_img_path = img_path.replace(".png", "_postprocessed.png")
    excesslimit(postproc_img_path, "left", 0.3)


The iron is excessed in /home/aistudio/work/PaddleSeg/output/result/pseudo_color_prediction/ec539f77-7061-4106-9914-8d66f450234d_postprocessed.png, and the excess portion is 0.040420855682695565


/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:51: DeprecationWarning: In future, it will be an error for 'np.bool_' scalars to be interpreted as an index

#可视化后处理结果
im = cv2.imread("/home/aistudio/work/PaddleSeg/output/result/pseudo_color_prediction/ec539f77-7061-4106-9914-8d66f450234d_fullpostprocessed.png")
# cv2.imshow("result", im)
plt.imshow(im)

<matplotlib.image.AxesImage at 0x7f4aad9064d0>

请添加图片描述

7. 模型推理与部署

7.1 使用 Paddle Inference 推理

导出训练好的模型为paddle inference模型

!export CUDA_VISIBLE_DEVICES=0 # Set a usable GPU.
# If on windows, Run the following command
# set CUDA_VISIBLE_DEVICES=0
!python export.py \
       --config /home/aistudio/work/pp_liteseg_stdc1.yml \
       --model_path output/mask_iron/best_model/model.pdparams \
       --save_dir output/inference

W0306 12:13:14.298084 14841 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0306 12:13:14.302328 14841 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
2023-03-06 12:13:15 [INFO]	Loading pretrained model from https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet1.tar.gz
2023-03-06 12:13:16 [INFO]	There are 145/145 variables loaded into STDCNet.
2023-03-06 12:13:16 [INFO]	Loaded trained params of model successfully.
2023-03-06 12:13:19 [INFO]	Model is saved in output/inference.

使用推理模型预测

!python deploy/python/infer.py \
    --config output/inference/deploy.yaml \
    --save_dir output/infer_result \
    --image_path /home/aistudio/data/dataset/bcd33bcd-d48c-4409-940d-51301c8a7697.jpg

2023-03-06 12:14:42 [INFO]	Use GPU
W0306 12:14:44.105751 15073 analysis_predictor.cc:1391] The one-time configuration of analysis predictor failed, which may be due to native predictor called first and its configurations taken effect.
[1m[35m--- Running analysis [ir_graph_build_pass][0m
[1m[35m--- Running analysis [ir_graph_clean_pass][0m
[1m[35m--- Running analysis [ir_analysis_pass][0m
[32m--- Running IR pass [is_test_pass][0m
[32m--- Running IR pass [simplify_with_basic_ops_pass][0m
[32m--- Running IR pass [conv_bn_fuse_pass][0m
I0306 12:14:44.260706 15073 fuse_pass_base.cc:59] ---  detected 39 subgraphs
[32m--- Running IR pass [conv_eltwiseadd_bn_fuse_pass][0m
I0306 12:14:44.283650 15073 fuse_pass_base.cc:59] ---  detected 4 subgraphs
[32m--- Running IR pass [embedding_eltwise_layernorm_fuse_pass][0m
[32m--- Running IR pass [multihead_matmul_fuse_pass_v2][0m
[32m--- Running IR pass [fused_multi_transformer_encoder_pass][0m
[32m--- Running IR pass [fused_multi_transformer_decoder_pass][0m
[32m--- Running IR pass [fused_multi_transformer_encoder_fuse_qkv_pass][0m
[32m--- Running IR pass [fused_multi_transformer_decoder_fuse_qkv_pass][0m
[32m--- Running IR pass [multi_devices_fused_multi_transformer_encoder_fuse_qkv_pass][0m
[32m--- Running IR pass [multi_devices_fused_multi_transformer_decoder_fuse_qkv_pass][0m
[32m--- Running IR pass [fuse_multi_transformer_layer_pass][0m
[32m--- Running IR pass [gpu_cpu_squeeze2_matmul_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_reshape2_matmul_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_flatten2_matmul_fuse_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_v2_to_mul_pass][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_v2_to_matmul_pass][0m
[32m--- Running IR pass [matmul_scale_fuse_pass][0m
[32m--- Running IR pass [multihead_matmul_fuse_pass_v3][0m
[32m--- Running IR pass [gpu_cpu_map_matmul_to_mul_pass][0m
[32m--- Running IR pass [fc_fuse_pass][0m
[32m--- Running IR pass [fc_elementwise_layernorm_fuse_pass][0m
[32m--- Running IR pass [conv_elementwise_add_act_fuse_pass][0m
[32m--- Running IR pass [conv_elementwise_add2_act_fuse_pass][0m
[32m--- Running IR pass [conv_elementwise_add_fuse_pass][0m
[32m--- Running IR pass [transpose_flatten_concat_fuse_pass][0m
[32m--- Running IR pass [constant_folding_pass][0m
[32m--- Running IR pass [runtime_context_cache_pass][0m
[1m[35m--- Running analysis [ir_params_sync_among_devices_pass][0m
I0306 12:14:44.666788 15073 ir_params_sync_among_devices_pass.cc:89] Sync params from CPU to GPU
[1m[35m--- Running analysis [adjust_cudnn_workspace_size_pass][0m
[1m[35m--- Running analysis [inference_op_replace_pass][0m
[1m[35m--- Running analysis [memory_optimize_pass][0m
I0306 12:14:44.693254 15073 memory_optimize_pass.cc:219] Cluster name : shape_1.tmp_0_slice_0  size: 8
I0306 12:14:44.693290 15073 memory_optimize_pass.cc:219] Cluster name : shape_0.tmp_0_slice_0  size: 8
I0306 12:14:44.693295 15073 memory_optimize_pass.cc:219] Cluster name : x  size: 12
I0306 12:14:44.693297 15073 memory_optimize_pass.cc:219] Cluster name : max_1.tmp_0  size: 4
I0306 12:14:44.693307 15073 memory_optimize_pass.cc:219] Cluster name : batch_norm_21.tmp_2  size: 2048
I0306 12:14:44.693315 15073 memory_optimize_pass.cc:219] Cluster name : batch_norm_21.tmp_0  size: 2048
I0306 12:14:44.693318 15073 memory_optimize_pass.cc:219] Cluster name : relu_28.tmp_0  size: 8192
I0306 12:14:44.693325 15073 memory_optimize_pass.cc:219] Cluster name : pool2d_5.tmp_0  size: 65536
I0306 12:14:44.693327 15073 memory_optimize_pass.cc:219] Cluster name : concat_5.tmp_0  size: 4096
I0306 12:14:44.693341 15073 memory_optimize_pass.cc:219] Cluster name : concat_1.tmp_0  size: 1024
I0306 12:14:44.693343 15073 memory_optimize_pass.cc:219] Cluster name : concat_3.tmp_0  size: 2048
[1m[35m--- Running analysis [ir_graph_to_program_pass][0m
I0306 12:14:44.716624 15073 analysis_predictor.cc:1314] ======= optimize end =======
I0306 12:14:44.718086 15073 naive_executor.cc:110] ---  skip [feed], feed -> x
I0306 12:14:44.719072 15073 naive_executor.cc:110] ---  skip [argmax_0.tmp_0], fetch -> fetch
W0306 12:14:44.962486 15073 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0306 12:14:44.966229 15073 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
2023-03-06 12:14:46 [INFO]	Finish

img_path = "/home/aistudio/work/PaddleSeg/output/infer_result/bcd33bcd-d48c-4409-940d-51301c8a7697.png"
im = cv2.imread(img_path, cv2.IMREAD_UNCHANGED)
plt.imshow(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))

<matplotlib.image.AxesImage at 0x7f4ab61111d0>

请添加图片描述

根据模型输出，判断钢筋是否超限

img_path = "/home/aistudio/work/PaddleSeg/output/infer_result/bcd33bcd-d48c-4409-940d-51301c8a7697.png"
largestcomponent(img_path)
postproc_img_path = img_path.replace(".png", "_postprocessed.png")
excesslimit(postproc_img_path, "right", 0.5)

The iron is excessed in /home/aistudio/work/PaddleSeg/output/infer_result/bcd33bcd-d48c-4409-940d-51301c8a7697_postprocessed.png, and the excess portion is 0.1875316840529423


/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:51: DeprecationWarning: In future, it will be an error for 'np.bool_' scalars to be interpreted as an index





(True, 0.1875316840529423)

# 可视化判断结果
img_path = "/home/aistudio/work/PaddleSeg/output/infer_result/bcd33bcd-d48c-4409-940d-51301c8a7697_fullpostprocessed.png"
im = cv2.imread(img_path)
plt.imshow(im)

<matplotlib.image.AxesImage at 0x7f4aff724490>

请添加图片描述

7.2 使用 FastDeploy 部署

FastDeploy是一款全场景、易用灵活、极致高效的AI推理部署工具。提供📦开箱即用的云边端部署体验, 支持超过 🔥150+ Text, Vision, Speech和跨模态模型，并实现🔚端到端的推理性能优化。包括图像分类、物体检测、图像分割、人脸检测、人脸识别、关键点检测、抠图、OCR、NLP、TTS等任务，满足开发者多场景、多硬件、多平台的产业部署需求。

# 安装 fastdeploy
!pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

import fastdeploy as fd
model_file = "/home/aistudio/work/PaddleSeg/output/inference/model.pdmodel"
params_file = "/home/aistudio/work/PaddleSeg/output/inference/model.pdiparams"
infer_cfg_file = "/home/aistudio/work/PaddleSeg/output/inference/deploy.yaml"

# 模型推理的配置信息
option = fd.RuntimeOption()
model = fd.vision.segmentation.PaddleSegModel(model_file, params_file, infer_cfg_file, option)

[INFO] fastdeploy/vision/common/processors/transform.cc(93)::FuseNormalizeHWC2CHW	Normalize and HWC2CHW are fused to NormalizeAndPermute  in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert	BGR2RGB and NormalizeAndPermute are fused to NormalizeAndPermute with swap_rb=1
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(226)::InitFromPaddle	Compile OpenVINO model on device_name:CPU.
[INFO] fastdeploy/runtime/runtime.cc(322)::CreateOpenVINOBackend	Runtime initialized with Backend::OPENVINO in Device::CPU.

# 预测结果
import cv2
img_path = "/home/aistudio/data/dataset/8f7fcf0a-a3ea-41f2-9e67-4cbaa61238a4.jpg"
im = cv2.imread(img_path)
result = model.predict(im)
print(result)

SegmentationResult Image masks 10 rows x 10 cols: 
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .....]
...........
result shape is: [1080 1920]

# 使用fastdeploy提供的可视化函数进行可视化
import matplotlib.pyplot as plt
vis_im = fd.vision.visualize.vis_segmentation(im, result, 0.5)
plt.imshow(cv2.cvtColor(vis_im, cv2.COLOR_BGR2RGB))

<matplotlib.image.AxesImage at 0x7f49223e7150>

请添加图片描述

# 判断钢筋是否超限
# 为了便于演示，兼容上面的判断接口。这处将结果导出为mask图片。
import numpy as np
mask = np.reshape(result.label_map, result.shape)
mask = np.uint8(mask)
mask_path = "/home/aistudio/work/PaddleSeg/output/infer_result/mask.png"
cv2.imwrite(mask_path, mask)
# print(mask_path)
largestcomponent(mask_path)
post_img_path = mask_path.replace(".png", "_postprocessed.png")
# print(post_img_path)
excesslimit(post_img_path, "right", 0.7)

The iron is not excessed in /home/aistudio/work/PaddleSeg/output/infer_result/mask_postprocessed.png, and the excess portion is 0.0

v2.cvtColor(vis_im, cv2.COLOR_BGR2RGB))





    <matplotlib.image.AxesImage at 0x7f49223e7150>




    
[外链图片转存中...(img-SadVrJ5V-1678806404026)]
    



```python
# 判断钢筋是否超限
# 为了便于演示，兼容上面的判断接口。这处将结果导出为mask图片。
import numpy as np
mask = np.reshape(result.label_map, result.shape)
mask = np.uint8(mask)
mask_path = "/home/aistudio/work/PaddleSeg/output/infer_result/mask.png"
cv2.imwrite(mask_path, mask)
# print(mask_path)
largestcomponent(mask_path)
post_img_path = mask_path.replace(".png", "_postprocessed.png")
# print(post_img_path)
excesslimit(post_img_path, "right", 0.7)

The iron is not excessed in /home/aistudio/work/PaddleSeg/output/infer_result/mask_postprocessed.png, and the excess portion is 0.0





(False, 0.0)

# 可视化判断结果
im_path = "/home/aistudio/work/PaddleSeg/output/infer_result/mask_fullpostprocessed.png"
im = cv2.imread(im_path)
plt.imshow(im)

<matplotlib.image.AxesImage at 0x7f49213344d0>

请添加图片描述

8. 总结

本项目演示了在工业实际场景下，使用PaddleSeg开发套件，进行语义分割模型训练，并使用FastDeploy进行部署应用，解决钢筋长度超限的自动监控问题。结果证明，本技术方案切实可行，可实现端到端全自动的钢筋长度超限监控，为企业生产降本增效。希望本应用范例可以给各行业从业人员和开发者带来有益的启发。

加入交流群，一起学习吧

更多关于分割的知识请移步PaddleSeg，如有帮助，欢迎Star⭐支持~

github地址：https://github.com/PaddlePaddle/PaddleSeg

欢迎加入PaddleSeg微信群，交流更多前沿技术。

AI Studio
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
Corrupt JPEG data: 2 extraneous bytes before marker 0x
jacke121的专栏
03-19 3万+
Corrupt JPEG data: 2 extraneous bytes before marker 0x
python运行问题记录
moonwhy的博客
06-27 2107
警告：Corrupt JPEG data: 2 extraneous bytes before marker 0xd9 原因：Corrupt Jpeg Data 2 Extraneous Bytes Before Marker 0xd9 errors can occur after frequent use of your PC. Opening programs will be slower a...
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
最新发布
人工智能视觉分析算法学习实践和经验分享。
06-18 1612
Corrupt JPEG data: 2 extraneous bytes before marker 0xd9警告产生的主要原因和解决办法，ps工具修复和python修复方法。
YOLOv5训练时出现Corrupt JPEG data: 2 extraneous bytes before marker 0xd9
weixin_46751388的博客
09-06 6423
在colab使用yolov5训练数据集时，出现Corrupt JPEG data: 2 extraneous bytes before marker 0xd9，但不影响训练结果。
报错解决 | Picodet训练报错
uuyymilkyl的博客
12-30 2553
原因：图像是单通道和三通道混在一起，把图像全部转成三通道再输入。
钢筋长度超限检测检数据集VOC+YOLO格式215张1类别.7z
05-12
数据集格式：Pascal VOC格式+YOLO格式(不包含分割路径的txt文件，仅仅包含jpg图片以及对应的VOC格式xml文件和yolo格式txt文件) 图片数量(jpg文件个数)：215 标注数量(xml文件个数)：215 标注数量(txt文件个数)：215 ...
钢筋长度更新_g钢筋编号_
09-29
需配合贱人工具箱5.9使用，还需建立一个属性块，包含钢筋长度 钢筋编号，钢筋半径
行业文档-设计装置-基于钢筋加工设备的嵌入式远程监控装置.zip
08-27
"行业文档-设计装置-基于钢筋加工设备的嵌入式远程监控装置.zip"这个压缩包文件为我们揭示了这个领域的前沿技术，主要涉及了嵌入式系统、远程监控以及钢筋加工设备的集成应用。以下是对这些关键知识点的详细阐述。 ...
钢筋长度计算
03-18
计算钢筋长度，这样可以方便的计算钢筋长度
基于YOLOV5的钢筋网面交叉点识别（这是压缩包2）
02-27
基于YOLOV5的钢筋网面交叉点识别 基于YOLOV5的钢筋网面交叉点识别
Corrupt JPEG data: 111 extraneous bytes before marker 0xd9...
热门推荐
lsh呵呵
10-15 1万+
问题描述 Corrupt JPEG data: 1 extraneous bytes before marker 0xdb libpng warning: iCCP: known incorrect sRGB profile Premature end of JPEG file Premature end of JPEG file Premature end of JPEG file Prema...
Huffman编码
奋起的Coder。。。
01-31 1630
Huffman编码 main.h #ifndef main_h #define main_h typedef struct HTNode { int weight; int lchild,rchild,parent; }HTNode; typedef struct HCNode { char ch; char bits[27]; int len; }HCNode; #endif
判定深度学习的图片和标注文件是否存在问题
AI小花猫的博客
07-25 1100
筛除深度学习图片和xml是否存在错误
opencv打开webcam出现Corrupt JPEG data: 2 extraneous解决办法
上善若水
01-17 5148
该问题是由opencv第三方库libjpeg引起的错误(具体原因本人猜测可能是由于jpeg编解码时,在格式上可能有问题),通过grep -R "Corrupt JPEG data,可以定位到thirdparty/libjpeg/jdmarker.cif (cinfo->marker->discarded_bytes != 0) { WARNMS2(cinfo, JWRN_EXTRANEOU
Corrupt JPEG data: 36 extraneous bytes before marker 0xd9，opencv获取imread报错方法
lcj1105的专栏
01-23 1万+
错误类型：Corrupt JPEG data: 36 extraneous bytes before marker 0xd9 原因：opencv imread默认有错误直接跳过，不会返回，需要修改源码才能使之报错 ① 修改 modules/highgui/src/grfmt_jpeg.cpp 文件，在error_exit()函数下面添加以下代码： METHODDEF(void)
日常笔记（一）
weixin_40516558的博客
07-30 270
1. How can I get a count of files in a directory using the command line? ls | wc -l 2. vim多行注释/取消多行注释 a)多行注释： 进入命令行模式，按ctrl + v进入 visual block模式（可视快模式），然后按j, 或者k选中多行，把需要注释的行标记起来 按大写字母i，再插入注释符，例如// 按e...
钢筋拉伸试验中钢筋长度取多少
03-25
在钢筋拉伸试验中，通常选取的钢筋长度为150mm，这个长度通常被称为标准试件长度。这是因为在这个长度范围内，钢筋的性质和力学特性都能得到较为准确的测试结果。此外，在进行钢筋拉伸试验时，需要在钢筋两端加上...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值