前言
对YOLOv8-pose手部关键点检测模型进行训练,并分析训练结果,从而调优训练超参数。
手部关键点检测数据集:基于YOLOv8-pose的手部关键点检测(1)- 手部关键点数据集获取(数据集下载、数据清洗、处理与增强)
1.训练参数设置
1.1 data.yaml
同手部检测中,只需增加 kpt_shape: [21, 2]。
1.2 setting.yaml
同手部检测中,也需要关闭马赛克增强;同时马赛克增强概率要置为1。
# Train settings -------------------------------------------------------------------------------------------------------
task: pose # (str) YOLO task, i.e. detect, segment, classify, pose
mode: train # (str) YOLO mode, i.e. train, val, predict, export, track, benchmark
data: ./hand-pose.yaml # (str, optional) path to data file, i.e. coco8.yaml
epochs: 500 # (int) number of epochs to train for
batch: 256 # (int) number of images per batch (-1 for AutoBatch)
imgsz: 480 # (int | list) input images size as int for train and val modes, or list[w,h] for predict and export modes
patience: 300 # (int) epochs to wait for no observable improvement for early stopping of training
device: 7 # (int | str | list, optional) device to run on, i.e. cuda device=0 or device=0,1,2,3 or device=cpu
project: ./ # (str, optional) project name
multi_scale: True # (bool) Whether to use multiscale during training
close_mosaic: 0 # (int) disable mosaic augmentation for final epochs (0 to disable)
resume: True # (bool) resume training from last checkpoint
# Hyperparameters ------------------------------------------------------------------------------------------------------
lr0: 0.01 # (float) initial learning rate (i.e. SGD=1E-2, Adam=1E-3)
lrf: 0.01 # (float) final learning rate (lr0 * lrf)
box: 8.0 # (float) box loss gain
cls: 0.5 # (float) cls loss gain (scale with pixels)
dfl: 1.5 # (float) dfl loss gain
pose: 14.0 # (float) pose loss gain
kobj: 0.0 # (float) keypoint obj loss gain
degrees: 0.0 # (float) image rotation (+/- deg)
translate: 0.1 # (float) image translation (+/- fraction)
scale: 0.5 # (float) image scale (+/- gain)
fliplr: 0.5 # (float) image flip left-right (probability)
如下图所示,handpose数据集的bbox约为[0.5, 0.5, 0.6, 0.6],如果关闭马赛克增强,会导致网络预测检测框为具体数值,而不是依靠外观特征去预测边界。
2.训练结果分析
2.1 准确性分析
由于只有手部的patch,且handpose中存在大量两只手只标了一只手的情况,导致前期bbox的训练还是挺不稳定的,不过pose的训练还是挺稳的:
2.2 LOSS分析
如下图红框所示,同手部检测在HaGRID上的训练类似,关闭马赛克增强后,导致LOSS“断崖式下跌”,因为网络去拟合具体数值了,在知道手部位置后,关键点的LOSS也随之下跌:
上述过程,也会导致网络在原始数据上的“过拟合”。如下图所示,如果将检测出的手直接进行预测,获得手部关键点像被“压缩”了一下,这里实际上,是网络想把关键点缩放在手部框的“3/5”范围内:
在将手部框放大2/3后再预测,即使右边(输入到手部关键点检测的patch)被截断,结果还是可以相对校准的预测对:
3.可视化结果
3.1 简单手部
也可以看到下面左图,关闭马赛克增强后,预测效果更好。
3.2 截断手部
3.3 多手+截断
下图右边的手预测了两次,所以最终在处理时候,对于每一个手部检测框,只要获取置信度最大的那个结果即可:
3.4 验证集行测试结果
Handpose_v2:
HaGRID:
HaGRID-rotate:
Hand_keypoint_26K:
4.结论
只要能看清手部的图片,基本都能识别。