一、 整体思路
1、训练素材库
| 行为名称 | 说明 |
|---|---|
| fighting | 打架/肢体冲突/校园暴力/校园欺凌 |
| handup | 举手 |
| playphone | 玩手机 |
| sleeping | 课堂睡觉/趴桌子睡觉 |
| writing | 写字 |
2、训练过程
①通过关键字爬取百度等搜索引擎的图片，通过Python脚本获取。
②素材存放到Git仓库上，便于素材的更新。
③通过Digits进行可视化训练，提高训练的效率，也便于实时查看训练的进度和训练的结果以及训练结果的验证。
3、模型更新步骤
①当图片素材有更新或者训练参数有调整等情况下，有新的Model产生后，需要将新的Model替换到目前生产环境上的Model。
②在新的Model发布前，应对Model进行测试，通过后再发布到生产环境。
4、行为识别步骤
①用户上传照片后，调用HTTP协议的行为识别接口。
②行为识别接口对图片进行多重处理：判断图片中是否有人、转换成openpose格式。
③调用caffe接口对图片进行识别。
④返回识别结果
5、框架图
二、 概述
1、整体思路
(1)收集需要训练的图片素材（python爬虫）；
(2)使用openpose转换成火柴人（使用python调用）；
(3)使用caffe训练出model（提供python接口）；
(4)提供可视化的训练界面（可以使用digits工具，也可以使用python编写）；
(5)使用python编写HTTP协议的统一测试接口，可以调用测试接口获取测试结果（行为识别前先使用人脸识别看是否有人）；
(6)行为训练平台（提供在线训练和结果查看等功能）、行为支撑平台（配置相关训练的参数、行为模型管理）、业务平台（行为接口配置）
2、训练过程
(1)收集需要训练的图片素材，可以通过爬虫工具或者写python脚本到网上抓取；
(2)使用openpose获取图片的肢体图；
(3)将图片缩放至统一大小；
(4)将图片转换成caffe能识别的leveldb或lmdb格式；
(5)开始训练，生成model；
(6)使用model测试图片。
3、图片转换成db格式
caffe不支持原始的jpg、png等图片格式，需要转换成能caffe能识别的leveldb/lmdb。
基本思路：
将图片按分类放到指定的目录下 → 使用脚本生成label.txt（标签清单）和train.txt（训练文件清单）→ 使用convert_imageset转换图片
三、 使用digits训练模型
1、准备图片
2、创建Datasets
①关键参数
| 参数名称 | 说明 |
|---|---|
| Image Type | |
| Image size (Width x Height) | |
| Resize Transformation | |
| Training Images | |
| Minimum samples per class | |
| Maximum samples per class | |
| Separate validation images folder | |
| Separate test images folder | |
| Image Encoding | |
| DB backend |
3、创建Models
①关键参数
| 参数名称 | 说明 |
|---|---|
| Training epochs | 越大精准度越大 |
| Snapshot interval (in epochs) | |
| Validation interval (in epochs) | |
| Random seed | |
| Batch size | 30-40为最佳 |
| Batch Accumulation | |
| Solver type | |
| Base Learning Rate | |
| Caffe Networks |
4、验证
①单文件验证
提供上传文件或者从系统中选择两种方式。
勾选上Show visualizations and statistics会在验证结果上显示详细的统计信息
②多文件验证
上传验证文件列表，一个文件一行，内容如下图：
四、 python调用caffe接口验证训练好的模型
1、文件结构
根目录为/data/python/motion，目录下的文件结构如下
2、编写python脚本调用caffe接口代码，将该脚本保存为serverstart.py，存放在根目录下
# coding=utf-8
import caffe
import numpy as np
root = '/data/python/' # 根目录
deploy = root + 'deploy.prototxt' # deploy文件
caffe_model = root + 'snapshot_iter_9500.caffemodel' # 训练好的 caffemodel
img_path = root + 'upload/' # 图片路径
labels_filename = root + 'labels.txt' # 类别名称文件，将数字标签转换回类别名称
mean_file_path = root + 'mean.npy'
net = caffe.Net(deploy, caffe_model, caffe.TEST) # 加载model和network
# 图片预处理设置
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape}) # 设定图片的shape格式(1,3,28,28)
transformer.set_transpose('data', (2, 0, 1)) # 改变维度的顺序，由原始图片(28,28,3)变为(3,28,28)
transformer.set_mean('data', np.load(mean_file_path).mean(1).mean(1)) # 减去均值，前面训练模型时没有减均值，这儿就不用
transformer.set_raw_scale('data', 255) # 缩放到【0，255】之间
# transformer.set_channel_swap('data', (2,1,0)) #交换通道，将图片由RGB变为BGR
im = caffe.io.load_image(root + filepath) # 加载图片
net.blobs['data'].data[...] = transformer.preprocess('data', im) # 执行上面设置的图片预处理操作，并将图片载入到blob中
# 执行测试
out = net.forward()
labels = np.loadtxt(labels_filename, str, delimiter='\t') # 读取类别名称文件
prob = net.blobs['softmax'].data[0].flatten() # 取出最后一层（Softmax）属于某个类别的概率值，并打印
print prob
order = prob.argsort()[-1] # 将概率值排序，取出最大值所在的序号
print 'the class is:', labels[order] # 将该序号转换成对应的类别名称，并打印
3、获取deploy.prototxt和caffemodel
这两个文件都存放在Job Directory目录下，直接拷贝过来即可
Job Directory获取方法：登录digits，点击对应的Job进来，截图的就是Job Directory
训练好的caffemodel文件有多个，获取数字最大的一个。
4、获取均值文件
均值文件存放在训练任务对应的DataSet的Job Directory下，命名为mean.binaryproto，将该文件拷贝过来。
使用以下脚本转换成python可以识别的格式，将该脚本保存在根目录下。执行后生成mean.npy
# coding=utf-8
import caffe
import numpy as np
# 编写一个函数，将二进制的均值转换为python的均值
def convert_mean(binMean,npyMean):
blob = caffe.proto.caffe_pb2.BlobProto()
bin_mean = open(binMean, 'rb' ).read()
blob.ParseFromString(bin_mean)
arr = np.array( caffe.io.blobproto_to_array(blob) )
npy_mean = arr[0]
np.save(npyMean, npy_mean )
# 调用函数转换均值
source_mean_file='mean.binaryproto'
mean_file_path='mean.npy'
convert_mean(source_mean_file,mean_file_path)
5、获取labels.txt
labels.txt存放在训练任务对应的DataSet的Job Directory下，拷贝到根目录下即可。
6、更新模型
（注：更新前要在digits上验证一下模型的精准度，保险起见，更新前将整个目录先备份一次）
仅更新训练模型（DataSet没更新）：将最新deploy.prototxt和caffemodel拷贝过来即可。
更新DataSet：
按照步骤4和步骤5更新labels.txt和均值文件
五、 Ubuntu命令行下安装openpose
1、Caffe、Cuda、Cudnn、opencv已经安装好，网上大部分教程都是上面几项已经安装好的情况来说明的。
2、参考链接：
3、BUILD_PATH错误
在build之前需要修改CMakeLists.txt文件的BUILD_PATH，设置为ON
4、选择第一种方式
make build && cd build
cmake ..
sudo make all
5、提供两种build方式，区别在于怎样地位opencv。选择了后一种方式（失败）
mkdir build && cd build
cmake -DOpenCV_CONFIG_FILE=/usr/local/share/OpenCV/OpenCVConfig.cmake \
-DCaffe_INCLUDE_DIRS=/root/caffe/include \
-DCaffe_LIBS=/root/caffe/.build_release/lib/libcaffe.so -DBUILD_CAFFE=OFF ..
6、通过conda安装opencv-python
source activate python27
conda install -c menpo opencv3
7、运行测试脚本
cd build/examples/tutorial_python
python 1_extract_pose.py
8、使用conda安装protobuf
系统存在多个版本的protobuf，在python27环境下的protobuf版本太高，无法编译
#无效
conda install -c https://conda.anaconda.org/anaconda protobuf==2.6.1
conda create --name proto26 protobuf=2.6.1
#
# To activate this environment, use
#
# $ conda activate proto26
#
# To deactivate an active environment, use
#
# $ conda deactivate
六、 python调用openpose接口
在/data/openpose/build/examples/tutorial_python目录下有两个样例文件
python 1_extract_pose.py
执行结果
OpenPose Library Python Wrapper
[[[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 4.78186859e+02 2.70649078e+02 8.00042748e-01]
[ 5.01679993e+02 2.66831116e+02 7.86151826e-01]
[ 5.05572449e+02 2.86316193e+02 4.13571924e-01]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 4.56053680e+02 2.73280487e+02 8.11772108e-01]
[ 4.41610779e+02 3.07168884e+02 5.77636838e-01]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 4.74285400e+02 3.38565308e+02 8.44646633e-01]
[ 4.86009979e+02 3.38562134e+02 8.05852056e-01]
[ 4.87359894e+02 3.86837311e+02 8.06244075e-01]
[ 4.96492218e+02 4.42966309e+02 8.55966508e-01]
[ 4.62479034e+02 3.38575165e+02 7.30708599e-01]
[ 4.63808350e+02 3.92034058e+02 8.52564096e-01]
[ 4.65176208e+02 4.48156738e+02 8.03685606e-01]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 4.87320374e+02 2.44568726e+02 2.38700688e-01]
[ 4.65147736e+02 2.44563675e+02 8.33556652e-01]
[ 4.46878204e+02 4.48183014e+02 6.03618026e-01]
[ 4.49464813e+02 4.52092133e+02 6.88863039e-01]
[ 4.72932648e+02 4.54707764e+02 8.35708082e-01]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 4.96496613e+02 4.52051514e+02 8.17793250e-01]]
[[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 4.20748138e+02 3.05936157e+02 8.62814486e-01]
[ 4.50771271e+02 3.08577179e+02 8.04276943e-01]
[ 4.72986481e+02 3.50232239e+02 9.03372645e-01]
[ 4.48197876e+02 3.69907104e+02 6.95415556e-01]
[ 3.89441620e+02 2.99392242e+02 8.49419296e-01]
[ 3.59476807e+02 3.29447540e+02 9.41369236e-01]
[ 3.81657806e+02 3.54195953e+02 8.33846211e-01]
[ 4.12920074e+02 3.82900238e+02 7.83051670e-01]
[ 4.31229919e+02 3.84211975e+02 7.17627883e-01]
[ 4.20770142e+02 4.45609924e+02 7.96183109e-01]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 3.95964996e+02 3.81593658e+02 7.03738511e-01]
[ 3.85513763e+02 4.35118866e+02 7.69885182e-01]
[ 3.73787231e+02 4.76935089e+02 2.26258084e-01]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 4.35154358e+02 2.83695099e+02 6.73812151e-01]
[ 4.11600952e+02 2.78518860e+02 9.06918287e-01]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]]
[[ 3.28098480e+02 2.19743713e+02 7.59945154e-01]
[ 3.22891296e+02 2.17185104e+02 8.33855808e-01]
[ 2.98027008e+02 2.23682495e+02 8.89204025e-01]
[ 2.77155914e+02 2.51099258e+02 8.65952015e-01]
[ 2.98095337e+02 2.73281158e+02 8.20839524e-01]
[ 3.49003632e+02 2.13271332e+02 9.29123282e-01]
[ 3.67261780e+02 2.35505341e+02 8.70975435e-01]
[ 3.68507965e+02 2.61543365e+02 8.30662727e-01]
[ 3.28062256e+02 2.87591522e+02 8.67168963e-01]
[ 3.12459900e+02 2.87643433e+02 7.83106804e-01]
[ 3.29364838e+02 3.38515503e+02 8.60394001e-01]
[ 3.38548401e+02 3.92021790e+02 7.72056162e-01]
[ 3.39850311e+02 2.86312347e+02 8.65545392e-01]
[ 3.43768066e+02 3.34620483e+02 7.76052594e-01]
[ 3.46382202e+02 3.72480255e+02 6.38539016e-01]
[ 3.24160156e+02 2.14515961e+02 6.87508643e-01]
[ 3.30749146e+02 2.14514191e+02 6.31730735e-01]
[ 3.16327240e+02 2.04152817e+02 7.02708900e-01]
[ 3.38600739e+02 2.05459396e+02 2.51609743e-01]
[ 3.59365387e+02 3.81637909e+02 3.02624315e-01]
[ 3.59387054e+02 3.80305817e+02 3.87343615e-01]
[ 3.47649780e+02 3.75101868e+02 3.57465029e-01]
[ 3.41147675e+02 4.10299042e+02 6.91359401e-01]
[ 3.35901062e+02 4.09023163e+02 6.37672305e-01]
[ 3.39819946e+02 3.98577362e+02 5.69236994e-01]]
[[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 3.73719215e+00 3.25488586e+02 3.63093436e-01]
[ 1.62481022e+01 3.30743683e+02 8.20027709e-01]
[ 2.53419247e+01 3.77681244e+02 8.59944463e-01]
[ 2.66692314e+01 4.02453552e+02 8.66162121e-01]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 1.49019012e+01 2.96809723e+02 8.90459180e-01]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00]]]
(output:15357): Gtk-WARNING **: 23:45:06.444: cannot open display:
在其他目录下需要使用openpose的接口，需要引用相关的models和python等目录，参考1_extract_pose.py文件里面的几个参数
# From Python
# It requires OpenCV installed for Python
import sys
import cv2
import os
from sys import platform
# Remember to add your installation path here
# Option a
dir_path = os.path.dirname(os.path.realpath(__file__))
if platform == "win32": sys.path.append(dir_path + '/../../python/openpose/');
else: sys.path.append('../../python');
# Option b
# If you run `make install` (default path is `/usr/local/python` for Ubuntu), you can also access the OpenPose/python module from there. This will install OpenPose and the python library at your desired installation path. Ensure that this is in your python path in order to use it.
# sys.path.append('/usr/local/python')
# Parameters for OpenPose. Take a look at C++ OpenPose example for meaning of components. Ensure all below are filled
try:
from openpose import *
except:
raise Exception('Error: OpenPose library could not be found. Did you enable `BUILD_PYTHON` in CMake and have this Python script in the right folder?')
params = dict()
params["logging_level"] = 3
params["output_resolution"] = "-1x-1"
params["net_resolution"] = "-1x368"
params["model_pose"] = "BODY_25"
params["alpha_pose"] = 0.6
params["scale_gap"] = 0.3
params["scale_number"] = 1
params["render_threshold"] = 0.05
# If GPU version is built, and multiple GPUs are available, set the ID here
params["num_gpu_start"] = 0
params["disable_blending"] = False
# Ensure you point to the correct path where models are located
params["default_model_folder"] = dir_path + "/../../../models/"
# Construct OpenPose object allocates GPU memory
openpose = OpenPose(params)
while 1:
# Read new image
img = cv2.imread("../../../examples/media/COCO_val2014_000000000192.jpg")
# Output keypoints and the image with the human skeleton blended on it
keypoints, output_image = openpose.forward(img, True)
# Print the human pose keypoints, i.e., a [#people x #keypoints x 3]-dimensional numpy object with the keypoints of all the people on that image
print(keypoints)
# Display the image
cv2.imshow("output", output_image)
cv2.waitKey(15)
扫一扫
1万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
