PP-Human人类属性的识别服务
一.功能说明
- 实现web服务,在页面上导入图片,输出人类属性
- 绘制在图片上,并显示
- 支持API接口
- 基于视频输入的属性识别,任务类型包含多目标跟踪和属性识别,具体如下:
#人形跟踪
- reid
- person
#人类属性
- pedestrian:pedestrian tracking.
- scores:
- gender:Male/Female
- age:['AgeLess18', 'Age18-60', 'AgeOver60']
- direction:['Front', 'Side', 'Back']
- glasses:Glasses
- hat:Hat
- hold obj:HoldObjects/HoldObjectsInFornt
- bag:['HandBag', 'ShoulderBag', 'Backpack']/No bag
- Upper:LongSleeve/ShortSleeve ['UpperStride', 'UpperLogo', 'UpperPlaid', 'UpperSplice']
- lower:['LowerStripe', 'LowerPattern', 'LongCoat', 'Trousers', 'Shorts','Skirt&Dress']
- shoe:No Boots/Boots
二.开发环境搭建
1.运行环境搭建
下载docker
然后pull镜像文件。cuda版本向下兼容
docker pull paddlepaddle/paddle:2.4.1-gpu-cuda10.2-cudnn7.6-trt7.0
创建docker
然后根据下载的镜像文件创建docker容器。
docker run --runtime nvidia \
-p 9292:9292 \
-p 9292:9292 \
--name test \
-shm-size=252G \
--network=host \
-v $PWD:/home/pphuman \
-dit paddlepaddle/paddle:2.4.1-gpu-cuda10.2-cudnn7.6-trt7.0 /bin/bash
-v $PWD:/home/pphuman:挂载工程目录
验证docker
然后用docker ps看一下当前运行的docker容器
docker ps #查看当前运行的docker容器。
docker ps -a #查看所有存在的docker容器。
结果如下:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
489e9f56374b paddlepaddle/paddle:2.4.1-gpu-cuda10.2-cudnn7.6-trt7.0 "/bin/bash" 41 hours ago Up 41 hours 22/tcp, 0.0.0.0:9292->9292/tcp test
进入docker
然后进入docker环境:
docker exec -it test /bin/bash
2.安装python依赖module
依赖module列表
cat >> requestsments.txt << EOF
numpy==1.21.6
tqdm
typeguard
visualdl>=2.2.0
opencv-python <= 4.6.0
PyYAML
shapely
scipy
terminaltables
Cython
pycocotools
setuptools
# for vehicleplate
pyclipper
# for mot
lap
motmetrics
sklearn==0.0
filterpy
# flask
Flask>=2.1.0
paddlepaddle==2.4.1
EOF
执行安装命令
pip install --upgrade pip -i https://pypi.douban.com/simple
pip install -r requirements.txt -i https://pypi.douban.com/simple
三.运行测试code
1.下载工程
git clone https://github.com/PaddlePaddle/PaddleDetection.git
cd PaddleDetection
git checkout release/2.5
2.测试demo
# 预测单张图片文件
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o MOT.enable=True REID.enable=True ATTR.enable=True --image_file "demo/hrnet_demo.jpg" --device=gpu --run_mode paddle
image_file:指定测试文件
四.配置项说明
1.配置文件
配置文件路径
deploy/pipeline/config/infer_cfg_pphuman.yml
配置文件中与属性相关的参数如下:
crop_thresh: 0.5
attr_thresh: 0.5
visual: True
#mtcnn人形检测
MOT:
model_dir: https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip
tracker_config: deploy/pipeline/config/tracker_config.yml # 车辆属性模型调用路径
batch_size: 1 # 模型预测时的batch_size大小
enable: True # 是否开启该功能
#人类属性检测
ATTR:
model_dir: https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip
batch_size: 8 # 模型预测时的batch_size大小
enable: True # 是否开启该功能
#
REID:
model_dir: https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip
batch_size: 16
enable: True
2.修改模型路径,有以下两种方式:
- 方法一:./deploy/pipeline/config/infer_cfg_pphuman.yml下可以配置不同模型路径,属性识别模型修改VEHICLE_ATTR字段下配置
- 方法二:直接在命令行中增加-o,以覆盖配置文件中的默认模型路径:
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml \
--video_file=test_video.mp4 \
--device=gpu \
-o MOT.enable=True REID.enable=True ATTR.enable=True
3.基于视频输入多目标跟踪及跨视频跟踪
多路视频跟踪人形数据
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o MOT.enable=True REID.enable=True --video_dir=demo/rtsp_videos --device=gpu --run_mode paddle
多路视频在并行跟踪生成每个视频对应的reid,通过聚类生成新的跨视频跟踪reid
PaddleDetection/deploy/pipeline/pipeline.py
def run_multithreads(self):
import threading
if self.multi_camera:
multi_res = []
threads = []
#基于多线程的方式对多路视频或视频文件并行处理
for idx, (predictor,
input) in enumerate(zip(self.predictor, self.input)):
thread = threading.Thread(
name=str(idx).zfill(3),
target=predictor.run,
args=(input, idx))
threads.append(thread)
for thread in threads:
thread.start()
for predictor, thread in zip(self.predictor, threads):
thread.join()
#获取视频跟踪的reid特征数据
collector_data = predictor.get_result()
multi_res.append(collector_data)
if self.enable_mtmct:
#对所有视频的reid数据进行聚类重新生成reid,实现跨视频跟踪
mtmct_process(
multi_res, #视频reid特征列表
self.input, #输入视频文件
mtmct_vis=self.vis_result, #重新生成聚类后reid视频数据
output_dir=self.output_dir) #生成视频存放目录
else:
self.predictor.run(self.input)
跨视频跟踪的原理
取每个视频对应的reid 质量最好的top5特征进行聚类
多视频跟踪生产对应的trackerid和特征trackerids.json
#导入特征值聚类函数
from PaddleDetection.deploy.pipeline.pphuman.mtmct import res2dict,sub_cluster
#特征值文件+trackerid
filepath = "trackerids.json"
with open(filepath,"r") as f:
multi_res = json.loads(f.read())
#tracker聚类
cid_tid_dict = res2dict(multi_res)
# if len(cid_tid_dict) == 0:
# print("no tracking result found, mtmct will be skiped.")
# return
map_tid = sub_cluster(cid_tid_dict)
print(map_tid)
聚合多路视频人形属性生成新的reid:
Using cosine as distance function during evaluation {‘c0_t4’: 1, ‘c0_t6’: 2, ‘c1_t9’: 2, ‘c0_t8’: 3, ‘c1_t2’: 3}
均值特征使用层次聚类
def get_labels(cid_tid_dict, cid_tids):
#compute cost matrix between features
cost_matrix = get_sim_matrix_new(cid_tid_dict, cid_tids)
#cluster all the features
cluster1 = AgglomerativeClustering(
n_clusters=None,
distance_threshold=0.5,
affinity='precomputed',
linkage='complete')
cluster_labels1 = cluster1.fit_predict(cost_matrix)
labels = get_match(cluster_labels1)
sub_cluster = get_cid_tid(labels, cid_tids)
return labels
层次聚类
cls = AgglomerativeClustering(n_clusters=group_size,linkage='ward')
linkage 参数说明:
- ward (默认值):每一个类簇的方差最小化
- average:每一个类簇之间的距离的平均值最小
- complete:每一个类簇之间的距离最大
- single:每一个类簇之间的距离最小
五.封装接口
1.调用工程模块,切换运行环境
import base64
import json
import logging
import os
import sys
import time
import paddle
import numpy as np
#运行环境切换到调用工程目录
ROOT_DIR = os.path.abspath(os.path.dirname(__file__))
PRO_PATH = os.path.join(ROOT_DIR,"PaddleDetection")
ENV_PATH = os.path.join(PRO_PATH,"deploy/pipeline")
WEIGHTS_PATH = os.path.join(ROOT_DIR,"weights/paddle/infer_weights")
sys.path.append(ENV_PATH)
from PaddleDetection.deploy.pipeline.pipeline import argsparser,merge_cfg,print_arguments,Pipeline
2.初始化模型
def init_pphuman():
"""初始化人类属性识别
Returns:
_type_: 人类属性识别对象
"""
paddle.enable_static()
# parse params from command
parser = argsparser()
# python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml --image_file=./deploy/pipeline/docs/images/pphumanplate.jpg --device=gpu
# python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o MOT.enable=True REID.enable=True --video_dir=demo/rtsp_videos --device=gpu --run_mode paddle
# python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml --video_dir=demo/rtsp_videos --device=gpu
# 输入初始化参数
FLAGS = parser.parse_args([
#测试结果输出到指定目录
'--output_dir','./out',
# 检测属性
"-o",f"MOT.enable=True",f"REID.enable=True",f"ATTR.enable=True",
#配置文件路径
'--config',os.path.join(PRO_PATH,'deploy/pipeline/config/infer_cfg_pphuman.yml'),
# 测试图片/视频路径
# '--video_dir',video_dir,
# '--rtsp',rtsp,
"--image_file","PaddleDetection/demo/hrnet_demo.jpg",
# #default:CPU,Choose the device you want to run, it can be: CPU/GPU/XPU, default is CPU.
'--device',"GPU"
# #default:paddle,mode of running(paddle/trt_fp32/trt_fp16/trt_int8)
'--run_mode',"paddle"
])
FLAGS.device = FLAGS.device.upper()
cfg = merge_cfg(FLAGS) # use command params to update config
print_arguments(cfg)
pipeline = Pipeline(FLAGS, cfg)
return pipeline
3.执行检测识别
def exec_pphuman(pipeline,input = None,output = None):
"""人类属性识别
Args:
pipeline (_type_): 人类属性识别对象
input (_type_, optional): 输入路径. Defaults to None.
output (_type_, optional): 输出目录. Defaults to None.
Returns:
_type_: _description_
"""
if input:
#输入路径['']
pipeline.input = input
if output:
#修改测试结果输出目录
pipeline.predictor.output_dir = output
# 执行人类检测识别
pipeline.run()
#返回结果
return pipeline.predictor.pipeline_res.res_dict
4.检测结果解析
def result_pphuman(out_path,res_dict,isprit=False):
"""人类属性识别返回结果解析
Args:
out_path (_type_): 输出图片路径
res_dict (_type_): 识别结果,返回结构化结果
isprit (bool, optional): 特殊格式. Defaults to False.
Returns:
_type_: _description_
"""
out_dict = {}
logging.warning(out_path)
img = None
if os.path.exists(out_path):
with open(out_path,"rb") as f:
img = str(base64.b64encode(f.read()),'utf-8')
boxes = res_dict["det"]["boxes"]
human_attrs = res_dict["attr"]["output"]
rec_res = []
num = 0
for box,attrs in zip(boxes,human_attrs):
score = box[1]
x1,y1,x2,y2 = box[2:]
if isprit:
rec_res.append([num,",".join(attrs),float(f'{score:.3f}')])
else:
atrr_dict = dict(attr.split(":") for attr in attrs)
atrr_dict["score"] = float(f'{score:.3f}')
atrr_dict["box"]={
"x1":float(f'{x1:.3f}'),
"y1":float(f'{y1:.3f}'),
"x2":float(f'{x2:.3f}'),
"y2":float(f'{y2:.3f}'),
}
rec_res.append(atrr_dict)
num = num + 1
out_dict["dets"] = rec_res
logging.warning(out_dict)
out_dict["image"] = img
return out_dict
六.开发Api,并Postman测试
1.Flask运行接口搭建
# -*- encoding: utf-8 -*-
# @Author: fy
# @Contact: yu.fu@deepcam.com
import os
import base64
import json
import time
import logging
from wsgiref.simple_server import make_server
import cv2
import numpy as np
from flask import Flask, render_template, request
from PPHuman import init_pphuman,exec_pphuman,result_pphuman
Logger.get_logger(cfg.PROJECT_NAME, level=cfg.LOGGING_LEVEL, log_dirs=cfg.LOG_DIR)
app = Flask(__name__)
# 调整接收文本大小
app.config['MAX_CONTENT_LENGTH'] = 30 * 1024 * 1024
@app.route('/')
def index():
return render_template('index.html')
@app.route('/api/v1/human/detect', methods=['POST'])
def ocr_idcard():
if request.method == 'POST':
try:
#接收检测图片
json_obj = request.get_json()
logging.warning(json_obj)
img_str = json_obj["image"]
image = base64.b64decode(img_str + '=' * (-len(img_str) % 4))
nparr = np.frombuffer(image, np.uint8)
except Exception as e:
logging.error(e)
return json.dumps({"code": -1001,"msg": u"Incomplete parameters","data": {}})
try:
image = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
if image.ndim == 2:
image = cv2.cvtColor(image, cv2.COLOR_GRAY2BGR)
dir_path = cfg.get_ramdisk_dir()
in_dir = os.path.join(dir_path,"in")
out_dir = os.path.join(dir_path,"out")
if not os.path.exists(in_dir):
os.makedirs(in_dir)
if not os.path.exists(out_dir):
os.makedirs(out_dir)
filename = f"{int(time.time()*1000000)}.jpg"
image_path = os.path.join(in_dir,filename)
cv2.imwrite(image_path,image)
except Exception as e:
logging.error(e)
return json.dumps({"code": -1101,"msg": u"Detection failed","data": {}})
image = None
out_dict = {}
try:
t1 = time.time()
# 3.执行检测识别
res_dict = exec_pphuman(pipeline=pphuman,input=[image_path],output=out_dir)
elapse = time.time() - t1
logging.warning(res_dict)
logging.warning(f'total_elapse: {elapse:.4f}')
out_path = os.path.join(out_dir,filename)
# 4.检测结果解析,格式化输出结果
out_dict = result_pphuman(out_path,res_dict)
except Exception as e:
logging.error(e)
return json.dumps({"code": -1102,"msg": u"Parsing failed","data": {}})
logging.warning(f'out_dict:{out_dict}')
return json.dumps({"code": 1000,"msg": "success","data": out_dict})
if __name__ == '__main__':
# 1.初始化模型,创建人类属性识别对象
pphuman = init_pphuman()
# 2.对象第一次执行,可能存在第一次耗时较长的问题
exec_pphuman(pipeline=pphuman)
ip = '0.0.0.0'
ip_port = 9990
logging.warning(f"http://{ip}:{ip_port}")
server = make_server(ip, ip_port, app)
server.serve_forever()
2.调用识别封装接口
- 初始化模型,创建人类属性识别对象
- 对象第一次执行,可能存在第一次耗时较长的问题
- 执行检测识别
- 检测结果解析,格式化输出结果