PP-Human人类属性的识别服务

weixin_44929001

已于 2023-02-28 17:08:42 修改

阅读量641

点赞数

分类专栏：人工智能文章标签： docker 容器运维

于 2023-02-22 20:18:17 首次发布

本文链接：https://blog.csdn.net/weixin_44929001/article/details/129168984

版权

人工智能专栏收录该内容

4 篇文章 1 订阅

订阅专栏

PP-Human人类属性的识别服务

一.功能说明

实现web服务，在页面上导入图片，输出人类属性
绘制在图片上，并显示
支持API接口
基于视频输入的属性识别，任务类型包含多目标跟踪和属性识别，具体如下：

#人形跟踪
- reid
- person

#人类属性
- pedestrian:pedestrian tracking.
- scores:
- gender:Male/Female
- age:['AgeLess18', 'Age18-60', 'AgeOver60']
- direction:['Front', 'Side', 'Back']
- glasses:Glasses
- hat:Hat
- hold obj:HoldObjects/HoldObjectsInFornt
- bag:['HandBag', 'ShoulderBag', 'Backpack']/No bag
- Upper:LongSleeve/ShortSleeve ['UpperStride', 'UpperLogo', 'UpperPlaid', 'UpperSplice']
- lower:['LowerStripe', 'LowerPattern', 'LongCoat', 'Trousers', 'Shorts','Skirt&Dress']
- shoe:No Boots/Boots

二.开发环境搭建

1.运行环境搭建

下载docker

然后pull镜像文件。cuda版本向下兼容

docker pull paddlepaddle/paddle:2.4.1-gpu-cuda10.2-cudnn7.6-trt7.0

创建docker

然后根据下载的镜像文件创建docker容器。

docker  run --runtime nvidia \
    -p 9292:9292 \
    -p 9292:9292 \
    --name test \
    -shm-size=252G \
    --network=host \
    -v $PWD:/home/pphuman \
    -dit paddlepaddle/paddle:2.4.1-gpu-cuda10.2-cudnn7.6-trt7.0 /bin/bash

-v $PWD:/home/pphuman:挂载工程目录

验证docker

然后用docker ps看一下当前运行的docker容器

docker ps     #查看当前运行的docker容器。
docker ps -a  #查看所有存在的docker容器。

结果如下：

CONTAINER ID   IMAGE                                           COMMAND       CREATED             STATUS             PORTS     NAMES
489e9f56374b        paddlepaddle/paddle:2.4.1-gpu-cuda10.2-cudnn7.6-trt7.0                     "/bin/bash"              41 hours ago        Up 41 hours                 22/tcp, 0.0.0.0:9292->9292/tcp      test

进入docker

然后进入docker环境：

docker exec -it test /bin/bash

2.安装python依赖module

依赖module列表

cat >> requestsments.txt << EOF

numpy==1.21.6

tqdm
typeguard
visualdl>=2.2.0
opencv-python <= 4.6.0
PyYAML
shapely
scipy
terminaltables
Cython
pycocotools
setuptools

# for vehicleplate
pyclipper

# for mot
lap
motmetrics
sklearn==0.0
filterpy

# flask
Flask>=2.1.0

paddlepaddle==2.4.1

EOF

执行安装命令

pip install --upgrade pip -i https://pypi.douban.com/simple
pip install -r requirements.txt -i https://pypi.douban.com/simple

三.运行测试code

1.下载工程

git clone https://github.com/PaddlePaddle/PaddleDetection.git
cd PaddleDetection
git checkout release/2.5

2.测试demo


# 预测单张图片文件
python deploy/pipeline/pipeline.py  --config deploy/pipeline/config/infer_cfg_pphuman.yml -o MOT.enable=True REID.enable=True ATTR.enable=True --image_file "demo/hrnet_demo.jpg" --device=gpu --run_mode paddle

image_file:指定测试文件

四.配置项说明

1.配置文件

配置文件路径
deploy/pipeline/config/infer_cfg_pphuman.yml

配置文件中与属性相关的参数如下：

crop_thresh: 0.5
attr_thresh: 0.5
visual: True

#mtcnn人形检测
MOT:
  model_dir: https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip
  tracker_config: deploy/pipeline/config/tracker_config.yml # 车辆属性模型调用路径
  batch_size: 1 # 模型预测时的batch_size大小
  enable: True # 是否开启该功能

#人类属性检测
ATTR:
  model_dir:  https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip
  batch_size: 8 # 模型预测时的batch_size大小
  enable: True # 是否开启该功能

#
REID:
  model_dir:  https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip
  batch_size: 16
  enable: True

2.修改模型路径，有以下两种方式：

方法一：./deploy/pipeline/config/infer_cfg_pphuman.yml下可以配置不同模型路径，属性识别模型修改VEHICLE_ATTR字段下配置
方法二：直接在命令行中增加-o，以覆盖配置文件中的默认模型路径：

python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml \
                                                   --video_file=test_video.mp4 \
                                                   --device=gpu \
                                                   -o MOT.enable=True REID.enable=True ATTR.enable=True

3.基于视频输入多目标跟踪及跨视频跟踪

多路视频跟踪人形数据

python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o MOT.enable=True REID.enable=True --video_dir=demo/rtsp_videos --device=gpu --run_mode paddle

多路视频在并行跟踪生成每个视频对应的reid，通过聚类生成新的跨视频跟踪reid

PaddleDetection/deploy/pipeline/pipeline.py

    def run_multithreads(self):
        import threading
        if self.multi_camera:
            multi_res = []
            threads = []
            #基于多线程的方式对多路视频或视频文件并行处理
            for idx, (predictor,
                      input) in enumerate(zip(self.predictor, self.input)):
                thread = threading.Thread(
                    name=str(idx).zfill(3),
                    target=predictor.run,
                    args=(input, idx))
                threads.append(thread)

            for thread in threads:
                thread.start()

            for predictor, thread in zip(self.predictor, threads):
                thread.join()
                #获取视频跟踪的reid特征数据
                collector_data = predictor.get_result()
                multi_res.append(collector_data)

            if self.enable_mtmct:
            	#对所有视频的reid数据进行聚类重新生成reid,实现跨视频跟踪
                mtmct_process(
                    multi_res,                  #视频reid特征列表
                    self.input,                 #输入视频文件
                    mtmct_vis=self.vis_result,  #重新生成聚类后reid视频数据
                    output_dir=self.output_dir) #生成视频存放目录

        else:
            self.predictor.run(self.input)

跨视频跟踪的原理

取每个视频对应的reid 质量最好的top5特征进行聚类

多视频跟踪生产对应的trackerid和特征trackerids.json

#导入特征值聚类函数
from PaddleDetection.deploy.pipeline.pphuman.mtmct import res2dict,sub_cluster

#特征值文件+trackerid
filepath = "trackerids.json"
with open(filepath,"r") as f:
    multi_res = json.loads(f.read())
 
#tracker聚类
cid_tid_dict = res2dict(multi_res)
# if len(cid_tid_dict) == 0:
#     print("no tracking result found, mtmct will be skiped.")
#     return
map_tid = sub_cluster(cid_tid_dict)
print(map_tid)

聚合多路视频人形属性生成新的reid:

Using cosine as distance function during evaluation {‘c0_t4’: 1, ‘c0_t6’: 2, ‘c1_t9’: 2, ‘c0_t8’: 3, ‘c1_t2’: 3}

均值特征使用层次聚类

def get_labels(cid_tid_dict, cid_tids):
    #compute cost matrix between features
    cost_matrix = get_sim_matrix_new(cid_tid_dict, cid_tids)
    #cluster all the features
    cluster1 = AgglomerativeClustering(
        n_clusters=None,
        distance_threshold=0.5,
        affinity='precomputed',
        linkage='complete')
    cluster_labels1 = cluster1.fit_predict(cost_matrix)
    labels = get_match(cluster_labels1)

    sub_cluster = get_cid_tid(labels, cid_tids)
    return labels

层次聚类

cls = AgglomerativeClustering(n_clusters=group_size,linkage='ward')

linkage 参数说明:

ward (默认值)：每一个类簇的方差最小化
average：每一个类簇之间的距离的平均值最小
complete：每一个类簇之间的距离最大
single：每一个类簇之间的距离最小

五.封装接口

1.调用工程模块,切换运行环境

import base64
import json
import logging
import os
import sys
import time

import paddle
import numpy as np
#运行环境切换到调用工程目录
ROOT_DIR = os.path.abspath(os.path.dirname(__file__))
PRO_PATH = os.path.join(ROOT_DIR,"PaddleDetection")
ENV_PATH = os.path.join(PRO_PATH,"deploy/pipeline")
WEIGHTS_PATH = os.path.join(ROOT_DIR,"weights/paddle/infer_weights")
sys.path.append(ENV_PATH)
from PaddleDetection.deploy.pipeline.pipeline import argsparser,merge_cfg,print_arguments,Pipeline

2.初始化模型

def init_pphuman():
    """初始化人类属性识别
    Returns:
        _type_: 人类属性识别对象
    """
    paddle.enable_static()

    # parse params from command
    parser = argsparser()
    # python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml --image_file=./deploy/pipeline/docs/images/pphumanplate.jpg --device=gpu
    # python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o MOT.enable=True REID.enable=True --video_dir=demo/rtsp_videos --device=gpu --run_mode paddle


    # python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml --video_dir=demo/rtsp_videos --device=gpu
    # 输入初始化参数
    FLAGS = parser.parse_args([
                            #测试结果输出到指定目录
                            '--output_dir','./out',
                            # 检测属性
                            "-o",f"MOT.enable=True",f"REID.enable=True",f"ATTR.enable=True",
                            #配置文件路径
                            '--config',os.path.join(PRO_PATH,'deploy/pipeline/config/infer_cfg_pphuman.yml'),
                            # 测试图片/视频路径
                            # '--video_dir',video_dir,
                            # '--rtsp',rtsp,
                            "--image_file","PaddleDetection/demo/hrnet_demo.jpg",
                            # #default:CPU,Choose the device you want to run, it can be: CPU/GPU/XPU, default is CPU.
                            '--device',"GPU"
                            # #default:paddle,mode of running(paddle/trt_fp32/trt_fp16/trt_int8)
                            '--run_mode',"paddle"
                            ])
    FLAGS.device = FLAGS.device.upper()
    cfg = merge_cfg(FLAGS)  # use command params to update config
    print_arguments(cfg)
    pipeline = Pipeline(FLAGS, cfg)
    return pipeline

3.执行检测识别

def exec_pphuman(pipeline,input = None,output = None):
    """人类属性识别
    Args:
        pipeline (_type_): 人类属性识别对象
        input (_type_, optional): 输入路径. Defaults to None.
        output (_type_, optional): 输出目录. Defaults to None.
    Returns:
        _type_: _description_
    """
    if input:
        #输入路径['']
        pipeline.input = input
    if output:
        #修改测试结果输出目录
        pipeline.predictor.output_dir = output
    # 执行人类检测识别
    pipeline.run()
    #返回结果
    return pipeline.predictor.pipeline_res.res_dict

4.检测结果解析

def result_pphuman(out_path,res_dict,isprit=False):
    """人类属性识别返回结果解析
    Args:
        out_path (_type_): 输出图片路径
        res_dict (_type_): 识别结果，返回结构化结果
        isprit (bool, optional): 特殊格式. Defaults to False.
    Returns:
        _type_: _description_
    """
    out_dict = {}
    logging.warning(out_path)
    img = None
    if os.path.exists(out_path):
        with open(out_path,"rb") as f:
            img = str(base64.b64encode(f.read()),'utf-8')
    boxes = res_dict["det"]["boxes"]
    human_attrs = res_dict["attr"]["output"]
    
    rec_res = []
    num = 0
    for box,attrs in zip(boxes,human_attrs):
        score = box[1]
        x1,y1,x2,y2 = box[2:]
        if isprit:
            rec_res.append([num,",".join(attrs),float(f'{score:.3f}')])
        else:
            atrr_dict = dict(attr.split(":") for attr in attrs)
            atrr_dict["score"] = float(f'{score:.3f}')
            atrr_dict["box"]={
                "x1":float(f'{x1:.3f}'),
                "y1":float(f'{y1:.3f}'),
                "x2":float(f'{x2:.3f}'),
                "y2":float(f'{y2:.3f}'),
                }
            rec_res.append(atrr_dict)
        num = num + 1
    out_dict["dets"] = rec_res
    logging.warning(out_dict)
    out_dict["image"] = img
    return out_dict

六.开发Api,并Postman测试

1.Flask运行接口搭建

# -*- encoding: utf-8 -*-
# @Author: fy
# @Contact: yu.fu@deepcam.com
import os
import base64
import json
import time
import logging
from wsgiref.simple_server import make_server

import cv2
import numpy as np
from flask import Flask, render_template, request



from PPHuman import init_pphuman,exec_pphuman,result_pphuman
Logger.get_logger(cfg.PROJECT_NAME, level=cfg.LOGGING_LEVEL, log_dirs=cfg.LOG_DIR)
app = Flask(__name__)
# 调整接收文本大小
app.config['MAX_CONTENT_LENGTH'] = 30 * 1024 * 1024



@app.route('/')
def index():
    return render_template('index.html')

        
@app.route('/api/v1/human/detect', methods=['POST'])
def ocr_idcard():
    if request.method == 'POST':
        try:
            #接收检测图片
            json_obj = request.get_json()
            logging.warning(json_obj)
            img_str = json_obj["image"]

            image = base64.b64decode(img_str + '=' * (-len(img_str) % 4))
            nparr = np.frombuffer(image, np.uint8)
        except Exception as e:
            logging.error(e)
            return json.dumps({"code": -1001,"msg": u"Incomplete parameters","data": {}})
        try:
            image = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
            
            if image.ndim == 2:
                image = cv2.cvtColor(image, cv2.COLOR_GRAY2BGR)
            
            dir_path = cfg.get_ramdisk_dir()
            in_dir = os.path.join(dir_path,"in")
            out_dir = os.path.join(dir_path,"out")
            if not os.path.exists(in_dir):
                os.makedirs(in_dir)
            if not os.path.exists(out_dir):
                os.makedirs(out_dir)
            filename = f"{int(time.time()*1000000)}.jpg"
            image_path = os.path.join(in_dir,filename)
            cv2.imwrite(image_path,image)
        except Exception as e:
            logging.error(e)
            return json.dumps({"code": -1101,"msg": u"Detection failed","data": {}})
        image = None
        out_dict = {}
        try:
            t1 = time.time()
            # 3.执行检测识别
            res_dict = exec_pphuman(pipeline=pphuman,input=[image_path],output=out_dir)
            elapse = time.time() - t1
            logging.warning(res_dict)
            logging.warning(f'total_elapse: {elapse:.4f}')
            out_path = os.path.join(out_dir,filename)
            # 4.检测结果解析,格式化输出结果
            out_dict = result_pphuman(out_path,res_dict)
        except Exception as e:
            logging.error(e)
            return json.dumps({"code": -1102,"msg": u"Parsing failed","data": {}})
        logging.warning(f'out_dict:{out_dict}')
        return json.dumps({"code": 1000,"msg": "success","data": out_dict})
if __name__ == '__main__':
    # 1.初始化模型，创建人类属性识别对象
    pphuman = init_pphuman()
    # 2.对象第一次执行，可能存在第一次耗时较长的问题
    exec_pphuman(pipeline=pphuman)
    ip = '0.0.0.0'
    ip_port = 9990
    logging.warning(f"http://{ip}:{ip_port}")
    server = make_server(ip, ip_port, app)
    server.serve_forever()