先说结论:
- 在CPU上,INT8略快;在GPU上,FP16最快;
- 使用GPU比使用CPU节约80%左右的CPU开销;
- 使用GPU会比使用CPU占用更多的内存,不知道是不是因为显卡没有现存,共享内存导致的;
- 模型加载时间差异很大,使用GPU可以大大节省CPU资源,但初始化加载模型的过程较慢;
测试环境:intel core i5 8500
应用:human_pose_estimation_demo
模型:human-pose-estimation-0001
数据:MOT16/test/MOT16-06
耗时(ms) | CPU | GPU |
---|---|---|
FP16-INT8 | 124559 | 179275 |
FP16 | 128788 | 81001.2 |
FP32 | 127422 | 103820 |
模型加载时间:
- CPU FP16-INT8:小于2s;
- GPU FP16:大约53s;
CPU占用率差异:
- CPU FP16-INT8:290%
- GPU FP16:60%
内存占用差异:
- CPU FP16-INT8:VIRT:1.18G,RES:136M
- GPU FP16:VIRT:1.59G,RES:528M
日志:
root@openpose:/openpose_lite# /root/omz_demos_build/intel64/Release/human_pose_estimation_demo -i MOT16/test/MOT16-06/img1/%6d.jpg -no_show -m intel/human-pose-estimation-0001/FP16-INT8/human-pose-estimation-0001.xml
InferenceEngine: 0x7f3ae5583030
Parsing input parameters
Reshape needed
To close the application, press ‘CTRL+C’ here
Total Inference time: 124559
Execution successful
root@openpose:/openpose_lite# /root/omz_demos_build/intel64/Release/human_pose_estimation_demo -i MOT16/test/MOT16-06/img1/%6d.jpg -no_show -m intel/human-pose-estimation-0001/FP16/human-pose-estimation-0001.xml
InferenceEngine: 0x7fbb0eaa2030
Parsing input parameters
Reshape needed
To close the application, press ‘CTRL+C’ here
Total Inference time: 128788
Execution successful
root@openpose:/openpose_lite# /root/omz_demos_build/intel64/Release/human_pose_estimation_demo -i MOT16/test/MOT16-06/img1/%6d.jpg -no_show -m intel/human-pose-estimation-0001/FP32/human-pose-estimation-0001.xml
InferenceEngine: 0x7f9136f9f030
Parsing input parameters
Reshape needed
To close the application, press ‘CTRL+C’ here
Total Inference time: 127422
Execution successful
root@openpose:/openpose_lite# /root/omz_demos_build/intel64/Release/human_pose_estimation_demo -i MOT16/test/MOT16-06/img1/%6d.jpg -no_show -d GPU -m intel/human-pose-estimation-0001/FP32/human-pose-estimation-0001.xml
InferenceEngine: 0x7f1a2209e030
Parsing input parameters
Reshape needed
To close the application, press ‘CTRL+C’ here
Total Inference time: 103820
Execution successful
root@openpose:/openpose_lite# /root/omz_demos_build/intel64/Release/human_pose_estimation_demo -i MOT16/test/MOT16-06/img1/%6d.jpg -no_show -d GPU -m intel/human-pose-estimation-0001/FP16/human-pose-estimation-0001.xml
InferenceEngine: 0x7fe9f13a0030
Parsing input parameters
Reshape needed
To close the application, press ‘CTRL+C’ here
Total Inference time: 81001.2
Execution successful
root@openpose:/openpose_lite# /root/omz_demos_build/intel64/Release/human_pose_estimation_demo -i MOT16/test/MOT16-06/img1/%6d.jpg -no_show -d GPU -m intel/human-pose-estimation-0001/FP16-INT8/human-pose-estimation-0001.xml
InferenceEngine: 0x7fe0ce749030
Parsing input parameters
Reshape needed
To close the application, press ‘CTRL+C’ here
Total Inference time: 179275
Execution successful