基于matlab平台
视频demo:bilibili
代码:GitHub
流程:
提取骨骼信息——绘制骨架线(但不显示)——存储骨架线数据为图片——对骨架线进行cnn识别——显示RGB图像(视频)并在其上绘制骨架线——将识别结果显示在上面
- 先设置一堆东西,close all必须有,否则识别过程中,只有第一次运行ok,以后在第二步骨架线会一直闪窗。
%% 动态识别动作,画在黑白骨架线上,两个图(不用读取白图,不会弹)
close all;
imaqhwinfo
colorVid = videoinput('kinect',1);
depthVid = videoinput('kinect',2);
triggerconfig(depthVid,'manual');
triggerconfig(colorVid,'manual');
depthVid.FramesPerTrigger = 1;
depthVid.TriggerRepeat = inf;
set(getselectedsource(depthVid),'EnableBodyTracking','on')
%triggerconfig(colorVid,'manual');%可删?和上面重复了但是我记得我cp上会有问题
%triggerconfig(depthVid,'manual');%可删?
colorVid.FramesPerTrigger = 1;
colorVid.TriggerRepeat = inf;
- 加载训练好的网络skeletonnet,然后一帧一帧地采集、识别、显示
nnet = skeletonnet
start(depthVid);
start(colorVid);
image2 = 255.*ones(1080,1920,3,'uint8');
himg = figure(2);
while ishandle(himg)
trigger(colorVid);
trigger(depthVid);
image = getdata(colorVid);
[depthMap,~,depthMetadata] = getdata(depthVid);
if sum(depthMetadata.IsBodyTracked) > 0
trackedSkeletons = find(depthMetadata.IsBodyTracked);
skeletonJoints = depthMetadata.ColorJointIndices(:, :, trackedSkeletons);
fh = figure(1);
set(fh,'Visible','off');
skeletonViewer2(skeletonJoints,image2,1);
I = getframe(gcf);
pic = I.cdata;
picture = imresize(pic,[227,227]);
label = classify(nnet, picture);
close;
fl = figure(2);
set(fl,'Visible','on');
skeletonViewer2(skeletonJoints,image,1);
title(char(label));
drawnow;
else
imshow(image);
end
end
stop(colorVid);
stop(depthVid);