基于人体关节夹角的人体动作识别算法（代码+数据集）

 为此本文提出了一个基于人体关节夹角的人体动作识别算法，主要做了以下工作：
（1）提出了一个可解释性强，耗费算力较少且鲁棒性较高的基于人体关节夹角的人体动作序列的特征抽取方法。
（2）本文所使用的分类模型是一个融合了SVM，lightgbm等6个分类模型的集成学习模型，并且使用了一个结合了Boosting和Bagging的集成学习策略。
最终本论文提出的方法在G3D数据集上的分类准确率为92.3%。该结果验证论文中方法的可行性

论文提出的算法的流程图

本文当中所提出的人体动作分类算法如上图所示，本算法主要分为人体3D关节点坐标获取，特征工程，模型集成，模型训练4个部分:
（1）人体关节点坐标获取部分：先使用Openpose的body25预训练模型结合训练集自带的深度图来提取出人体关键点3D坐标
（2）特征工程部分：先抽取人体动作特征向量，具体方法为：
①在每一个人体动作序列中等间距取50张图片
②基于得到的人体关键点3D坐标，每一张图片我们都可以得到 24 个肢体向量。然后通过24 个肢体向量两两之间的夹角的余弦值，可以得到一个长度为276的向量
③将50张图片得到的向量按照顺序拼接成长为13800的向量而后用基于方差的Filter方法和基于随机森林的RFE方法来特征选择。
（3）模型集成部分：因为logistic回归拟合能力较弱，所以先让它进行Adaboost增强拟合能力，而后再把它和SVM，KNN，随机森林，Xgboost，lightgbm一起进行基于软投票策略的Bagging方法后输出最终的分类结果。
（4）模型训练部分：按照训练集和测试集5:5的比例划分数据集，而后按照划分的数据集使用网格调参法调出每一个基分类模型的最佳参数，而后将其带入集成学习模型当中进行最终的模型分类预测。

论文方法的实验以及实验结果结果

本文先使用网格调参法调出基分类器的最佳参数，具体调参的过程如下图所示：

根据调参的结果，各个基分类器的最佳参数如下：
（1）logistic回归：“L2”惩罚项，C取0.214
（2）支持向量机：核函数取“poly”，C取0.238
（3）KNN：使用kdtree进行搜索，K值取3
（4）随机森林：estimators的最佳参数为142，criterion选择“entropy”这个选项
（5）Xgboost：最佳estimators的个数为25
（6）Lightgbm：最佳estimators的个数为67
（7）在实验当中Adaboost中的基分类器的个数设定为100
将调参结果输入到集成分类学习模型当中以后分类准确率达到了92.3%，得到的分类结果的混淆矩阵如下图所示：

一些重要东西的下载

①数据集使用的是G3D数据集：call me
②Openpose预处理以后的数据： call me 

代码运行

cd ./code
①集成分类模型运行结果 python proposed_method.py
②SVM画图 python SVM.py
①KNN画图 python KNN.py
④随机森林画图 python randomforest.py
⑤xgboost和Lightgbm画图 python xgboost_and_lightgbm.py
⑥logistic画图 python logistic.py
⑦特征抽取和特征选择：先把Openpose预处理以后的数据解压好以后放到code/data目录下，而后运行 python feature_extracting.py ,但是使用这个代码运行所得到的特征抽取结果来运行模型就可能得到和论文得到的结果不一样，因为RFE的运行结果每一次都会有细小的差异
⑧如果你想直接使用论文实验的数据的话就把Experimental data 文件夹里面的数据放到code文件夹下面，这样就不用再跑一遍特征抽取了.

feature_extracting.py

import  numpy as np
import pandas as pd
import os
import sklearn as sk
import matplotlib.pyplot as plt
import json
import math
import os
from PIL import Image
from sklearn.feature_selection import SelectFromModel
from sklearn.ensemble import RandomForestClassifier as RFC
from sklearn.feature_selection import RFE
from sklearn.feature_selection import VarianceThreshold
def fun(x):
    return x >> 3
def depth2xyz(depth_map, depth_cam_matrix, flatten=False, depth_scale=1000):
    fx, fy = depth_cam_matrix[0, 0], depth_cam_matrix[1, 1]
    cx, cy = depth_cam_matrix[0, 2], depth_cam_matrix[1, 2]
    h, w = np.mgrid[0:depth_map.shape[0], 0:depth_map.shape[1]]
    z = depth_map/518
    x = w
    y = h
    xyz = np.dstack((x, y, z))
    return xyz

def getdata(filename1,filename2):
    jsonname = filename1
    picurename = filename2
    picurename = picurename.replace('Colour', 'Depth')
    with open(jsonname) as f:
        keypoints = json.load(f)
    body25 = keypoints['people'][0]['pose_keypoints_2d']
    img = Image.open(picurename)
    img = np.array(img)
    img = fun(img)
    img = np.where(img > 8192, img - 8192, img)

    depth_map = img.T

    depth_cam_matrix = np.array([[518, 0, 325.5],
                                 [0, 519, 253.5],
                                 [0, 0, 1]])
    targrt = depth2xyz(depth_map, depth_cam_matrix)

    x = []
    y = []
    z = []
    for i in range(0, 74, 3):
        D = targrt[int(body25[i])-1][int(body25[i + 1])-1]
        x.append(body25[i])
        y.append(body25[i + 1])
        z.append(D[2])

    return x, y, z


def VectorCosine(x, y):
    a = 0
    b = 0
    c = 0

    for i in range(0, len(x)):
        a += x[i] * y[i]
        b += x[i] * x[i]
        c += y[i] * y[i]
        c += 0.00001
        b += 0.00001
    result = a / (math.sqrt(b) * math.sqrt(c))
    return result


def cosvalues(filename1,filename2):
    x, y, z = getdata(filename1,filename2)
    result = []
    bodys = [
        [15, 17], [15, 0], [16, 0], [16, 18], [0, 1], [2, 1], [8, 1], [5, 1],
        [2, 3], [3, 4], [5, 6], [6, 7], [8, 9], [8, 12], [9, 10], [12, 13],
        [10, 11], [13, 14], [11, 24], [14, 21], [11, 22], [14, 19], [22, 23], [19, 20]
    ]
    i = 0
    bodys1 = bodys
    lengths = []

    for body in bodys:
        for body1 in bodys1[i + 1:24:1]:
            vector1 = np.array([x[body[0]] - x[body[1]], y[body[0]] - y[body[1]], z[body[0]] - z[body[1]]])
            vector2 = np.array([x[body1[0]] - x[body1[1]], y[body1[0]] - y[body1[1]], z[body1[0]] - z[body1[1]]])
            array = np.array([np.linalg.norm(vector1), np.linalg.norm(vector2)])

            cosine = VectorCosine(vector1, vector2)
            result.append(cosine)
        i = i + 1

    return result
Root="data/feature-extracting-result"
Sports=os.listdir(Root)
print(Sports)

for Sport in Sports:
    sportpath=Root+'/'+Sport
    Actors=os.listdir(sportpath)
    for Actor in Actors:
        Actionspath=sportpath+'/'+Actor
        Actions=os.listdir(Actionspath)
        for Action in Actions:
            Finalpath = Actionspath + '/' + Action
            for i in np.linspace(1, 1, 1):
                i = int(i)
                jsonpath = Finalpath+ '/output/' + str(i) + '_keypoints.json'
                depthpath = Finalpath+ '/depth/' + str(i) + '.png'

                Matrix = np.array(cosvalues(jsonpath, depthpath))

            for i in np.linspace(2, 50, 49):
                i = int(i)
                jsonpath = Finalpath+ '/output/' + str(i) + '_keypoints.json'
                depthpath = Finalpath+ '/depth/' + str(i) + '.png'

                Vector = np.array(cosvalues(jsonpath, depthpath))
                Matrix = np.vstack((Matrix, Vector))
            Matrixname = Finalpath + '/' + "Matrix.csv"
            Matrix.tofile(Matrixname, sep=',', format='%1.5f')
    print(Sport,"is Done")
Root="data/feature-extracting-result"
sports=os.listdir(Root)
Class=0
'''Bowling是,0，Drving是1.Fighting是2，FPS是3，Golf是4，Misc是5，Tennis是6'''
result=pd.read_csv("data/feature-extracting-result/Bowling/Actor_1/1/Matrix.csv",header=None)
result["class"]=1
data=pd.DataFrame(columns=result.columns)
for sport in sports:
    Actors=os.listdir(Root+"/"+sport)
    for Actor in Actors:
        bianshus=os.listdir(Root+"/"+sport+"/"+Actor)
        for bianshu in bianshus:
            filename=Root + "/" + sport + "/" + Actor + "/" + bianshu+"/Matrix.csv"
            data1=pd.read_csv(filename,header = None)
            data1["class"]=Class
            data=data.append(data1)
    Class=Class+1
data.to_csv("future_extracting_result.csv")
data=pd.read_csv("future_extracting_result.csv")
target=data["class"]
traindata=data.drop(["class"],axis=1)
sel = VarianceThreshold(threshold=0.16)
###选取0.16的原因：若特征是伯努利随机变量，假设p=0.8，即二分类特征中某种分类占到80%以上的时候删除特征
#因为有80%都是相似数据，对于数据的整体的特征已经没有太多的区分意义
traindata = sel.fit_transform(traindata)
print("variance filtler is Done")
RFC_ = RFC(n_estimators =25,random_state=0)
traindata = RFE(RFC_,n_features_to_select=500).fit_transform(traindata,target)
print("RFE via randomforest is Done")
selectdata=pd.DataFrame(traindata)
selectdata.to_csv("future_selection_result.csv",index=False)
print("Feature extraction is Done")

 logistic.py

import numpy
import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.feature_selection import VarianceThreshold
from sklearn.svm import SVC
import numpy as np
import pandas as pd
from lightgbm import LGBMClassifier
from sklearn import svm
from sklearn.feature_selection import SelectFromModel
from sklearn.ensemble import RandomForestClassifier as RFC
from xgboost import XGBClassifier
from sklearn.neighbors import  KNeighborsClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.feature_selection import RFE
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif']='SimHei' #设置字体
plt.rcParams["font.size"]=10 #设置字号
plt.rcParams["axes.unicode_minus"]=False #正常显示负号
data=pd.read_csv("future_extracting_result.csv")
target=data["class"]
traindata=pd.read_csv("future_selection_result.csv")
X_train,X_test, y_train, y_test=train_test_split(traindata,target,test_size=0.5,random_state=2)
model1=LogisticRegression(random_state=2)
C=np.linspace(0.0001,1,20)
A=[]
B=[]
for i in C:
    model=LogisticRegression(C=i,random_state=2)
    model.fit(X_train, y_train)
    y_pred=model.predict(X_test)
    A.append(i)
    B.append(accuracy_score(y_test,y_pred))
print(A)
print(B)
plt.ylabel("分类准确率")#横坐标名字
plt.xlabel("C")#纵坐标名字
plt.plot(A, B)
plt.show()

proposed_method.py

import numpy
import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.feature_selection import VarianceThreshold
from sklearn.svm import SVC
import numpy as np
import pandas as pd
from lightgbm import LGBMClassifier
from sklearn import svm
from sklearn.feature_selection import SelectFromModel
from sklearn.ensemble import RandomForestClassifier as RFC
from xgboost import XGBClassifier
from sklearn.neighbors import  KNeighborsClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.feature_selection import RFE
from sklearn.ensemble import AdaBoostClassifier
from sklearn.ensemble import VotingClassifier
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif']='SimHei' #设置字体
plt.rcParams["font.size"]=10 #设置字号
plt.rcParams["axes.unicode_minus"]=False #正常显示负号
data=pd.read_csv("future_extracting_result.csv")
target=data["class"]
traindata=pd.read_csv("future_selection_result.csv")
print(traindata.shape)
X_train,X_test, y_train, y_test=train_test_split(traindata,target,test_size=0.5,random_state=2)
#--------------------------------（1）logistic回归------------------------------------------
logistic=LogisticRegression(random_state=2,penalty="l2",C=0.214)
model1=AdaBoostClassifier(estimator=logistic,n_estimators=100,random_state=1)
model1.fit(X_train,y_train)
y_pred1=model1.predict(X_test)
print("构建的logistic+Adaboost分类模型的准确率为：",accuracy_score(y_test,y_pred1))
#---------------------------------(2)LGBT-----------------------------------------------
lgbt=LGBMClassifier(n_estimators=67,booster="gbtree",random_state=2)
lgbt.fit(X_train,y_train)
y_pred2=lgbt.predict(X_test)
print("构建的LGBT分类模型的准确率为：",accuracy_score(y_test,y_pred2))
#-----------------------------------(3)XGboost---------------------------------------------
xgb=XGBClassifier(n_estimators=25,booster="gbtree",random_state=2)
xgb.fit(X_train,y_train)
y_pred3=xgb.predict(X_test)
print("构建的XGboost分类模型的准确率为：",accuracy_score(y_test,y_pred3))
#-----------------------------------(4)KNN---------------------------------------------
KNN=KNeighborsClassifier(n_neighbors=3)
KNN.fit(X_train,y_train)
y_pred4=KNN.predict(X_test)
print("构建的KNN分类模型的准确率为：",accuracy_score(y_test,y_pred4))
#-----------------------------------(5)RandomForest---------------------------------------------
RF=RFC(n_estimators=142,criterion="entropy",random_state=1)
RF.fit(X_train,y_train)
y_pred5=RF.predict(X_test)
print("构建的RandomFores分类模型的准确率为：",accuracy_score(y_test,y_pred5))
#-----------------------------------（6）SVM多分类器---------------------------------------------------
svm=SVC(probability=True)
svm.fit(X_train,y_train)
y_pred6=svm.predict(X_test)
print("构建的SVM分类模型的准确率为：",accuracy_score(y_test,y_pred6))
#------------------------------Bagging集成模型一-----------------------------------------------------
voting=VotingClassifier(estimators=[('log',model1),("lgbt",lgbt),('KNN',KNN),('RF',RF),('svc',svm),('xgb',xgb)],voting='soft',weights=[1,1,1,1,1,1])
voting.fit(X_train,y_train)
y_pred=voting.predict(X_test)
print("构建的Bagging分类模型的准确率为：",accuracy_score(y_test,y_pred))
confusion = confusion_matrix(y_test, y_pred)
plt.imshow(confusion, cmap=plt.cm.Blues)
indices = range(len(confusion))

plt.colorbar()

plt.xlabel("真实标签")

plt.ylabel("预测的标签")

plt.title("混淆矩阵")
plt.xticks(indices, ['打保龄球','驾驶',"搏击",'射击','高尔夫','混合动作',"网球"])
plt.yticks(indices, ['打保龄球','驾驶',"搏击",'射击','高尔夫','混合动作',"网球"])
for first_index in range(len(confusion)):
    for second_index in range(len(confusion)):
        plt.text(first_index, second_index, confusion[first_index][second_index])

plt.show()

randomforest.py

import numpy
import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.feature_selection import VarianceThreshold
from sklearn.svm import SVC
import numpy as np
import pandas as pd
from lightgbm import LGBMClassifier
from sklearn import svm
from sklearn.feature_selection import SelectFromModel
from sklearn.ensemble import RandomForestClassifier as RFC
from xgboost import XGBClassifier
from sklearn.neighbors import  KNeighborsClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.feature_selection import RFE
from matplotlib import pyplot as plt
from matplotlib import font_manager
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif']='SimHei' #设置字体
plt.rcParams["font.size"]=10 #设置字号
plt.rcParams["axes.unicode_minus"]=False #正常显示负号
data=pd.read_csv("future_extracting_result.csv")
target=data["class"]
traindata=pd.read_csv("future_selection_result.csv")
print(traindata)
X_train,X_test, y_train, y_test=train_test_split(traindata,target,test_size=0.5,random_state=2)
estimators=np.linspace(1,200,50).astype(int)
A=[]
B=[]
A1=[]
A2=[]
A3=[]
for i in estimators:
    model=RFC(n_estimators=i,criterion="gini",random_state=2)
    model.fit(X_train, y_train)
    y_pred=model.predict(X_test)
    A.append(i)
    B.append(accuracy_score(y_test,y_pred))
for i in estimators:
    model=RFC(n_estimators=i,criterion="entropy",random_state=2)
    model.fit(X_train, y_train)
    y_pred=model.predict(X_test)
    A2.append(accuracy_score(y_test,y_pred))
for i in estimators:
    model=RFC(n_estimators=i,criterion="log_loss",random_state=2)
    model.fit(X_train, y_train)
    y_pred=model.predict(X_test)
    A3.append(accuracy_score(y_test,y_pred))
print(estimators)
print(A2)
print(B)
plt.ylabel("分类准确率")#横坐标名字
plt.xlabel("基学习器个数")#纵坐标名字
plt.plot(A, B, c='green',  label="基尼",markersize = 2,alpha=0.7)
plt.plot(A,A2,c='blue', linestyle='--' ,label="熵",markersize = 2,alpha=0.5)
plt.grid(True, linestyle='--', alpha=0.5)
plt.legend(loc='lower right')
plt.show()