LARa、PKU-MMD数据集介绍以及可视化

LARa、PKU-MMD数据集介绍以及可视化

LARa数据集介绍以及可视化

介绍

原文

OpenDataLab下载

其他下载

LARa是第一个用于人类活动识别的免费物流数据集。在TU多特蒙德大学的 “物流创新实验室混合服务” 中，使用OMoCap，IMUs和RGB相机记录了两个具有14个主题的拣选和一个包装方案。758分钟的记录由12个注释者在474人小时内标记。随后的修订由4名审校在143人小时内进行。所有给定的数据都已标记并分类为8个活动类和19个二进制粗语义描述，也称为属性。22个骨骼点。

R collar
R wrist end
R elbow
L tibia
L collar
L wrist
R humerus
R tibia
lower back
L femur
R foot
L elbow
L humerus
L wrist end
root
R wrist
head end
L foot
R toe
head
R femur
L toe

数据解释

• RX、RY、RZ：这些通常分别表示围绕X轴、Y轴和Z轴的旋转。
• TX、TY、TZ：这些通常分别表示沿X轴、Y轴和Z轴的平移（移动）。

在三维图形或机器人领域，这些坐标通常用于描述物体在空间中的位置和方向。例如，RX 可能表示物体围绕X轴旋转的角度，而 TX 可能表示物体沿X轴移动的距离。

标注工具

下载

动作类别

0	C1_Standing
1	C2_Walking
2	C3_Cart
3	C4_Handling(upwards)
4	C5_Handling(centred)
5	C6_Handling(downwards)
6	C7_Synchronization
7	C8_None

PKU-MMD数据集介绍以及可视化

PKU-MMD 是一个新的大规模基准数据集，专注于连续多模态的3D人体动作理解，涵盖了广泛的复杂人类活动，并提供了完善的标注信息。PKU-MMD 包含两个阶段，针对不同难度的动作检测任务。第一阶段是大间隔动作检测任务，第二阶段是小间隔动作检测任务。我们的数据集还提供了多模态数据来源，包括RGB、深度、红外辐射和骨骼数据。25个骨骼点

下载

动作类别

Label	Action
1	bow
2	brushing hair
3	brushing teeth
4	check time (from watch)
5	cheer up
6	clapping
7	cross hands in front (say stop)
8	drink water
9	drop
10	eat meal/snack
11	falling
12	giving something to other person
13	hand waving
14	handshaking
15	hopping (one foot jumping)
16	hugging other person
17	jump up
18	kicking other person
19	kicking something
20	make a phone call/answer phone
21	pat on back of other person
22	pickup
23	playing with phone/tablet
24	point finger at the other person
25	pointing to something with finger
26	punching/slapping other person
27	pushing other person
28	put on a hat/cap
29	put something inside pocket
30	reading
31	rub two hands together
32	salute
33	sitting down
34	standing up
35	take off a hat/cap
36	take off glasses
37	take off jacket
38	take out something from pocket
39	taking a selfie
40	tear up paper
41	throw
42	touch back (backache)
43	touch chest (stomachache/heart pain)
44	touch head (headache)
45	touch neck (neckache)
46	typing on a keyboard
47	use a fan (with hand or paper)/feeling warm
48	wear jacket
49	wear on glasses
50	wipe face
51	writing

骨骼点分布

可视化

import matplotlib.pyplot as plt
import numpy as np

class PKUDrawer:

    def __init__(self, points):
        self.points = points # T, 25, 3

    def _plot3dSkeleton(self, point: np.ndarray, elev=-82, azim=-630):
        '''
        point: (V, C)
        elev, azim: 3d view angle
        '''
    
        arm = [21, 7, 6, 5, 4, 20, 8, 9, 10, 11, 23] # green
        body = [3, 2, 20, 1, 0] # yellow
        rightLeg = [0, 12, 13, 14, 15] # blue
        leftLeg = [0, 16, 17, 18, 19] # blue

        plt.ioff()
        fig = plt.figure()
        ax = fig.add_subplot(111, projection='3d')
        ax.scatter(point[:, 0], point[:, 1], point[:, 2], c='red', s=40.0)
        ax.plot(point[arm, 0], point[arm, 1], point[arm, 2], c='green', lw=2.0)
        ax.plot(point[body, 0], point[body, 1], point[body, 2], c='yellow', lw=2.0)
        ax.plot(point[rightLeg, 0], point[rightLeg, 1], point[rightLeg, 2], c='blue', lw=2.0)
        ax.plot(point[leftLeg, 0], point[leftLeg, 1], point[leftLeg, 2], c='blue', lw=2.0)
        ax.set_xlim(point[:, 0].min(), point[:, 0].max())
        ax.set_ylim(point[:, 1].min(), point[:, 1].max())
        ax.set_zlim(point[:, 2].min(), point[:, 2].max())
        ax.view_init(elev=elev, azim=azim)

        # 设置隐藏坐标轴数值但不隐藏坐标轴
        ax.set_xticklabels([])
        ax.set_yticklabels([])
        ax.set_zticklabels([])

        def on_move(event):
            if event.inaxes == ax and event.button == 1:  # Check if the left mouse button is pressed
                print(f"Elevation: {ax.elev}, Azimuth: {ax.azim}")

        fig.canvas.mpl_connect('motion_notify_event', on_move)

        plt.show()

    def plot3dSkeleton(self, i):
        self._plot3dSkeleton(self.points[i])

    def project3dSkeleton(self, points: np.ndarray, elev=-82, azim=-630):
        '''
        将 3D 骨骼点投影到 2D 平面。

        参数:
            points: (V: 骨骼点数, C: 维度数), 输入的 3D 骨骼点坐标。
            elev: 俯仰角 (elevation angle), 默认值为 -82。
            azim: 方位角 (azimuth angle), 默认值为 -630。

        返回:
            projected_points: (V, 2), 投影后的 2D 坐标。
        '''
        # 检查输入维度
        if points.shape[1] != 3:
            raise ValueError("输入的点坐标必须是 3D 坐标 (V, 3)。")

        # 创建 3D 图形
        fig = plt.figure()
        ax = fig.add_subplot(111, projection='3d')

        # 设置视角
        ax.view_init(elev=elev, azim=azim)

        # 绘制 3D 点
        ax.scatter(points[:, 0], points[:, 1], points[:, 2], c='b', marker='o')

        # 获取 2D 投影坐标
        projected_points = np.array(ax.get_proj()).dot(np.vstack([points.T, np.ones(points.shape[0])])).T
        projected_points = projected_points[:, :2]  # 取前两维 (x, y)

        # 关闭图形
        plt.close(fig)

        return projected_points

    def _plot2dSkeleton(self, points: np.ndarray):
        '''
        绘制 2D 骨骼点。

        参数:
            points: (V: 骨骼点数, 2), 输入的 2D 骨骼点坐标。
        '''
        # 创建图形
        fig, ax = plt.subplots()

        fig.set_size_inches(6, 6)

        # 绘制骨骼点
        visable_sk_index = list(range(21))
        for i in [7,11,15,19,]:
            visable_sk_index.remove(i)
        ax.scatter(points[visable_sk_index, 0], points[visable_sk_index, 1], c='black', marker='o')

        arm = [6, 5, 4, 20, 8, 9, 10]
        body = [3, 2, 20, 1, 0]
        rightLeg = [0, 12, 13, 14]
        leftLeg = [0, 16, 17, 18]

        for limb in [arm, body, rightLeg, leftLeg]:
            ax.plot(points[limb, 0], points[limb, 1], c='black', lw=2.0)
    
        ax.set_xlim(points[:, 0].min() * 2, points[:, 0].max() * 2)
        ax.set_ylim(points[:, 1].min() * 1.1, points[:, 1].max() * 1.1)

        # 隐藏坐标轴
        ax.axis('off')

        plt.show()

    def plot2dSkeleton(self, i):
        points = self.project3dSkeleton(self.points[i])
        self._plot2dSkeleton(points)


if __name__ == '__main__':
    sk = np.load('/data/PKU-view/features/A05N11-L.npy').reshape(-1,2,25,3)[:,0,:,:]
    PKUDrawer(sk).plot3dSkeleton(400)
    PKUDrawer(sk).plot2dSkeleton(400)