多种机械手：MANO，shadow，LEAP

最新推荐文章于 2025-02-15 16:27:45 发布

泥土味

最新推荐文章于 2025-02-15 16:27:45 发布

阅读量2.9k

点赞数 30

分类专栏：学习笔记文章标签：机器人 python 深度学习

本文链接：https://blog.csdn.net/2302_79813879/article/details/142730970

版权

学习笔记专栏收录该内容

4 篇文章

订阅专栏

mano hand

MANO（Metric-Affine Hand Model）是一个用于手部姿态估计的三维参数化模型，它能够模拟手部的形状和运动。MANO 模型是由德国马克斯·普朗克研究所（Max Planck Institute）开发的，旨在提供一个逼真的、低维度的、并且能够捕捉到手部非刚性变化的模型。MANO 模型能够适配任何人类的手形，并且与标准的图形软件包兼容。

请添加图片描述

meno hand有61个参数
- 3个相机参数：与x，y，z的旋转角
- 45个关节姿态参数，除去0号和手指尖的关节，其他15个关节各有3个参数
- 10个形状参数，用来确定手的形状，比如手指长短，手掌的宽度

Shadow Hand

共有31个参数
- 3个相机参数：与x，y，z的旋转角
- 6个旋转参数：用于控制手部的旋转，通过Gram-Schmidt 正交化可以唯一确定一个3x3的旋转矩阵
- 22个关节参数
  - 5个手指的关节参数
    - 每个手指的掌指既可以旋转也可以弯曲，有两个参数
    - 每个手指的剩下两个手指只能弯曲，有一个参数
    - 每个手指的最远端没有自由度，所以没有参数
  - 2个手腕的关节参数
    - 小指旁边的关节可以旋转，有一个参数
    - 手腕的关节可以旋转，有一个参数

translation_values = [0.1, 0.2, 0.3]  # 手部的位置
rot = [0.1, 0.2,  0.1, 0.1, 0.1, 0.1,]  # 手腕的旋转
joint_angles = [0, 1, 1, 1, # fore   旋转2+弯曲2
                0, 0, 0, 0, # middle 旋转2+弯曲2
                0, 1, 1, 1, # ring   旋转2+弯曲2
                0,          # 小指处手腕
                0, 1, 1, 1, # little 旋转2+弯曲4
                0,          # 拇指处手腕
                1, 1, 0, -1 # thumb  弯曲2+旋转2
                ] 
# 构造 hand_pose_ 数组
hand_pose_ = np.array(translation_values + rot + joint_angles)

hand_pose = torch.tensor(hand_pose_, dtype=torch.float, device="cpu").unsqueeze(0)

print(hand_pose.shape) # (1, 31)

hand_model.set_parameters(hand_pose)

hand_mesh = hand_model.get_trimesh_data(0)

hand_mesh.show()

请添加图片描述

Allegro Hand

Allegro Hand 是一种性价比高且自适应性强的灵巧机器人手。它由 Wonik Robotics 开发，具有以下主要参数 1 2：

手指数量：4根手指
关节数量：16个独立的电流控制关节（每根手指4个关节）
- 参数：16个，分别控制每个关节
控制方式：电流控制
应用领域：抓握和操作研究
重量：轻便且便携
最大承重：可承受高达5公斤的重量
设计特点：仿人设计，适用于多种物体几何形状的抓握算法

Leap Hand

请添加图片描述

参数包括
- 4 个 MCP 关节的侧向角度
- 4 个 MCP 关节的前后角度
- 4 个 PIP 关节的角度
- 4 个 DIP 关节的角度
具体来说
- 【MCP1侧向角度, MCP1前后角度, PIP1角度, DIP1角度,
- MCP2侧向角度, MCP2前后角度, PIP2角度, DIP2角度,
- MCP3侧向角度, MCP3前后角度, PIP3角度, DIP3角度,
- 拇指CMC侧向角度, 拇指CMC前后角度, 拇指MCP角度, 拇指IP角度】

具体来自这份代码

import numpy as np
from typing import Optional, Union
import torch

from manotorch.anchorlayer import AnchorLayer
from manotorch.axislayer import AxisLayerFK
from manotorch.manolayer import ManoLayer, MANOOutput

'''
This converts MANO to LEAP Hand by simple direct joint-to-joint angle mapping.

This is a simple retargeting scheme, I HIGHLY recommend you tune/visualize the outputs to make it useful for your particular purpose and tune the offsets/scaling a bit.

This is based on this repository, see here for more details:
https://github.com/lixiny/manotorch
Follow the instructions from that repo to install manotorch.
This is different to using mano params directly, which is the rotation of fingers w/ respect to the previous joint, including more than just joint angles but also how the joints are aligned with respect to the wrist.

In our case, we care about ee.  ee is the Euler angles of the rotation of the joints in the human hand, anatomically aligned.
The order of ee is Twist(abnormal for most humans), Spread (side to side/abduction adduction), Bend (normal for all joints).   And the joint order is MCP, PIP, DIP from Index, Middle, Ring, Pinky and then Thumb.
T_g_a is the joint positions of the anatomically correct hand....we don't use the positions but the joint angles directly.
'''

class ManoRetargeter:
    def __init__(
        self,
        device: Optional[Union[str, torch.device]] = None):
        # 这是一个手部的模型，用于将手部的关节角度转换为手部的位置
        self.mano_layer = ManoLayer(
            rot_mode="axisang",
            use_pca=False,
            side="right",
            center_idx=None,
            mano_assets_root="/home/kshaw/manotorch/assets/mano",  #change this to your user
            flat_hand_mean=False,
        )
        # 这是一个手部的模型，用于将手部的位置转换为手部的关节角度
        self.axis_layer = AxisLayerFK(mano_assets_root="/home/kshaw/manotorch/assets/mano") #change this to your user         
    
    def mano_retarget(self, joint_pose, shape_params):
        '''
        input:  joint_pose_of_MANO (B, 48), 
                shape_params_of_MANO (B, 10)
                
        output: leap_hand_output (B, 16)
        '''
        # 通过mano模型，输入mano的参数获取手部的位置(MANOOutput)
        mano_results: MANOOutput = self.mano_layer(joint_pose, shape_params)
        
        # 通过手部的位置获取手部的关节角度
        T_g_p = mano_results.transforms_abs  # (B, 16, 4, 4)
        
        # axis_layer 计算解剖对齐的欧拉角 ee
        T_g_a, R, ee = self.axis_layer(T_g_p)  # ee (B, 16, 3)  this is anatomy aligned euler angle
        
        # _get_poses 方法将欧拉角转换为 LEAP 手部模型的关节角度
        # ee (48) -> leap_hand_output (16)
        ee = ee.flatten().tolist()
        output = self._get_poses(ee)
        return output
        #  transform order of right hand
        #         15-14-13-\
        #                   \
        #    3-- 2 -- 1 -----0   
        #   6 -- 5 -- 4 ----/
        #   12 - 11 - 10 --/
        #    9-- 8 -- 7 --/
        '''
        This takes the euler angles from the axis_layer outputs 
        and maps them directly to the 16 joint angles in the LEAP Hand 
        by extracting the right component of the euler.
        '''
    def _get_poses(self, finger_joints):
        # finger_joints is a list of 48 elements
        finger_joints = np.reshape(finger_joints, (16, 3)) 
 
        # MCP (Metacarpophalangeal Joint)
        # 掌指关节，连接掌骨和指骨的关节。是手指的第一个关节，靠近手掌
        finger_mcp_id = [1, 4, 10]#,7]  ##this is not a bug, 7,8,9 is pinky(小拇指) which we ignore
        # PIP (Proximal Interphalangeal Joint)
        # 近节间关节，是指的第二个关节，靠近指尖
        finger_pip_id = [2, 5, 11]#, 8]
        # DIP (Distal Interphalangeal Joint)
        # 远节间关节，是指的第三个关节，靠近指尖
        finger_dip_id = [3, 6, 12]#, 9]
        '''
        末端执行器End-Effector简称 EE
        是指机器人系统中与外部环境直接交互的部分。
        它通常位于机器人手臂或机械臂的末端，用于执行特定任务，如抓取、焊接、喷涂等。
        末端执行器的设计和功能可以根据具体应用进行定制。
        '''
        # 提取手指的关节的欧拉角
        ee_mcps = finger_joints[finger_mcp_id]  # shape (3, 3)
        ee_pips = finger_joints[finger_pip_id]  # shape (3, 3)
        ee_dips = finger_joints[finger_dip_id]  # shape (3, 3)

        # 侧向角度
        joint_mcp_side = -ee_mcps[:,1] #shape (3)
        # 前后角度
        joint_mcp_forward = ee_mcps[:,2] #shape (3)
        
        joint_pip = ee_pips[:,2] #shape (3)
        joint_dip = ee_dips[:,2] #shape (3)

        # 大拇指的单独计算
        thumb_cmc_side = finger_joints[13:14,1] #shape (1)
        thumb_cmc_forward = finger_joints[13:14,2] #shape (1)
        thumb_mcp = finger_joints[14:15,2]  #shape (1)
        thumb_ip = finger_joints[15:,2]  #shape (1)
        
        output = []
        
        # 手指和拇指的关节角度组织成一个特定格式的输出列表
        # 将 MCP、PIP、DIP 和拇指关节的角度按顺序添加到输出列表中
        for i in range(0,3):
            output +=  [joint_mcp_side[i],joint_mcp_forward[i], joint_pip[i], joint_dip[i]]
        
        output += [thumb_cmc_side[0], thumb_cmc_forward[0], thumb_mcp[0], thumb_ip[0]]
        
        # 返回手部的关节角度，是一个长度为 16 的列表
        # 参数包括
        #     4 个 MCP 关节的侧向角度
        #     4 个 MCP 关节的前后角度
        #     4 个 PIP 关节的角度
        #     4 个 DIP 关节的角度
        
        # [MCP1侧向角度, MCP1前后角度, PIP1角度, DIP1角度,
        #  MCP2侧向角度, MCP2前后角度, PIP2角度, DIP2角度,
        #  MCP3侧向角度, MCP3前后角度, PIP3角度, DIP3角度,
        # 拇指CMC侧向角度, 拇指CMC前后角度, 拇指MCP角度, 拇指IP角度]
        
        return output

if __name__ == "__main__":
    mano_retargeter = ManoRetargeter()
    
    mano_grasp = np.zeros(58)  ##Replace this with the 58 dimensional MANO you want
    
    joint_pose = mano_grasp[0:48] ##First 48 is the joint params
    
    shape_params = mano_grasp[48:58] ##last 10 is the shape params
    
    #You can also do this in a batch, but I don't in this example.
    
    leap_hand_output = mano_retargeter.mano_retarget(joint_pose, shape_params)
    
    print(leap_hand_output)