Hololens入门之手势识别(手检测反馈)

Hololens入门之手势识别(手检测反馈)

本文实现当使用者手出现在Hololens视野范围内时，跟踪手并给出反馈的效果。

1、在Manager上添加HandsManager脚本组件，用于追踪识别手

HandsManager.cs如下（直接使用HoloTooKit中脚本）

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.

using System.Collections.Generic;
using UnityEngine.VR.WSA.Input;

namespace HoloToolkit.Unity
{
    /// <summary>
    /// HandsManager determines if the hand is currently detected or not.
    /// </summary>
    public partial class HandsManager : Singleton<HandsManager>
    {
        /// <summary>
        /// HandDetected tracks the hand detected state.
        /// Returns true if the list of tracked hands is not empty.
        /// </summary>
        public bool HandDetected
        {
            get { return trackedHands.Count > 0; }
        }

        private HashSet<uint> trackedHands = new HashSet<uint>();

        void Awake()
        {
            //识别到来源
            InteractionManager.SourceDetected += InteractionManager_SourceDetected;
            //来源丢失
            InteractionManager.SourceLost += InteractionManager_SourceLost;
        }

        private void InteractionManager_SourceDetected(InteractionSourceState state)
        {
            // 检测来源是否为手，如果是手则加入跟踪集合
            if (state.source.kind != InteractionSourceKind.Hand)
            {
                return;
            }

            trackedHands.Add(state.source.id);
        }

        private void InteractionManager_SourceLost(InteractionSourceState state)
        {
            // 检测丢失的来源是否为手，如果是手则从跟踪集合中去除
            if (state.source.kind != InteractionSourceKind.Hand)
            {
                return;
            }

            if (trackedHands.Contains(state.source.id))
            {
                trackedHands.Remove(state.source.id);
            }
        }

        void OnDestroy()
        {
            InteractionManager.SourceDetected -= InteractionManager_SourceDetected;
            InteractionManager.SourceLost -= InteractionManager_SourceLost;
        }
    }
}

该脚本中使用到了底层API   Interaction Input

底层API运行获得输入来源的更多详细信息，例如它在世界中的位置和速度。

如何处理底层交互事件
使用底层交互是很容易的：
1） 注册InteractionManager事件
2） 处理事件
停止它也很容易：
1） 取消注册事件

处理底层交互事件
一旦注册了底层交互事件，在事件发生时你就可以得到回调。你可以使用获取到的时间信息来处理应用行为。

void InteractionManager_SourcePressed(InteractionSourceState state)
{
// state变量里包含以下信息:
// 当前凝视射线信息
// 来源是否被点击
// 位置、速度之类的属性
// 来源id和来源类型 （ hand, voice, controller或其他）
}

如何停止交互事件
当你不再想要关注一些事件后，只需要取消时间注册即可。

InteractionManager.SourcePressed -= InteractionManager_SourcePressed;

输入源变化事件
这些事件描述了输入源的当前状态：
1) detected（ 即将激活）
2) lost（ 即将取消激活）
3) updates（ 移动或者一些状态在变化）
4) is pressed（ 点击、按钮按下或者语音选中）
5) is released（ 点击结束，按钮松开，语音选中结束）


输入源状态
每个事件都会有一个InteractionSourceState参数，这个参数代表了实时输入源状态：
1) 是否是点击状态
2) InteractionSourceProperties包含了输入源位置信息 InteractionSourceLocation，能够获得当前输入源位置和速度信息
3) 凝视射线信息，用于判断事件发生时用户是否在注视目标
4) 来源类型信息，包括hand、voice、controller或者其他类型

2、在Cursor下新建Empty对象，并重命名为CursorBillboard，并添加Billboard脚本组件

Billboard脚本如下(可以直接在HoloToolKit中找到)

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.

using UnityEngine;

namespace HoloToolkit.Unity
{
    public enum PivotAxis
    {
        // Rotate about all axes.
        Free,
        // Rotate about an individual axis.
        X,
        Y
    }

    /// <summary>
    /// The Billboard class implements the behaviors needed to keep a GameObject
    /// oriented towards the user.
    /// </summary>
    public class Billboard : MonoBehaviour
    {
        /// <summary>
        /// The axis about which the object will rotate.
        /// </summary>
        [Tooltip("Specifies the axis about which the object will rotate (Free rotates about both X and Y).")]
        public PivotAxis PivotAxis = PivotAxis.Free;

        /// <summary>
        /// Overrides the cached value of the GameObject's default rotation.
        /// </summary>
        public Quaternion DefaultRotation { get; private set; }

        private void Awake()
        {
            // Cache the GameObject's default rotation.
            DefaultRotation = gameObject.transform.rotation;
        }

        /// <summary>
        /// Keeps the object facing the camera.
        /// </summary>
        private void Update()
        {
            // Get a Vector that points from the Camera to the target.
            Vector3 forward;
            Vector3 up;

            // Adjust for the pivot axis. We need a forward and an up for use with Quaternion.LookRotation
            switch (PivotAxis)
            {
                // If we're fixing one axis, then we're projecting the camera's forward vector onto
                // the plane defined by the fixed axis and using that as the new forward.
                case PivotAxis.X:
                    Vector3 right = transform.right; // Fixed right
                    forward = Vector3.ProjectOnPlane(Camera.main.transform.forward, right).normalized;
                    up = Vector3.Cross(forward, right); // Compute the up vector
                    break;

                case PivotAxis.Y:
                    up = transform.up; // Fixed up
                    forward = Vector3.ProjectOnPlane(Camera.main.transform.forward, up).normalized;
                    break;

                // If the axes are free then we're simply aligning the forward and up vectors
                // of the object with those of the camera. 
                case PivotAxis.Free:
                default:
                    forward = Camera.main.transform.forward;
                    up = Camera.main.transform.up;
                    break;
            }


            // Calculate and apply the rotation required to reorient the object
            transform.rotation = Quaternion.LookRotation(forward, up);
        }
    }
}

3、在Cursor上添加CursorFeedback脚本组件

1） 在HoloToolkit -> Input -> Prefabs中找到HandDetectedFeedback Prefab 并拖到CursorFeedback的hand detected asset上

2） 将刚才创建的CursorBillboard拖到CursorFeedback的Feedback Parent上

CursorFeedback脚本如下（(可以直接在HoloToolKit中找到)）

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.

using UnityEngine;

namespace HoloToolkit.Unity
{
    /// <summary>
    /// CursorFeedback class takes GameObjects to give cursor feedback
    /// to users based on different states.
    /// </summary>
    public class CursorFeedback : MonoBehaviour
    {
        [Tooltip("Drag a prefab object to display when a hand is detected.")]
        public GameObject HandDetectedAsset;
        private GameObject handDetectedGameObject;

        [Tooltip("Drag a prefab object to parent the feedback assets.")]
        public GameObject FeedbackParent;

        void Awake()
        {
            if (HandDetectedAsset != null)
            {
                handDetectedGameObject = InstantiatePrefab(HandDetectedAsset);
            }
            else
            {
                Debug.LogError("Missing a required game object asset.  Check HandDetectedAsset is not null in editor.");
            }
        }

        private GameObject InstantiatePrefab(GameObject inputPrefab)
        {
            GameObject instantiatedPrefab = null;

            if (inputPrefab != null && FeedbackParent != null)
            {
                instantiatedPrefab = GameObject.Instantiate(inputPrefab);
                // Assign parent to be the FeedbackParent
                // so that feedback assets move and rotate with this parent.
                instantiatedPrefab.transform.parent = FeedbackParent.transform;

                // Set starting state of the prefab's GameObject to be inactive.
                instantiatedPrefab.gameObject.SetActive(false);
            }
            else
            {
                Debug.LogError("Missing a required game object asset.  Check FeedbackParent is not null in editor.");
            }

            return instantiatedPrefab;
        }

        void Update()
        {
            UpdateHandDetectedState();
        }

        private void UpdateHandDetectedState()
        {
            if (handDetectedGameObject == null)
            {
                return;
            }

            handDetectedGameObject.SetActive(HandsManager.Instance.HandDetected);
        }
    }
}

4、运行测试

当手出现在Hololens视野中时，手被检测到，在凝视射线处出现一个蓝色的小手（Hololens模拟器中需要处于hold状态才会出现蓝色小手，真机上只要手举起就可以）