Hololens入门之语音识别(语音命令)

Hololens入门之语音识别(语音命令)

对于HoloLens，语音输入是三大基本输入方式之一，广泛地运用在各种交互中。HoloLens上语音输入有三种形式，分别是：

1）语音命令 Voice Command

2）听写 Diction

3）语法识别 Grammar Recognition

本文介绍语音命令的使用，开发者可以通过为应用设定关键词，和对应的行为，来为用户提供语音命令体验。当用户说出关键词时，预设的动作就会被调用。

本文示例在 Hololens入门之手势识别(使用Manipulation gesture控制物体平移) 的基础上进行修改

1、修改CubeScript.cs,添加改变物体颜色的处理方法

using UnityEngine;
using System.Collections;
using HoloToolkit.Unity;

public class CubeScript : MonoBehaviour {

    private Vector3 manipulationPreviousPosition;

    // Use this for initialization
    void Start () {
	
	}
	
	// Update is called once per frame
	void Update () {
        
    }

    void PerformManipulationStart(Vector3 position)
    {
        //设置初始位置
        manipulationPreviousPosition = position;
    }

    void PerformManipulationUpdate(Vector3 position)
    {
        if (GestureManager.Instance.IsManipulating)
        {
            //计算相对位移，然后更新物体的位置   
            Vector3 moveVector = Vector3.zero;
            moveVector = position - manipulationPreviousPosition;
            manipulationPreviousPosition = position;
            transform.position += moveVector;
        }
    }

    private void OnTap()
    {
        gameObject.GetComponent<MeshRenderer>().material.color = Color.blue;
    }

    private void OnDoubleTap()
    {
        gameObject.GetComponent<MeshRenderer>().material.color = Color.green;
    }

    //新增改变物体颜色的方法，当收到改变颜色的指令，且凝视射线投射到该物体上时，修改当前物体颜色
    public void ChangeColor(string color)
    {
        if (GazeManager.Instance.FocusedObject == gameObject)
        {
            switch (color)
            {
                case "red":
                    gameObject.GetComponent<MeshRenderer>().material.color = Color.red;
                    break;
                case "yellow":
                    gameObject.GetComponent<MeshRenderer>().material.color = Color.yellow;
                    break;
                default:
                    break;
            }
            
        }
    }
}

2、在Manager中添加KeywordManager.cs脚本组件，并且添加语音命令关键字，添加处理函数和传入的参数

KeywordManager.cs 脚本直接使用HoloToolkit中的脚本，脚本内容如下：

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.

using System.Collections.Generic;
using System.Linq;
using UnityEngine;
using UnityEngine.Events;
using UnityEngine.Windows.Speech;

namespace HoloToolkit.Unity
{
    /// <summary>
    /// KeywordManager allows you to specify keywords and methods in the Unity
    /// Inspector, instead of registering them explicitly in code.
    /// This also includes a setting to either automatically start the
    /// keyword recognizer or allow your code to start it.
    ///
    /// IMPORTANT: Please make sure to add the microphone capability in your app, in Unity under
    /// Edit -> Project Settings -> Player -> Settings for Windows Store -> Publishing Settings -> Capabilities
    /// or in your Visual Studio Package.appxmanifest capabilities.
    /// </summary>
    public partial class KeywordManager : MonoBehaviour
    {
        [System.Serializable]
        public struct KeywordAndResponse
        {
            [Tooltip("The keyword to recognize.")]
            public string Keyword;
            [Tooltip("The KeyCode to recognize.")]
            public KeyCode KeyCode;
            [Tooltip("The UnityEvent to be invoked when the keyword is recognized.")]
            public UnityEvent Response;
        }

        // This enumeration gives the manager two different ways to handle the recognizer. Both will
        // set up the recognizer and add all keywords. The first causes the recognizer to start
        // immediately. The second allows the recognizer to be manually started at a later time.
        public enum RecognizerStartBehavior { AutoStart, ManualStart };

        [Tooltip("An enumeration to set whether the recognizer should start on or off.")]
        public RecognizerStartBehavior RecognizerStart;

        [Tooltip("An array of string keywords and UnityEvents, to be set in the Inspector.")]
        public KeywordAndResponse[] KeywordsAndResponses;

        private KeywordRecognizer keywordRecognizer;
        private Dictionary<string, UnityEvent> responses;

        void Start()
        {
            if (KeywordsAndResponses.Length > 0)
            {
                // Convert the struct array into a dictionary, with the keywords and the keys and the methods as the values.
                // This helps easily link the keyword recognized to the UnityEvent to be invoked.
                responses = KeywordsAndResponses.ToDictionary(keywordAndResponse => keywordAndResponse.Keyword,
                                                              keywordAndResponse => keywordAndResponse.Response);
<span style="white-space:pre">		</span>//通过注册关键词，来初始化KeywordRecognizer
                keywordRecognizer = new KeywordRecognizer(responses.Keys.ToArray());
                keywordRecognizer.OnPhraseRecognized += KeywordRecognizer_OnPhraseRecognized;

                if (RecognizerStart == RecognizerStartBehavior.AutoStart)
                {
                    keywordRecognizer.Start();
                }
            }
            else
            {
                Debug.LogError("Must have at least one keyword specified in the Inspector on " + gameObject.name + ".");
            }
        }

        void Update()
        {
            ProcessKeyBindings();
        }

        void OnDestroy()
        {
            if (keywordRecognizer != null)
            {
                StopKeywordRecognizer();
                keywordRecognizer.OnPhraseRecognized -= KeywordRecognizer_OnPhraseRecognized;
                keywordRecognizer.Dispose();
            }
        }

        private void ProcessKeyBindings()
        {
            foreach (var kvp in KeywordsAndResponses)
            {
                if (Input.GetKeyDown(kvp.KeyCode))
                {
                    kvp.Response.Invoke();
                    return;
                }
            }
        }

        private void KeywordRecognizer_OnPhraseRecognized(PhraseRecognizedEventArgs args)
        {
            UnityEvent keywordResponse;

            // Check to make sure the recognized keyword exists in the methods dictionary, then invoke the corresponding method.
            if (responses.TryGetValue(args.text, out keywordResponse))
            {
                keywordResponse.Invoke();
            }
        }

        /// <summary>
        /// Make sure the keyword recognizer is off, then start it.
        /// Otherwise, leave it alone because it's already in the desired state.
        /// </summary>
        public void StartKeywordRecognizer()
        {
            if (keywordRecognizer != null && !keywordRecognizer.IsRunning)
            {
                keywordRecognizer.Start();
            }
        }

        /// <summary>
        /// Make sure the keyword recognizer is on, then stop it.
        /// Otherwise, leave it alone because it's already in the desired state.
        /// </summary>
        public void StopKeywordRecognizer()
        {
            if (keywordRecognizer != null && keywordRecognizer.IsRunning)
            {
                keywordRecognizer.Stop();
            }
        }
    }
}

3、打开microphone权限，由于使用到语音识别，所以需要有打开microphone的权限

4、运行测试

对着microphone 说出指令 "change red", Cube将变成红色

对着microphone 说出指令 "change yellow", Cube将变成黄色