Leap Motion 探究 【第一篇】

 

       由于团队开发需要,今天拿到了Leap Motion做测试开发,也就是历动,一款手部识别传感器。
       拿到历动之前已经对它有所了解,然而拿到手后发现确实不如想象中的那么没好,由于基础图像识别,肯定有一些弊端,例如手部遮盖部分识别出错,应用体验一般,应用也比较少等,给我的感觉好像这是一款还处于最后优化的产品,并不能代替现有的交互操作体验。不过,在一些简单的交互上,Leap还是给我了一个很好的反馈,比如手掌的左右倾斜,手指简单的点击操作等。
       结合VR交互,这款产品应该是一个颠覆性体验,抛弃了传统的遥控器式手柄,完全手部操纵,所见即所得的感觉,冲击还是很大。所以打算开一个专题,记录下开发Leap的点滴。

 
       万事当先,Leap提供了很友好的SDK!这对开发者极其重要,不知道牛长什么样怎么去喂牛?Leap支持的平台有:C, C#, Unity, Object-C, Java, Python, JavaScript, Unreal Engine。 非常庞大的支持了,然而百度搜索到的开发文档还是寥寥无几,谷歌到的东西也不多,不知道是体验不够优秀,还是什么原因,这么好的一款产品应该大家一起来优化才对。
       由于最近.net平台开发比较多,所以还是以CS做例子,看一下官方的源码,然后写一个自己的例子看看怎么处理消息。
 
class Sample
{
	public static void Main ()
	{
		// Create a sample listener and controller
		SampleListener listener = new SampleListener ();
		Controller controller = new Controller ();

		// Have the sample listener receive events from the controller
		controller.AddListener (listener);

		// Keep this process running until Enter is pressed
		Console.WriteLine ("Press Enter to quit...");
		Console.ReadLine ();

		// Remove the sample listener when done
		controller.RemoveListener (listener);
		controller.Dispose ();
	}
}
 
       主程序段很简单, 首先实例化两个类,SampleListener(这个类后面详细讲,是Leap的核心部分)和Controller,然后给Controller增加一个监听,监听SampleListener反馈的消息。关闭时注意需要停止监听,并且丢弃Controller。
       下面开始讲SampleListener,即Leap中的Listener,不过由于Listener内消息都要自己编写,所以必须创建一个新的Listener去override Leap中的Listener。
 
class SampleListener : Listener
{
	private Object thisLock = new Object ();

	private void SafeWriteLine (String line)
	{
		lock (thisLock) {
			Console.WriteLine (line);
		}
	}

	public override void OnInit (Controller controller)
	{
		SafeWriteLine ("Initialized");
	}

	public override void OnConnect (Controller controller)
	{
		SafeWriteLine ("Connected");
		controller.EnableGesture (Gesture.GestureType.TYPE_CIRCLE);
		controller.EnableGesture (Gesture.GestureType.TYPE_KEY_TAP);
		controller.EnableGesture (Gesture.GestureType.TYPE_SCREEN_TAP);
		controller.EnableGesture (Gesture.GestureType.TYPE_SWIPE);
	}

	public override void OnDisconnect (Controller controller)
	{
        //Note: not dispatched when running in a debugger.
		SafeWriteLine ("Disconnected");
	}

	public override void OnExit (Controller controller)
	{
		SafeWriteLine ("Exited");
	}

	public override void OnFrame (Controller controller)
	{
		// Get the most recent frame and report some basic information
		Frame frame = controller.Frame ();

		SafeWriteLine ("Frame id: " + frame.Id
                    + ", timestamp: " + frame.Timestamp
                    + ", hands: " + frame.Hands.Count
                    + ", fingers: " + frame.Fingers.Count
                    + ", tools: " + frame.Tools.Count
                    + ", gestures: " + frame.Gestures ().Count);

		foreach (Hand hand in frame.Hands) {
			SafeWriteLine ("  Hand id: " + hand.Id
						+ ", palm position: " + hand.PalmPosition);
			// Get the hand's normal vector and direction
			Vector normal = hand.PalmNormal;
			Vector direction = hand.Direction;

			// Calculate the hand's pitch, roll, and yaw angles
			SafeWriteLine ("  Hand pitch: " + direction.Pitch * 180.0f / (float)Math.PI + " degrees, "
                        + "roll: " + normal.Roll * 180.0f / (float)Math.PI + " degrees, "
                        + "yaw: " + direction.Yaw * 180.0f / (float)Math.PI + " degrees");

			// Get the Arm bone
            Arm arm = hand.Arm;
			SafeWriteLine ("  Arm direction: " + arm.Direction
                        + ", wrist position: " + arm.WristPosition
                        + ", elbow position: " + arm.ElbowPosition);

			// Get fingers
			foreach (Finger finger in hand.Fingers) {
				SafeWriteLine ("    Finger id: " + finger.Id
					        + ", " + finger.Type.ToString()
					        + ", length: " + finger.Length
					        + "mm, width: " + finger.Width + "mm");

				// Get finger bones
				Bone bone;
				foreach (Bone.BoneType boneType in (Bone.BoneType[]) Enum.GetValues(typeof(Bone.BoneType)))
				{
					bone = finger.Bone(boneType);
					SafeWriteLine("      Bone: " + boneType
						        + ", start: " + bone.PrevJoint
						        + ", end: " + bone.NextJoint
						        + ", direction: " + bone.Direction);
				}
			}

		}

		// Get tools
		foreach (Tool tool in frame.Tools) {
			SafeWriteLine ("  Tool id: " + tool.Id
				        + ", position: " + tool.TipPosition
				        + ", direction " + tool.Direction);
		}

		// Get gestures
		GestureList gestures = frame.Gestures ();
		for (int i = 0; i < gestures.Count; i++) {
			Gesture gesture = gestures [i];

			switch (gesture.Type) {
			case Gesture.GestureType.TYPE_CIRCLE:
				CircleGesture circle = new CircleGesture (gesture);

                // Calculate clock direction using the angle between circle normal and pointable
				String clockwiseness;
				if (circle.Pointable.Direction.AngleTo (circle.Normal) <= Math.PI / 2) {
					//Clockwise if angle is less than 90 degrees
					clockwiseness = "clockwise";
				} else {
					clockwiseness = "counterclockwise";
				}

				float sweptAngle = 0;

                // Calculate angle swept since last frame
				if (circle.State != Gesture.GestureState.STATE_START) {
					CircleGesture previousUpdate = new CircleGesture (controller.Frame (1).Gesture (circle.Id));
					sweptAngle = (circle.Progress - previousUpdate.Progress) * 360;
				}

				SafeWriteLine ("  Circle id: " + circle.Id
                               + ", " + circle.State
                               + ", progress: " + circle.Progress
                               + ", radius: " + circle.Radius
                               + ", angle: " + sweptAngle
                               + ", " + clockwiseness);
				break;
			case Gesture.GestureType.TYPE_SWIPE:
				SwipeGesture swipe = new SwipeGesture (gesture);
				SafeWriteLine ("  Swipe id: " + swipe.Id
                               + ", " + swipe.State
                               + ", position: " + swipe.Position
                               + ", direction: " + swipe.Direction
                               + ", speed: " + swipe.Speed);
				break;
			case Gesture.GestureType.TYPE_KEY_TAP:
				KeyTapGesture keytap = new KeyTapGesture (gesture);
				SafeWriteLine ("  Tap id: " + keytap.Id
                               + ", " + keytap.State
                               + ", position: " + keytap.Position
                               + ", direction: " + keytap.Direction);
				break;
			case Gesture.GestureType.TYPE_SCREEN_TAP:
				ScreenTapGesture screentap = new ScreenTapGesture (gesture);
				SafeWriteLine ("  Tap id: " + screentap.Id
                               + ", " + screentap.State
                               + ", position: " + screentap.Position
                               + ", direction: " + screentap.Direction);
				break;
			default:
				SafeWriteLine ("  Unknown gesture type.");
				break;
			}
		}

		if (!frame.Hands.IsEmpty || !frame.Gestures ().IsEmpty) {
			SafeWriteLine ("");
		}
	}
}
       SampleListener中前几个事件不做说明,要点在于onFrame这个东西。这个东西和OpenCV中Frame概念一致,即“在每一帧上的数据”,关于“Frame”这个东西,我们能获取到以下数据:
Console.WriteLine("Frame id: " + frame.Id
                    + ", timestamp: " + frame.Timestamp
                    + ", hands: " + frame.Hands.Count
                    + ", fingers: " + frame.Fingers.Count
                    + ", tools: " + frame.Tools.Count
                    + ", gestures: " + frame.Gestures ().Count);
1. Frame ID
2.手个数
3.手指个数
4.工具个数(Leap可以识别出棍子这样的东西)
5.手势个数
 
       这些都是些统计数据,没有什么特别的意义,官方example在Frame中提供了上面五个玩意的详细使用方法。第一个就是手的细节参数。
       细读程序,可以发现,手包括了:手心,手方向,手臂,手指,骨头(关节翻译的比较合适一些),图示是官方的一个配置程序,可以看到,Hand包含的所有要素。
 
 
           手的角度,采用了PitchRowYaw坐标系定义,这样减轻了很多计算负担,只以手自身姿态为参照,确实给开发省去不少事情。手臂提供了关节和手腕的方向。手指提供了手指编号(即手指类型),长度(精度竟然是毫米!可以做为解锁用了)。骨头提供了类型,开始结束(即先后关节),方向。很明显,Leap程序是以人关节位置作为参考点处理的,非常聪明,一般做OpenCV时候,我们只是处理外手形状,找明显分割点来处理的,精度还行,不过极易被外部环境干扰,比如胖子的脖子2333333。以Leap这样处理,精细,稳定,对开发者来说,提供了友好的方式处理手势,为开发提供了方便。
 
      工具,简单,只有ID,position,direction,没有其他东西,够用就好,乔布斯说过,手指是最好的工具,要触摸笔干嘛。
 
       手势,内置了几种手势操作,在激发时候系统能够自动识别的,分别是 画圈,横扫,点击,向屏幕点击。 在操作时,记得按照官方给的属性操作就好。
 
  • Circle — A single finger tracing a circle.
  • Swipe — A long, linear movement of a finger.
  • Key Tap — A tapping movement by a finger as if tapping a keyboard key.
  • Screen Tap — A tapping movement by the finger as if tapping a vertical computer screen.
 
       值得一提的是,Leap提供了一个单独的 Touch Emulation,可以模仿平时大家熟悉的手机触摸操作。这点单独拿出来,看来Leap也是动了心思的,识别阈值甚至可以达到毫米级别,当然要这么用,你开心就好。
       分析了官方提供的Example后,自己写一个简单的参数获取就异常简单了,今天早上拿到的Leap,下午上课+调试飞机,晚上就将测试程序写了出来,有个几个小点要注意,一是记得引用LeapCSharp.NET3.5.dll或者LeapCSharp.NET4.0.dll,然后添加Leap.dll  Leap.lib  LeapCSharp.dll到项目中,二是不知为何,需要在application startup文件夹下,把LeapCSharp.NETx.0.dll添加进去,否则实例化时会出错。
       简单的Leap就这样创建好了,这几天试试做一个简单的东西来利用Leap的这些数据。
自己实例化的样例:http://download.csdn.net/detail/prius0304/9506700
相关推荐
©️2020 CSDN 皮肤主题: 编程工作室 设计师:CSDN官方博客 返回首页