文章目录
Summary
Spatial Mapping是通过扫描并学习真实的周围环境然后与虚拟世界结合的结果;
- 扫描环境并将数据传输到的holoLens中;
- 使用着色器并应用到显示空间中;
- 使用网格处理将房间格划分为简单的平面;
- 比Holograms101更高级的全息图的放置技术,并提供有关全息图可放置在环境中的位置得反馈;
- 学习遮挡效果,当全息图在真实世界物体的背后时,仍然可以看到;
Chapter1 —— Scanning 扫描
Objectives 目标
- 了解SufaceObserver及其设置对体验的HoloLens的性能的影响;
- 创建房间扫描并且收集房间风格;
Instructions 说明
- 在项目视图中HoloToolkit-SpatialMapping-230 \ SpatialMapping \ Prefabs文件夹中,找到SpatialMapping预制体;
- 将SpatialMapping预制件拖放到层次结构中的根目录;
Build and Deploy
Part 1 设置
Part 2 性能影响
在Unity中,选择Window > Profiler。
点击Add Profiler > GPU。
点击Active Profiler > <输入IP>。
输入你的HoloLens 的IP地址。
点击连接。
观察GPU渲染帧所花费的毫秒数。
停止应用程序在设备上运行。
Chapter 2 —— Visualization 可视化
Objectives 目标
- 了解着色器的基础知识;
- 显示当前所处的周围环境;
Instructions 说明
- 在层次结构视图中选中SpatialMapping对象,在属性面板中找到SpatialMappingManager组件;
- 将Surface Material属性设置为BlueLinesOnWalls;
- 在项目视图中的Shaders文件夹双击BlueLinesOnWalls用vs打开;
- 将顶点的位置转换成为世界空间;
- 检查顶点的法线以确定像素是否垂直;
- 设置渲染像素的颜色;
Build and Deploy 构建和部署
BlueLinesOnWalls材质
* LineScale :设置线条精细;
* LinesPerMeter:每面墙上显示的线条数;
Chapter 3 —— Processing 处理
Objectives 目标
- 处理空间映射数据用于应用程序;
- 分析空间映射数据以找到平面并去除三角形;
- 使用飞机进行全息的放置;
Instructions 说明
-
在项目视图中,选择Holograms > SpatialProcessing拖入到层次面板中,SpatialProcessing预制体包含用于处理空间映射数据的组件;
- SurfaceMeshesToPlanes.cs 根据空间映射数据查找并生成平面;
- RemoveSurfaceVertices.cs 可以从空间映射网格中移除顶点,可以用来在网格中创建孔,或者删除不需要的多余三角形(因这平面可以被替代的)
-
在项目面板中,Holograms > SpaceCollection预制体拖放到层次结构视图中;
-
选中SpatialProcessing,找到PlaySpaceManager组件;
- 超过扫描时间限制(10秒)后,停止收集空间映射数据;
- 处理空间映射数据;
a. 使用SurfaceMeshesToPlans用平面去创建一个简单的世界;
b. 使用RemoveSurfaceVertices去除落在平面边界内的曲面三角形; - 会在靠近用户的墙壁和地板上会生成一些全息图;
using Academy.HoloToolkit.Unity;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class PlaySpaceManager_TEST : Singleton<PlaySpaceManager_TEST>
{
/// <summary>
/// 当选中的时候,SurfaceObserver将在指定的的时间停止运行;
/// </summary>
public bool limitScanningByTime = true;
/// <summary>
/// SurfaceObserver将运行多少时间,使用“按时间限制扫描”时使用的的时间(秒);
/// </summary>
public float scanTime = 30f;
/// <summary>
/// 在SpatialMapping绘制空间映射风格时使用的材质;
/// </summary>
public Material defaultMaterial;
/// <summary>
/// 在SpatialMapping停止绘制空间映射风格时使用的材质;
/// </summary>
public Material secondaryMaterial;
/// <summary>
/// 扫描/处理所需要的最少层数;
/// </summary>
public uint minimumFloors = 1;
/// <summary>
/// 扫描/处理的最少面板数;
/// </summary>
public uint minimumWalls = 1;
/// <summary>
/// 表面网格和处理是否完成;
/// </summary>
private bool meshesProcessed = false;
// Use this for initialization
void Start()
{
//设置扫描时候使用的材质;
SpatialMappingManager.Instance.SetSurfaceMaterial(defaultMaterial);
//用于处理平面事件
SurfaceMeshesToPlanes.Instance.MakePlanesComplete += Instance_MakePlanesComplete;
}
/// <summary>
///
/// </summary>
/// <param name="source"></param>
/// <param name="args"></param>
private void Instance_MakePlanesComplete(object source, System.EventArgs args)
{
//用来设置水平项目的地板或者台面的集合;
List<GameObject> horizontal = new List<GameObject>();
//用于收集垂直的面板;
List<GameObject> vertical = new List<GameObject>();
//GetActivePlanes() 获取桌面和地面的平面;
horizontal = SurfaceMeshesToPlanes.Instance.GetActivePlanes(PlaneTypes.Table | PlaneTypes.Floor);
//获取所以垂直平面
vertical = SurfaceMeshesToPlanes.Instance.GetActivePlanes(PlaneTypes.Wall);
if (horizontal.Count >= minimumFloors && vertical.Count >= minimumWalls)
{
//删除与我们要放置的的平面相交的空间映射网格的三角面片
RemoveVertices(SurfaceMeshesToPlanes.Instance.ActivePlanes);
//扫描结束,改变应用于空间映射的网格材质,替换材质;
SpatialMappingManager.Instance.SetSurfaceMaterial(secondaryMaterial);
//生成世界中可放置对象的集合,并将他们旋转到相对匹配的平面上;
SpaceCollectionManager.Instance.GenerateItemsInWorld(horizontal, vertical);
}
else
{
SpatialMappingManager.Instance.StartObserver();
meshesProcessed = false;
}
}
/// <summary>
/// 在空间映射表面创建平面;
/// </summary>
private void CreatePlanes()
{
//基于空间地图生成平面;
SurfaceMeshesToPlanes surfaceMeshes = SurfaceMeshesToPlanes.Instance;
if (surfaceMeshes != null && surfaceMeshes.enabled)
{
surfaceMeshes.MakePlanes();
}
}
/// <summary>
/// 从空间映射的表面移除三角形;
/// </summary>
/// <param name="boundingObjects"></param>
private void RemoveVertices(IEnumerable<GameObject> boundingObjects)
{
RemoveSurfaceVertices removeSurface = RemoveSurfaceVertices.Instance;
if (removeSurface != null && removeSurface.enabled)
{
removeSurface.RemoveSurfaceVerticesWithinBounds(boundingObjects);
}
}
private void OnDestroy()
{
if (SurfaceMeshesToPlanes.Instance != null)
{
SurfaceMeshesToPlanes.Instance.MakePlanesComplete -= Instance_MakePlanesComplete;
}
}
// Update is called once per frame
void Update()
{
//确认空间映射的数据是否处理完成,是否扫描时间超过了限制时间了;
if (meshesProcessed && limitScanningByTime)
{
// If we have not processed the spatial mapping data
// and scanning time is limited...
// Check to see if enough scanning time has passed
// since starting the observer.
//检查是否有时间
if (limitScanningByTime && ((Time.time - SpatialMappingManager.Instance.StartTime) < scanTime))
{
// If we have a limited scanning time, then we should wait until
// enough time has passed before processing the mesh.
}
else
{
//扫描环境,处理空间映射数据
//判断空间映射管理器是否还在运行
if (SpatialMappingManager.Instance.IsObserverRunning())
{
SpatialMappingManager.Instance.StopObserver();
}
//创建平面;
CreatePlanes();
//显示完成;
meshesProcessed = true;
}
}
}
}
Chapter 4 —— Placement 安置
Objectives 目标
- 确定全息图是否适合扫描得到的平面;
- 如果全息图不能放置在某个平面上向用户提出反馈;
Instructions 说明
- 选中SpatialProcessing对象,其他他有一个SurfaceMeshesToPlanes.cs的组件;
- 将DrawPlanes属性的Nothing值,更改为Wall,用来仅仅渲染墙这个平面;
- 项目视图中 > Scripts > Placeable.cs;
这个脚本可以在已查找并完成创建的平面中放置海报和投影箱;- 通过从边界立方体的中心和四角进行光线投射来确定全息图是否适合表面;
- 检查表面法线以确定表面是否足够光滑,来保证全息图平齐;
- 在放置的时候,渲染全息图周围的边框显示实际尺寸;
- 在全息图下面或者背面投射阴影,用来显示它将旋转在地板或者墙上的位置;
- 如果全息图不能放置在表面上,将阴影渲染成红色,如果可以渲染成绿色;
- 将全息图重新定向与符合的表面类型(垂直或者水平)对齐;
- 将全息图平滑的放置在选定的表面上,避免有跳跃或者折断的现象;
using Academy.HoloToolkit.Unity;
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public enum PlacementSurfaces_TEST
{
/// <summary>
/// 水平表面的一个指向上方的单位向量;
/// </summary>
Horizontal=1,
/// <summary>
/// 垂直平面的单位向量;
/// </summary>
Vertical=2
}
public class Placeable_TEST : MonoBehaviour
{
/// <summary>
/// 用于放置,物体边界的材质;
/// </summary>
public Material PlaceableBoundsMaterial = null;
/// <summary>
/// 在不允许放置的时候,物体边界的材质;
/// </summary>
public Material NotPlaceableBoundsMaterial = null;
/// <summary>
/// 当允许放置的时候的阴影;
/// </summary>
public Material PlaceableShadowMaterial = null;
public Material NotPlaceableShowMaterial = null;
/// <summary>
/// 可放置物体的表面的类型;
/// </summary>
public PlacementSurfaces_TEST PlacementSurface = PlacementSurfaces_TEST.Horizontal;
/// <summary>
/// 放置期间子对象隐藏;
/// </summary>
public List<GameObject> ChildrenToHide = new List<GameObject>();
/// <summary>
/// 指示对象是否正在处于放置的过程当中;
/// </summary>
public bool IsPlacing { get; private set; }
/// <summary>
/// 与表面的最近距离,当用户的视线和空间映射网格不相交时用来定位对象的;
/// </summary>
private float lastDistance = 2.0f;
/// <summary>
/// 远离目标表面的距离,物体在放置之前应悬停;
/// </summary>
private float hoverDistance = 0.15f;
/// <summary>
/// 阀值(接近于0)用于确定表面是否平坦;
/// </summary>
private float distaneThreshold = 0.02f;
/// <summary>
/// 阀值(接近于1)用于确定表面是否垂直;
/// </summary>
private float upNormalThreshold = 0.9f;
/// <summary>
/// 允许对象放置的最大距离;
/// </summary>
private float maximumPlacementDisstance = 5.0f;
/// <summary>
/// 物体在放置时沉降到表面(1最快)
/// </summary>
private float plaementVelocity = 0.06f;
/// <summary>
/// 指示这个脚本是否存在BoxCollider
/// </summary>
private bool managingBoxCollider = false;
/// <summary>
/// 用于确定目标的BoxCollider的位置,也被用来表示边界的大小;
/// </summary>
private BoxCollider boxCollider = null;
/// <summary>
/// 用于显示
/// </summary>
private GameObject boundsAsset = null;
/// <summary>
///
/// </summary>
private GameObject ShadowAsset = null;
/// <summary>
/// 对象放置的位置;
/// </summary>
private Vector3 targetPosition;
private void Awake()
{
targetPosition = gameObject.transform.position;
boxCollider = gameObject.GetComponent<BoxCollider>();
if (boxCollider == null)
{
managingBoxCollider = true;
boxCollider = gameObject.AddComponent<BoxCollider>();
boxCollider.enabled = false;
}
//创建并用于指示游戏对象边界的对象;
boundsAsset = GameObject.CreatePrimitive(PrimitiveType.Cube);
boundsAsset.transform.parent = gameObject.transform;
boundsAsset.SetActive(false);
//创建并用指示用作阴影的对象;
ShadowAsset = GameObject.CreatePrimitive(PrimitiveType.Quad);
ShadowAsset.transform.parent = gameObject.transform;
ShadowAsset.SetActive(false);
}
/// <summary>
/// 当对象被选中时调用
/// </summary>
public void OnSelected()
{
if (!IsPlacing)
{
//当对象不是在被放置的时,开始放置;
OnPlacementStart();
}
else
{
// //当对象在被放置的时,停止放置;
OnPlacementStop();
}
}
/// <summary>
/// 开始放置对象;
/// </summary>
private void OnPlacementStart()
{
//启用BoxCollider用于后面操作游戏对象;
if (managingBoxCollider)
{
boxCollider.enabled = true;
}
//隐藏子对象;
for (int i = 0; i < ChildrenToHide.Count; i++)
{
ChildrenToHide[i].SetActive(false);
}
//通知手势可以对这个对象进行操作;
GestureManager.Instance.OverrideFocusedObject = gameObject;
//处于放置的过程;
IsPlacing = true;
}
private void OnPlacementStop()
{
Vector3 position;
Vector3 surfaceNormal;
if (!ValidatePlacement(out position, out surfaceNormal))
{
return;
}
targetPosition = position + (0.01f * surfaceNormal);
OrientObject(true, surfaceNormal);
if (managingBoxCollider)
{
boxCollider.enabled = false;
}
GestureManager.Instance.OverrideFocusedObject = null;
IsPlacing = false;
}
private void OrientObject(bool alignToVerticalSurface, Vector3 surfaceNormal)
{
Quaternion rotation = Camera.main.transform.localRotation;
if (alignToVerticalSurface && (PlacementSurface == PlacementSurfaces_TEST.Vertical))
{
if (Mathf.Abs(surfaceNormal.y) <= (1 - upNormalThreshold))
{
rotation = Quaternion.LookRotation(-surfaceNormal, Vector3.up);
}
}
else
{
rotation.x = 0f;
rotation.z = 0f;
}
gameObject.transform.rotation = rotation;
}
/// <summary>
/// 验证对象是否可以被放置
/// </summary>
/// <param name="position">表面上的目标位置</param>
/// <param name="surfaceNormal">物体放置的表面上的法线</param>
/// <returns>目标位置是否可以被放置,可以为true,否则返回false</returns>
private bool ValidatePlacement(out Vector3 position, out Vector3 surfaceNormal)
{
Vector3 raycastDirection = gameObject.transform.forward;
if (PlacementSurface == PlacementSurfaces_TEST.Horizontal)
{
raycastDirection = -(Vector3.up);
}
position = Vector3.zero;
surfaceNormal = Vector3.zero;
Vector3[] facePoints = GetColliderFacePoints();
for (int i = 0; i < facePoints.Length; i++)
{
facePoints[i] = gameObject.transform.TransformVector(facePoints[i]) + gameObject.transform.position;
}
RaycastHit centerHit;
if (!Physics.Raycast(facePoints[0], raycastDirection, out centerHit, maximumPlacementDisstance, SpatialMappingManager.Instance.LayerMask))
{
return false;
}
position = centerHit.point;
surfaceNormal = centerHit.normal;
for (int i = 0; i < facePoints.Length; i++)
{
RaycastHit hitInfo;
if (Physics.Raycast(facePoints[0], raycastDirection, out hitInfo, maximumPlacementDisstance, SpatialMappingManager.Instance.LayerMask))
{
if (!IsEquivalentDistance(centerHit.distance, hitInfo.distance))
{
return false;
}
}
else
{
return false;
}
}
return true;
}
private Vector3[] GetColliderFacePoints()
{
Vector3 extents = boxCollider.size / 2;
float minX = boxCollider.center.x - extents.x;
float maxX = boxCollider.center.x + extents.x;
float minY = boxCollider.center.y - extents.y;
float maxY = boxCollider.center.y + extents.y;
float minZ = boxCollider.center.z - extents.z;
float maxZ = boxCollider.center.z + extents.z;
Vector3 center;
Vector3 corner0;
Vector3 corner1;
Vector3 corner2;
Vector3 corner3;
if (PlacementSurface == PlacementSurfaces_TEST.Horizontal)
{
center = new Vector3(boxCollider.center.x, minY, boxCollider.center.z);
corner0 = new Vector3(minX, minY, minZ);
corner1 = new Vector3(minX, minY, maxZ);
corner2 = new Vector3(maxX, minY, minY);
corner3 = new Vector3(maxX, minY, maxZ);
}
else
{
center = new Vector3(boxCollider.center.x, boxCollider.center.y, maxZ);
corner0 = new Vector3(minX, minY, maxZ);
corner1 = new Vector3(minX, maxY, maxZ);
corner2 = new Vector3(maxX, minY, maxZ);
corner3 = new Vector3(maxX, maxY, maxZ);
}
return new Vector3[] { center, corner0, corner1, corner2, corner3 };
}
private bool IsEquivalentDistance(float distance1, float distance2)
{
float dist = Mathf.Abs(distance1 - distance2);
return (dist <= distaneThreshold);
}
private void OnDestroy()
{
Destroy(boundsAsset);
boundsAsset = null;
Destroy(ShadowAsset);
ShadowAsset = null;
}
// Update is called once per frame
void Update()
{
if (IsPlacing)
{
// 移动选中的物体
Move();
Vector3 targetPosition;
Vector3 surfaceNormal;
//验证物体是否能够被放置,并设置目标位置,和表面的法线的值
bool canBePlaced = ValidatePlacement(out targetPosition, out surfaceNormal);
//显示物体边框
DisplayBounds(canBePlaced);
//显示阴影
DisplayShadow(targetPosition, surfaceNormal, canBePlaced);
}
else
{
// 隐藏物体边框
boundsAsset.SetActive(false);
//隐藏阴影
ShadowAsset.SetActive(false);
// 将物体放在目标位置的表面上;
float dist = (gameObject.transform.position - targetPosition).magnitude;
if (dist > 0)
{
gameObject.transform.position = Vector3.Lerp(gameObject.transform.position, targetPosition, plaementVelocity / dist);
}
else
{
// 将子物体显示;
for (int i = 0; i < ChildrenToHide.Count; i++)
{
ChildrenToHide[i].SetActive(true);
}
}
}
}
private void Move()
{
Vector3 moveTo = gameObject.transform.position;
Vector3 surfaceNormal = Vector3.zero;
RaycastHit hitInfo;
bool hit = Physics.Raycast(Camera.main.transform.position,
Camera.main.transform.forward,
out hitInfo,
20f,
SpatialMappingManager.Instance.LayerMask);
if (hit)
{
float offsetDistance = hoverDistance;
// Place the object a small distance away from the surface while keeping
// the object from going behind the user.
if (hitInfo.distance <= hoverDistance)
{
offsetDistance = 0f;
}
moveTo = hitInfo.point + (offsetDistance * hitInfo.normal);
lastDistance = hitInfo.distance;
surfaceNormal = hitInfo.normal;
}
else
{
// The raycast failed to hit a surface. In this case, keep the object at the distance of the last
// intersected surface.
moveTo = Camera.main.transform.position + (Camera.main.transform.forward * lastDistance);
}
// Follow the user's gaze.
float dist = Mathf.Abs((gameObject.transform.position - moveTo).magnitude);
gameObject.transform.position = Vector3.Lerp(gameObject.transform.position, moveTo, plaementVelocity / dist);
// Orient the object.
// We are using the return value from Physics.Raycast to instruct
// the OrientObject function to align to the vertical surface if appropriate.
OrientObject(hit, surfaceNormal);
}
/// <summary>
/// 显示物体的边框
/// </summary>
/// <param name="canBePlaced">对象是否位于有效的位置上</param>
private void DisplayBounds(bool canBePlaced)
{
boundsAsset.transform.localPosition = boxCollider.center;
boundsAsset.transform.localScale = boxCollider.size;
boundsAsset.transform.rotation = gameObject.transform.rotation;
// Apply the appropriate material.
if (canBePlaced)
{
boundsAsset.GetComponent<Renderer>().sharedMaterial = PlaceableBoundsMaterial;
}
else
{
boundsAsset.GetComponent<Renderer>().sharedMaterial = NotPlaceableBoundsMaterial;
}
//显示边界
boundsAsset.SetActive(true);
}
/// <summary>
/// 显示放置阴影
/// </summary>
/// <param name="position">放置阴影的位置</param>
/// <param name="surfaceNormal">放置物体的表面法线</param>
/// <param name="canBePlaced">指定放置对象的位置是否有效</param>
private void DisplayShadow(Vector3 position, Vector3 surfaceNormal,bool canBePlaced)
{
// Rotate and scale the shadow so that it is displayed on the correct surface and matches the object.
float rotationX = 0.0f;
if (PlacementSurface == PlacementSurfaces_TEST.Horizontal)
{
rotationX = 90.0f;
ShadowAsset.transform.localScale = new Vector3(boxCollider.size.x, boxCollider.size.z, 1);
}
else
{
ShadowAsset.transform.localScale = boxCollider.size;
}
Quaternion rotation = Quaternion.Euler(rotationX, gameObject.transform.rotation.eulerAngles.y, 0);
ShadowAsset.transform.rotation = rotation;
if (canBePlaced)
{
//可以被放置
ShadowAsset.GetComponent<Renderer>().sharedMaterial = PlaceableShadowMaterial;
}
else
{
//不可被放置
ShadowAsset.GetComponent<Renderer>().sharedMaterial = NotPlaceableBoundsMaterial;
}
// Show the shadow asset as appropriate.
if (position != Vector3.zero)
{
// Position the shadow a small distance from the target surface, along the normal.
ShadowAsset.transform.position = position + (0.01f * surfaceNormal);
ShadowAsset.SetActive(true);
}
else
{
ShadowAsset.SetActive(false);
}
}
}
https://docs.microsoft.com/zh-cn/windows/mixed-reality/holograms-230