ARCore Augmented Image示例+传送门

想要的效果就是将marker变成一个窗口，通过这个窗口可以看见里面的虚拟内容

做的东西有点类似于传送门，但与网上常见的AR传送门又有一点不同，网上常见的AR传送门其中“门”的位置通常是任意的，b站上有一个老哥的视频，可以看看AR传送门的效果。

【附源码】Unity+ARcore 实现传送门 小意思VR AR教程

我想做的这个“门”是和marker同样大小的，有点像官方的一个演示视频（官方的这个示例里从marker到虚拟物体的过渡做的很流畅）

还有之前看到的一个国外网友做的东西。

效果大概就是这个样子，最终做出的demo：

识别marker

ARCore组件中的Example下有一个名为增强图像（Augmented Image）的示例，展示了ARCore如何识别marker并在上面显示模型。初学记录一下做的过程和理解。

Augmented Image示例

代码主要有两部分AugmentedImageExampleController和AugmentedImageVisualizer。AugmentedImageVisualizer部分是ARCore检测到marker图像之后，负责在其上显示模型的组件。就像HelloAR中显示模型一样。

能够发现在AugmentedVisualizer脚本上留有左下角、右下角、左上角和右上角这四个部分的模型等待绑定。

Prefab目录下的AugmentedImageVisualizer上绑定了同名的script，并为其绑定了对应的四个模型。 在prefab的Inspector中我们看到的如下。

prefab与实际使用时看到的不一样是因为在Visualizer的script中对4部分的位置都分别做了调整。 

 然后在AugmentedImageExampleController中绑定了上面的prefab

坐标系

就算不熟悉ARCore、unity的坐标系，通过Vector3的forward、back等6个向量加上prefab中四个模型和Visualizer脚本中的代码，我们也可以推出来坐标系的样子。如下图：

unity中的世界坐标系是左手坐标系，（用左手做出一个手枪枪毙的动作，将中指指向手张开时掌心方向，大拇指代表X轴，食指代表Y轴，中指代表Z轴）

6个单位向量的xyz值

Vector3.forward  (0,0,1)

Vector3.left  (-1,0,0)

Vector3.up  (0,1,0)

back，right，down分别相反

着色器Shader

网上已经有很多这种“透明物体遮挡实体”的效果了，参考：

https://jingyan.baidu.com/article/9faa7231f98071473c28cb85.html

Unity Shader 之 Mask Shader 实现透明物体遮挡实体的效果_u3dportalmaskshader-CSDN博客

最近正在看《Unity Shader入门精要》这本，微信读书上就有，对入门算是很友好了。

为了实现透明且同时遮挡后面物体的效果，需要单独创建一个Shader，使用VS打开，修改其代码

Shader "Unlit/MaskShader"
{
    SubShader
    {
        Tags { "Queue" = "Geometry-10"}

		Lighting off

		ZTest LEqual 

		ZWrite On

		ColorMask 0
		Pass{}
	}
}

与传送门的不同

与传送门不同的是，要更改“门”的大小到和marker一致，这个很简单，因为ARCore能够估计marker的宽高并返回值。

public AugmentedImage Image;

可通过Image的ExtentX和ExtentZ属性来获得ARCore对增强图像尺寸的估计

代码

 Controller脚本中倒是没什么需要修改的，主要都是在Visualizer中。

用更简单的情况来看，当marker和box的一个顶面一样大。A1是box的一个侧面，为了实现透明墙壁的效果，需要另一个和A1一样大的侧面A2。为A2添加之前准备好的Shader，然后使其和A1“重合”。说重合是不准确的，因为不能使A1和A2的坐标完全重合，否则在实际应用中就会不断闪烁。需要使A2比A1更靠外一些，加一个向外的偏移量，哪怕只有一点点点点点点就行。 

下图中有前后左右底共计5个侧面，同样需要5个透明的侧面。

//-----------------------------------------------------------------------
// <copyright file="AugmentedImageVisualizer.cs" company="Google LLC">
//
// Copyright 2018 Google LLC. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// </copyright>
//-----------------------------------------------------------------------

namespace GoogleARCore.Examples.AugmentedImage
{
    using System;
    using System.Collections.Generic;
    using System.Runtime.InteropServices;
    using GoogleARCore;
    using GoogleARCoreInternal;
    using UnityEngine;

    /// <summary>
    /// Uses 4 frame corner objects to visualize an AugmentedImage.
    /// </summary>
    public class MyVisualizer : MonoBehaviour
    {
        /// <summary>
        /// The AugmentedImage to visualize.
        /// </summary>
        //增强图像Image
        public AugmentedImage Image;

        //box变成marker的几倍
        //必须为大于等于1的值，最好大于等于2
        public float box_scale = 3.0f;
        //box_depth为box的深度
        //注意box要下降的深度应当为box_depth的一半，同时roof是不需要下降的
        public float box_depth = 3.0f;

        /// <summary>
        /// A model for the lower left corner of the frame to place when an image is detected.
        /// </summary>
        /// 
        public GameObject roof_left;
        public GameObject roof_right;
        public GameObject roof_up;
        public GameObject roof_down;
        public GameObject wall_forward;
        public GameObject wall_back;
        public GameObject wall_left;
        public GameObject wall_right;
        public GameObject wall_bottom;
        public GameObject ball_1;
        public GameObject ball_2;
        public GameObject ball_3;
        public GameObject ball_4;
        public GameObject Mask_roof_left;
        public GameObject Mask_roof_right;
        public GameObject Mask_roof_up;
        public GameObject Mask_roof_down;
        public GameObject Mask_wall_forward;
        public GameObject Mask_wall_back;
        public GameObject Mask_wall_left;
        public GameObject Mask_wall_right;
        public GameObject Mask_wall_bottom;

        /// <summary>
        /// The Unity Update method.
        /// </summary>
        //unity的刷新函数update是从Monobehaviour类中继承来的
        //当检测到增强图像时，这个update函数才会不断调用（因为这个函数的用处就是显示在增强图像上的模型）
        public void Update()
        {
           // UnityEngine.Debug.Log("update MyVisualizer");
            //1、当没有预先设置图像时
            //2、图像检索的状态并不是搜索中
            //将四个模型的Active都设置为False
            if (Image == null || Image.TrackingState != TrackingState.Tracking)
            {
                roof_left.SetActive(false);
                roof_right.SetActive(false);
                roof_up.SetActive(false);
                roof_down.SetActive(false);
                wall_forward.SetActive(false);
                wall_back.SetActive(false);
                wall_left.SetActive(false);
                wall_right.SetActive(false);
                wall_bottom.SetActive(false);
                ball_1.SetActive(false);
                ball_2.SetActive(false);
                ball_3.SetActive(false);
                ball_4.SetActive(false);
                Mask_wall_forward.SetActive(false);
                Mask_wall_back.SetActive(false);
                Mask_wall_left.SetActive(false);
                Mask_wall_right.SetActive(false);
                Mask_wall_bottom.SetActive(false);
                Mask_roof_left.SetActive(false);
                Mask_roof_right.SetActive(false);
                Mask_roof_up.SetActive(false);
                Mask_roof_down.SetActive(false);


                return;
            }

            float roof_thickness = 0.0001f;
            //估计图像（图像是2维的）物理宽度、高度的一半
            float halfWidth = Image.ExtentX / 2;
            float halfHeight = Image.ExtentZ / 2;

            //mask缩放和移动时使用的偏移量
            float offset_for_mask_scale = 0.005f;
            float offset_for_mask_pos = 0.001f;//此处应和墙的厚度一致，不然会出现闪烁
            //临时存储数据
            float temp_for_mask_x;
            float temp_for_mask_z;
           
            Vector3 temp;

            //ball
            float min = Math.Min(halfHeight, halfWidth);

            ball_1.transform.localScale =
                new Vector3(0.01f,0.01f,0.01f);

            temp = ball_2.transform.localScale;
            temp.x = min;
            temp.y = min ;
            temp.z = min ;
            ball_2.transform.localScale =
                new Vector3(temp.x, temp.y, temp.z);

            temp = ball_3.transform.localScale;
            temp.x = min ;
            temp.y = min ;
            temp.z = min ;
            ball_3.transform.localScale =
                new Vector3(temp.x, temp.y, temp.z);

            temp = ball_4.transform.localScale;
            temp.x = min ;
            temp.y = min ;
            temp.z = min ;
            ball_4.transform.localScale =
                new Vector3(temp.x,temp.y,temp.z);
            
            
            //roof_left和right的缩放
            temp = roof_left.transform.localScale;
            temp.x = halfWidth * (box_scale - 1);
            temp.z = halfHeight * 2 * box_scale;
            temp_for_mask_x = temp.x + offset_for_mask_scale;
            temp_for_mask_z = temp.z + offset_for_mask_scale;
            roof_left.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            roof_right.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            Mask_roof_left.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);
            Mask_roof_right.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);
            //roof up和down的缩放
            temp = roof_up.transform.localScale;
            temp.x = halfWidth * 2;
            temp.z = halfHeight *(box_scale-1);
            temp_for_mask_x = temp.x + offset_for_mask_scale;
            temp_for_mask_z = temp.z + offset_for_mask_scale;
            roof_up.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            roof_down.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            Mask_roof_up.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);
            Mask_roof_down.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);

            //forward和back的缩放
            temp = wall_forward.transform.localScale;
            temp.x = halfWidth * 2* box_scale;
            temp_for_mask_x = temp.x + offset_for_mask_scale;
            temp.y = box_depth;
            wall_forward.transform.localScale = new Vector3(temp.x, temp.y, temp.z);
            wall_back.transform.localScale = new Vector3(temp.x, temp.y, temp.z);
            Mask_wall_forward.transform.localScale = new Vector3(temp_for_mask_x, temp.y, temp.z);
            Mask_wall_back.transform.localScale = new Vector3(temp_for_mask_x, temp.y, temp.z);

            //left和right的缩放
            temp = wall_left.transform.localScale;
            temp.z = halfHeight * 2* box_scale;
            temp_for_mask_z = temp.z + offset_for_mask_scale;
            temp.y = box_depth;
            wall_left.transform.localScale = new Vector3(temp.x, temp.y, temp.z);
            wall_right.transform.localScale = new Vector3(temp.x, temp.y, temp.z);
            Mask_wall_left.transform.localScale = new Vector3(temp.x, temp.y, temp_for_mask_z);
            Mask_wall_right.transform.localScale = new Vector3(temp.x, temp.y, temp_for_mask_z);
            //bottom的缩放
            temp = wall_bottom.transform.localScale;
            temp.x = halfWidth * 2* box_scale;
            temp.z = halfHeight * 2* box_scale;
            temp.y = box_depth;
            wall_bottom.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            Mask_wall_bottom.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);

            ball_1.transform.localPosition =
                  (box_depth / 2 - ball_2.transform.localScale.y / 2) * Vector3.down;
            ball_2.transform.localPosition = 
                -halfWidth*box_depth*5/2*Vector3.left + (box_depth/2 - ball_2.transform.localScale.y/2)*Vector3.down + halfHeight*box_depth*5/2*Vector3.forward;
            ball_3.transform.localPosition =
                halfWidth*box_depth*5/2* Vector3.left + (box_depth / 2 - ball_2.transform.localScale.y / 2) * Vector3.down + halfHeight*box_depth*5/2 * Vector3.back;
            ball_4.transform.localPosition =
                 (box_depth / 2 - ball_2.transform.localScale.y / 2) * Vector3.down;
            // (halfHeight * (box_scale + 1) / 2 + offset_for_mask_pos) * Vector3.forward;
            //修改roof的位置
            roof_up.transform.localPosition =
                (halfHeight * (box_scale + 1) / 2 + offset_for_mask_pos) * Vector3.forward;
            roof_down.transform.localPosition =
                 (halfHeight * (box_scale + 1) / 2 + offset_for_mask_pos) * Vector3.back;
            roof_left.transform.localPosition =
                 (halfWidth * (box_scale + 1)/2 + offset_for_mask_pos)*Vector3.left;
            roof_right.transform.localPosition =
                 (halfWidth * (box_scale + 1)/2 + offset_for_mask_pos) * Vector3.right;
            //修改roof mask的位置
            Mask_roof_up.transform.localPosition =
                (offset_for_mask_pos * Vector3.up) + halfHeight * (box_scale + 1) / 2 * Vector3.forward;
            Mask_roof_down.transform.localPosition =
                (offset_for_mask_pos * Vector3.up) + halfHeight * (box_scale + 1) / 2 * Vector3.back;
            Mask_roof_left.transform.localPosition =
                (offset_for_mask_pos * Vector3.up) + halfWidth * (box_scale + 1) / 2 * Vector3.left;
            Mask_roof_right.transform.localPosition =
                (offset_for_mask_pos * Vector3.up) + halfWidth * (box_scale + 1) / 2 * Vector3.right;
            //修改位置，让这个box“下沉”，陷入到marker里面。
            wall_forward.transform.localPosition =
                 (box_depth/2 * Vector3.down) + (box_scale*halfHeight * Vector3.forward);
            wall_back.transform.localPosition =
                (box_depth/2 * Vector3.down) + (box_scale * halfHeight * Vector3.back);
            wall_left.transform.localPosition =
                (box_depth/2 * Vector3.down) + (box_scale * halfWidth * Vector3.left);
            wall_right.transform.localPosition =
                (box_depth/2 * Vector3.down) + (box_scale * halfWidth * Vector3.right);
            wall_bottom.transform.localPosition = box_depth/2 * Vector3.down;
            //修改mask的位置，保证能够覆盖墙面的同时，还要不与墙面重叠（否则会有闪烁）
            Mask_wall_forward.transform.localPosition =
                 (box_depth / 2 * Vector3.down) + ((box_scale * halfHeight + offset_for_mask_pos) * Vector3.forward);
            Mask_wall_back.transform.localPosition =
                (box_depth / 2 * Vector3.down) + ((box_scale * halfHeight + offset_for_mask_pos) * Vector3.back);
            Mask_wall_left.transform.localPosition =
                (box_depth / 2 * Vector3.down) + ((box_scale * halfWidth + offset_for_mask_pos) * Vector3.left);
            Mask_wall_right.transform.localPosition =
                (box_depth / 2 * Vector3.down) + ((box_scale * halfWidth + offset_for_mask_pos) * Vector3.right);
            Mask_wall_bottom.transform.localPosition = (box_depth+ offset_for_mask_pos) * Vector3.down;

            roof_left.SetActive(true);
            roof_right.SetActive(true);
            roof_up.SetActive(true);
            roof_down.SetActive(true);
            Mask_roof_left.SetActive(true);
            Mask_roof_right.SetActive(true);
            Mask_roof_up.SetActive(true);
            Mask_roof_down.SetActive(true);

            wall_forward.SetActive(true);
            wall_back.SetActive(true);
            wall_left.SetActive(true);
            wall_right.SetActive(true);
            wall_bottom.SetActive(true);
            ball_1.SetActive(true);
            ball_2.SetActive(true);
            ball_3.SetActive(true);
            ball_4.SetActive(true);
            Mask_wall_forward.SetActive(true);
            Mask_wall_back.SetActive(true);
            Mask_wall_left.SetActive(true);
            Mask_wall_right.SetActive(true);
            Mask_wall_bottom.SetActive(true);

        }
    }
}

最后

感觉用pad来显示marker，因为有屏幕反光的原因，会有一定的抖动。改天把marker打印出来试试抖动会不会小一些。打印出的marker是黑白的也没有关系。

检测仅基于高对比度的点，所以彩色和黑白图像都会被检测到，无论使用彩色还是黑白参考图像。

很简单的东西，因为自己太蠢踩了很多坑，其中大多是因为不熟悉unity和C#绕的弯路，感觉可以做点有意思的小玩意。

初见可能遇到的坑

是我蠢逼了。

1、绑定模型的问题

在换成自己的模型之后，我在Visualizer脚本中试图修改模型的缩放和位置，但是没有效果，模型显示的时候还是出现在初始位置且是初始的缩放比例。在Augmented Image示例中，当改变marker的大小时，模型的位置和大小也会发生改变，也就是说，从一开始模型和marker的相对大小就是固定的，而脚本中有修改模型位置的代码。我的问题就在于，这个修改位置的代码不起作用。

是因为我在Inspector上脚本处绑定物体不对。我绑定的是模型文件夹下的，而不是prefab内的模型。二者是相互独立的，在场景中显示的是prefab内的模型，我们修改的也应当是prefab的组成部分，也就是说之前修改的都修改错了对象。

2、关于localscale缩放大小的问题，要么在script中设置，要么在Inspector中设置，不要两头同时做。

C#中结构体struct的坑

C#中的结构体

localScale.x = 1.0f;

这样是不行的，要修改必须将localScale整个修改，可以搜索关键字“C#结构体”来了解一下这个问题。

localScale = new Vector3(x,y,z);