黑苹果facetime_苹果如何将增强现实整合到FaceTime中，但尚未实现

黑苹果facetime

Have you been as disappointed by Augmented Reality (AR) as I have? I watched the Apple events and was mildly impressed, if a little bored after the first 30 seconds. Don’t get me wrong, putting virtual objects in a physical world is clever.

您是否像我一样对增强现实(AR)感到失望？ 我观看了Apple事件 ，并在开始30秒后感到有些无聊，这给我留下了深刻的印象。 不要误会我的意思，将虚拟对象放在物理世界中是很聪明的。

• Yes, it is a novelty to turn a bedroom into the virtual storyboard of Goldilocks.
  是的，将一间卧室变成Goldilocks的虚拟故事板是一种新颖。
• It is intriguing to discover how good Ikea furniture would look in my living room.
  有趣的是，发现宜家家具在我的客厅中看起来有多好。

• I even tried the Measure app, although I’m not sure why.

  我什至尝试了Measure应用程序 ，尽管我不确定为什么。

They’re all things you try once or twice, out of curiosity and then forget. AR is the technology that I wish that I could find a use for.

它们都是您出于好奇而尝试一两次的东西，然后忘记了。 AR是我希望可以使用的技术。

I found inspiration in an unlikely place. I was watching the original (and best) Star Wars film. Obi Wan Kenobi presses something on R2D2 and an image of Princess Leia flickers into view.

我在不太可能的地方找到灵感。 我当时在看《星球大战》的原创(也是最佳) 电影 。 Obi Wan Kenobi在R2D2上按了某些内容，然后看到Leia公主的图像闪烁。

What has that to do with anything? It’s AR, and I think you could do something like it with Apple technology. I think it could be used with FaceTime to make the person you’re talking with seem to be in the same room. Add Apple Glasses to the mix, and you’d have something special.

这和什么有关系？ 它是AR ，我认为您可以使用Apple技术来做类似的事情。 我认为它可以与FaceTime一起使用，以使与您交谈的人似乎在同一房间。 将Apple Glasses加入眼镜中，您会感到特别。

You’re sceptical? I’m hear to explain how. Just for grins.

你很怀疑吗？ 我听说要解释一下。 只为咧嘴笑。

那怎么办呢？ (How would that work then?)

Let’s pretend the software and hardware just exist for the moment and focus on the experience.

让我们假装暂时存在的软件和硬件，并专注于体验。

1. I decide to spend some downtime with my friend, Sam, who is as big an Apple fanatic as I am. I decide to use FaceTime AR.
   我决定和我的朋友Sam一起度过一段停机时间，他和我一样，是Apple的狂热分子。 我决定使用FaceTime AR。
2. I find a room with a little space and call her on my iPhone 15. She accepts.
   我找到一个空间很小的房间，然后用我的iPhone 15打电话给她。她接受了。
3. There is a brief pause, and then the iPhone’s AR hardware and camera activate.
   短暂停顿一下，然后激活iPhone的AR硬件和相机。
4. I look through the iPhone’s screen at the room in front of me, and there is Sam.
   我翻阅iPhone的屏幕在我前面的房间，那里是Sam。
5. It’s as if she’s in the room. I see her movements and she mine. We talk with perfect lip sync, and then she wants me to look at a book she’s holding.
   好像她在房间里一样。 我看到她的动作，她也很开心。 我们用完美的口型同步说话，然后她要我看看她拿着的书。
6. I walk over and read the cover.
   我走过去阅读封面。
7. When we’re done, I hang up, and Sam’s image disappears from my living room.
   完成后，我挂了电话，Sam的图像从我的客厅消失了。

Here’s the set-up:

这是设置：

Diagram by author

作者图

I think most of that could be done with hardware Apple will release in the next two years. I know, bold talk, but hear me out.

我认为大多数事情都可以通过苹果在未来两年内发布的硬件来完成。 我知道，大胆的讲话，但请听我说。

iPhone + iPad (iPhone + iPad)

What would need to be there for that to work?

要使它起作用需要什么？

I don’t think that it’s possible with just an iPhone at each end. There will need to be a 3D scanner in front of each person, sitting a little distance away. That’s needed to track each person’s position and movements.

我认为两端都没有iPhone是不可能的。 每个人前面都需要有一台3D扫描仪，距离一定距离。 这是跟踪每个人的位置和动作所必需的。

Let’s use iPads as scanners. People might have an iPad and an iPhone, but I doubt many people would have two iPhones.

让我们使用iPad作为扫描仪。 人们可能拥有iPad和iPhone，但我怀疑许多人会拥有两部iPhone。

Let’s try that FaceTime AR call again, but add in a little detail.

让我们再次尝试FaceTime AR调用，但添加一些细节。

1. I place my iPad on the ground in front of me, and call my friend, Sam on my iPhone 15.
   我将iPad放在我面前的地面上，并在iPhone 15上给我的朋友Sam打电话。
2. Sam accepts my FaceTime AR call and puts her iPad on the ground in front of her.
   Sam接了我的FaceTime AR电话，并将iPad放在她面前的地上。
3. Each iPhone pairs with its respective iPad.
   每个iPhone与各自的iPad配对。
4. My iPad’s 3D scanner turns on and scans me. It starts sending 3D data to Sam’s iPhone.
   我的iPad的3D扫描仪会打开并扫描我。 它开始向Sam的iPhone发送3D数据。
5. Sam’s iPhone starts receiving my 3D data and creates an AR image of me, which it places in Sam’s room. Her iPhone and iPad reverse the process.
   Sam的iPhone开始接收我的3D数据并创建我的AR图像，并将其放置在Sam的房间中。 她的iPhone和iPad逆转了这一过程。
6. Sam’s AR image appears in my living-room.
   Sam的AR图像出现在我的客厅里。
7. We talk and it’s as if we’re in the same room. I look at the cover of her book.
   我们说话，就好像我们在同一个房间里一样。 我看她的书的封面。

Diagram by author

作者图

That wasn’t too bad. I didn’t need to dream up anything that isn’t already out there. The 2020 iPad Pro has a really good 3D scanner, and I suspect the 2020 iPhone will too.

那还不错。 我不需要幻想任何尚未存在的事物。 2020 iPad Pro具有非常好的3D扫描仪，我怀疑2020 iPhone也会如此。

What are the big challenges aside from the unimaginable complexity of the software?

除了软件难以想象的复杂性之外，还有哪些重大挑战？

• The volume of data (for sure).
  数据量(肯定)。
• Latency (of course).
  延迟(当然)。
• Looking at the cover of the book (really?).
  看着书的封面(真的吗？)。

迎接挑战 (Rising to challenges)

数据量 (Volume of data)

Is the volume of data really a big issue? The current hardware already easily handles 4K video.

数据量真的是一个大问题吗？ 当前的硬件已经可以轻松处理4K视频。

Handling 3D images massively increases the amount of information you have to send. Let me prove that to you.

处理3D图像会大大增加您必须发送的信息量。 让我向您证明这一点。

I’m going to oversimplify and ignore any image compression. Suppose I want to send a 2D image of a square, which is 10 pixels wide x 10 pixels high. How many pixels do I send? Easy, 10 x 10 = 100 pixels of information.

我将简化并忽略任何图像压缩。 假设我要发送一个正方形的2D图像，该图像是10像素宽x 10像素高。 我要发送多少像素？ 轻松，10 x 10 = 100像素的信息。

Let’s go 3D. Suppose I have a cube that is 10 pixels x 10 pixels x 10 pixels. I’d have to send 6 sides x 10 x 10 pixels of information = 600 pixels. That six times more information than the square, and that’s a really simple object

让我们开始3D。 假设我有一个10像素x 10像素x 10像素的立方体。 我必须发送6面x 10 x 10像素的信息= 600像素。 比正方形多六倍的信息，这是一个非常简单的对象

Of course, clever people will use all sorts of tricks to compress images, but it’s still a lot more data, especially if you have to deal with 8K video when it turns up. I suspect we’ll need a few more iterations of ARM processors before we can handle that.

当然，聪明的人会使用各种技巧来压缩图像，但是仍然有很多数据 ，尤其是当您必须处理8K视频时。 我怀疑在处理该问题之前，我们将需要更多几次ARM处理器迭代。

What do we do in the meantime? We could simplify the AR 3D image by turning it into a cartoon. As the hardware speeds up, the image becomes more and more realistic. The hardware could also build images progressively. It starts off quite blocky and then goes high-res.

同时，我们要做什么？ 我们可以通过将AR 3D图像变成卡通来简化它。 随着硬件的加速，图像变得越来越逼真。 硬件也可以逐步构建映像。 它开始时非常块状，然后变为高分辨率。

潜伏 (La…ten….cy)

The challenges don’t stop with the volume of data. You have to analyse all that data in real-time, send it across the network, and reassemble it at the other end. The other end then has to project it on to the room. That’s a room it is also analysing in real time!

挑战不止于数据量 。 您必须实时分析所有数据，通过网络发送它们，然后在另一端重新组装。 然后另一端必须将其投影到房间上。 这是一个正在实时分析的房间！

All this processing risks causing latency, which causes big problems for speech. As I speak, my lips move. The 3D scanner captures that and the iPad has to analyse it, package it up and send it off. In the meantime, the sounds have long since gone, because that’s easy to process.

所有这些处理过程都有导致延迟的风险，这会导致语音问题。 我说话时，嘴唇动了动。 3D扫描仪会捕获到该图像，然后iPad必须对其进行分析，打包并发送出去。 同时，声音早已消失，因为它很容易处理。

How could we get around that?

我们如何解决这个问题？

Not all parts of the AR image are equally important. I’d prioritise idetails that really matter. I’d focus my efforts on facial expressions and movements. They have to be bang on. I don’t think it would matter if an arm movement or something like that is not exactly in sync.

并非AR图像的所有部分都同样重要。 我会优先考虑真正重要的细节。 我会将精力集中在面部表情和动作上。 他们必须努力。 我认为手臂运动或类似动作并不完全同步并不重要。

What about looking at the cover of Sam’s book? That’s a problem because the scanning iPad can’t see it.

看山姆的书封面怎么样？ 这是一个问题，因为正在扫描的iPad无法看到它。

从书的封面判断AR幻觉 (Judging the AR illusion by the book’s cover)

If FaceTime AR is to live up to the hype, each participant needs to be able to move around and see in AR what would see in real life.

如果FaceTime AR要不辜负宣传，则每个参与者都需要能够四处走动，并在AR中看到现实生活中会看到什么。

Sam, for example, wants to show me a book she’s holding. I walk over to look at it. That’s an issue, because the book would be flat and the scanner iPad woudn’t be able to ‘see’ the top of the book.

例如，萨姆(Sam)想给我看她拿着的一本书。 我走过去看看。 这是一个问题，因为书本会是平坦的，而扫描仪iPad无法“看到”书本的顶部。

There is at least one solution. Sam points her iPhone down, and its camera scans the cover of the book. AI and raw processing power in the iPhone and the iPads make the image appear in my the AR view I’m enjoying.

至少有一种解决方案。 山姆将她的iPhone指向下方，并且其相机会扫描这本书的封面。 iPhone和iPad的AI和原始处理能力使图像显示在我喜欢的AR视图中。

That’s one possible way around that challenge. You could also have camera all around the room, but who wants that?

这是解决挑战的一种可能方法。 您还可以在整个房间内都配备摄像头，但是谁想要呢？

苹果眼镜 (Apple Glasses)

One of the other big stumbling blocks with AR is that you have to walk around with a phone in front of you. You look like an idiot, and you might walk into a post. For FaceTime AR that’s not a showstopper, because it’ll all happen in the privacy of your front room. Still, it’s a bit awkward.

AR的另一个最大绊脚石是，您必须随身携带一部手机。 您看起来像个白痴，您可能会走进一个职位。 对于FaceTime AR而言，这不是一件好事，因为这一切都将发生在您前厅的私密环境中。 还是有点尴尬。

Enter Apple Glasses. I know Apple hasn’t released the product yet, but I bet they will do. AR just isn’t practical without them, and Apple have made a really big deal about AR. So here’s FaceTime AR with Apple Glasses. It looks a lot more fun, doesn’t it?

输入苹果眼镜。 我知道Apple尚未发布该产品，但我敢打赌他们会发布。 没有它们 ， AR就是不切实际的 ，而且苹果公司对AR做出了很大的贡献。 所以这是带Apple Glasses的FaceTime AR。 看起来好多了，不是吗？

Diagram by author

作者图

The iPhone acts as a consolidation point for the iPad and the glasses. It takes the 3D data from the iPad and sends to the other end. It receives remote 3D information and packages it up for the glasses. I suspect that the glasses would do nothing more than display whatever the iPhone tells it to.

iPhone是iPad和眼镜的整合点。 它从iPad提取3D数据并发送到另一端。 它接收远程3D信息并将其打包为眼镜。 我怀疑这只眼镜除了显示iPhone指示的内容外无非。

新希望 (A new hope)

I’ve described how a technology that I previously thought just a novelty, AR, could actually do something really cool. Allied with FaceTime, it could project the people we care about into our living room, even if they’re on the other side of the world.

我已经描述了我以前认为只是一种新颖的技术AR实际上可以做些非常酷的事情。 与FaceTime结盟，即使他们在世界的另一端，也可以将我们关心的人投射到客厅。

• I described in very broad terms how it might work with technology that already exists — sort of. It may need a little extra speed :-).
  我用非常广泛的术语描述了它如何与现有技术结合使用。 它可能需要一点额外的速度:-)。
• I drew attention to what I thought the big stumbling blocks would be and suggested ways to get around them.
  我提请大家注意我认为最大的绊脚石，并提出解决这些障碍的方法。
• I went crazy and added technology that Apple hasn’t released, Apple Glasses. I speculated that they would perfect the illusion.
  我发疯了，并添加了Apple尚未发布的技术Apple Glass。 我推测他们会完善错觉。

It doesn’t really matter whether that’s how it works or not in the end. What is cool is that there are still new ways technology could improve our lives. I was getting a little worried it was all turning into better, faster, but meh.

到底这是如何工作并不重要。 很棒的是，技术仍然可以改善我们的生活。 我有点担心这一切都会变得更好，更快，但是。

获得访问专家视图的权限- 订阅DDI Intel (Gain Access to Expert View — Subscribe to DDI Intel)

翻译自: https://medium.com/datadriveninvestor/how-apple-could-integrate-augmented-reality-into-facetime-but-probably-not-yet-74835427e5e8

黑苹果facetime