Hololens 学习摘要及翻译记录六 Spatial sound

最新推荐文章于 2020-11-03 16:51:56 发布

翕翕堂

最新推荐文章于 2020-11-03 16:51:56 发布

阅读量1k

点赞数

分类专栏： Hololens 文章标签： Hololens 微软文档

本文链接：https://blog.csdn.net/zhousanxi123/article/details/52599328

版权

Hololens 专栏收录该内容

8 篇文章 0 订阅

订阅专栏

阅朗微软官方文档并作出简要记录：

https://developer.microsoft.com/zh-cn/windows/holographic/documentation

个人学习记录，有错误欢迎指出。

HoloLens的六个基本概念之五

Spatial sound: 空间音效 : https://developer.microsoft.com/zh-cn/windows/holographic/spatial_sound

1.When objects are out of our line of sight, one of the ways that we can perceive what's going on around us is through sound. On HoloLens, the audio engine provides the aural component of the mixed-reality experience by simulating 3D sound using direction, distance, and environmental simulations. Using spatial sound in an application allows developers to convincingly place sounds in a 3 dimensional space (sphere) all around the user. Those sounds will then seem as if they were coming from real physical objects or the mixed reality holograms in the user's surroundings. Given that holograms are objects made of light and sometimes sound, the sound component helps ground holograms making them more believable and creating a more immersive holographic experience.

当对象离开我们的视野，可以通过音效来告诉我们它们在哪。在HoloLens里面，音频引擎通过模拟3D音效的方向来源，距离衰减，环境模拟等等来提供混合现实的听觉体验。在应用中使用空间音效，开发者能够很方便的在使用者周围的球形三维空间中放置声音。这些声音听起来就像来自使用者周围的真实对象或全息影像。由于全息影像都是由光和音效组成的，音效能够让全息影像更加的真实可信，也能创造出更加逼真的全息体验。

2.Although holograms can only appear visually where the user's gaze is pointing, your app's sound can come from all directions; above, below, behind, to the side, etc. You can use this feature to draw attention to an object that might not currently be in the user's view. A user can perceive sounds to be emanating from a source in the mixed-reality world. For example, as the user gets closer to an object or the object gets closer to them, the volume increases. Similarly, as objects move around a user, or vice versa, spatial sounds give the illusion that sounds are coming directly from the object.

尽管全息影像只能在凝视的方向出现，但是音效却可以来自四面八方。不在视野范围内的对象可以用这个特性来吸引使用者的注意。使用者能够知道音效是从混合现实中哪儿传来的。比如，使用者靠近一个对象，或者对象靠近使用者，则增加音量，就好像对象正在朝使用者移动，反之亦然。空间音效给人一种错觉就好像音效是直接从对象身上发出的一样。

A.Simulating the perceived location and distance of sounds : 模拟可感知位置和距离的音效

1.By analyzing how sound reaches both our ears, our brain determines the distance and direction of the object emitting the sound. HRTF, or Head Related Transfer Functions, simulates this interaction by modeling the spectral response that characterizes how an ear receives sound from a point in space. The spatial audio engine uses personalized HRTFs to expand the mixed-world holographic experience, and simulate sounds that are coming from various directions and distances.

通过分析耳朵如何接收声音，发现其实是我们大脑确定了对象所发出声音的距离和方向。类人脑传输功能（函数）通过光谱反馈来建模的，通过光谱反馈将耳朵是如何从空间中的一点接收声音的过程特征化（具象化，就是下面第2点）。空间音频引擎使用个性化HRTF来加强混合世界的全息体验，并模拟从来自各个方向和距离的声音。

2.Left or right audio (azimuth) cues originate from differences in the time sound arrives at each ear. Up and down cues originate from spectral changes produced by the outer ear shape (pinnae). By designating where audio is coming from, the system can simulate the experience of sound arriving at different times to our ears. Note that on HoloLens, while azimuth spatialization is personalized, the simulation of elevation is based on an average set of anthropometrics. Thus, elevation accuracy may be less accurate than azimuth accuracy.

产生左右（方位）的音频是因为声音到达每个耳朵的时间不一样。产生上下（方位）的音频，是根据耳廓形状变化所形成的光谱所决定的。通过指定音频的来源，系统能够模拟声音在不同时间达到各个耳朵的体验。要注意的是，在HoloLens上，虽然空间方位是固定的，但是高度的模拟却是根据平均的人体测量设定的，因此，高度的精确性可能没有空间的精确性高。

3.The characteristics of sounds also change based on the environment in which they exist. For instance, shouting in a cave will cause your voice to bounce off the walls, floors, and ceilings, creating an echo effect. The room model setting of spatial sound reproduces these reflections to place sounds in a particular audio environment. You can use this setting to match the user's actual location for simulation of sounds in that space to create a more immersive audio experience.

音效的特征也会依据其所在环境而改变。比如，在一个洞穴中射击，将会使你的声音在墙，地面，天花板上来回反弹，从而产生回音效果。空间音效的房间模式，会使放置在特定音频环境中的声音产生反射。你可以使用这些设定来匹配使用者的真实位置，然后模拟空间中的声音，创建更加真实的音频体验。

B.Integrating spatial sound : 空间音效集合

1.Because the general principle of HoloLens is to ground holograms into the real world, most sounds from holograms should be spatialized. Naturally, there are CPU and memory budget considerations, but you can use 10-12 spatial sound voices while using less than ~12% of the CPU (~70% of one of the four cores). Recommended use for spatial sound voices include:

由于真实世界中全息影像的通用原理，大部分来自全息影像的音效需要应该被空间化。一般来说，从cpu和memory的性能消耗考虑，你可以使用10~12个空间音效，将消耗cpu控制在小于12%（4核cup，其中之一的%70）.推荐空间音效的几点原因：

Gaze Mixing (highlighting objects, particularly when out of view). When a hologram needs a user's attention, play a sound on that hologram (e.g. have a virtual dog bark). This helps the user to find the hologram when it is not in view.
当全息影像在视野外时可以通过空间音效帮助使用者找到他们在哪。
Audio Haptics (reactive audio for touchless interactions). For example, play a sound when the user's hand enters and exits the gesture frame. Or play a sound when the user selects a hologram
可触音频（相对的是无触交互音频）。比如，当使用者的手进入手势框中的时候播放声音。或者当选择一个全息影像时播放声音。
Immersion (ambient sounds surrounding the user).
逼真（环境音环绕）

2.Window's spatial sound engine only supports 48k samplerate for playback. Most middleware programs, like Unity, will automatically convert sound files into the supported format, but when using Windows Audio APIs directly please match the format of the content to the format supported by the effect.

Windows系统空间音效引擎仅仅支持48k采样率。大部分的中间程序，比如Unity，会自动转换音频文件称可支持的格式，但是如果直接使用Windows Audio APIs 是需要自行匹配格式的。