【3D数据传输策略】VIA（VIsibility-Aware）

论文《VIA: Visibility-aware Web-based Virtual Reality》的阅读笔记

打分模块

与视线夹角+视锥剔除

打包模块（静态打包）

每个对象一个数据包
重用对象会多次传输

x(i,k) : k号块是否需要资源i
y(k,j) : 对象j是否需要k号块
r(i,j) : 对象j是否需要资源i
X=R ： z号块需要的资源就是对象z需要的资源
Y=I ：对象z仅需要z号块

具有可见性感知能力的基于Web的虚拟现实

摘要
问题
we found users can suffer from high latency while waiting for all 3D objects appear in their field-of-view.
原因
This is because storage and fetching of 3D objects in WebXR (and its underlying WebGL libraries) are agnostic to the user’s orientation and location, leading to latency issues.
Specifically, fetching of texture files in arbitrary order results in 3D objects waiting on their texture dependencies, and the storage of all objects’ geometry data in one large file blocks individual objects from rendering even if their texture dependencies are satisfied.
处理方法
To address these issues, we propose a systematic prioritization of which 3D objects and their dependencies should be fetched first,based on the user’s position and orientation in the VR scene. To improve efficiency, the geometry data belonging to each 3D object are optimally grouped together to minimize the average latency.
效果
Our experiments with various WebXR scenes under different network conditions show that our scheme can significantly reduce the time to all 3D objects appearing in the user’s field-of-view, by up to 50%,compared the default WebXR behavior.
1.简介
资源加载与对象渲染的关系
WebXR works by calling WebGL Javascript libraries and retrieving the relevant assets from a remote web server. Once an object has all its dependent assets (geometry and texture data), the object is rendered on the VR display.
难点
Firstly, it is not clear in what order objects should be retrieved, as some objects are entirely contained within the FoV, while other objects may span the entire FoV, or even be out of the FoV entirely.
Secondly, the geometry data associated with WebXR objects is stored in very coarse-grained format (one large binary file), preventing fine-grained resource requests.
解决方法
To address the first challenge, we propose a new scoring function that prioritizes which objects to download, based on whether objects are potentially visible and their orientation from the center of the FoV.
To address the second challenge, we propose grouping geometry data together into logically-meaningful chunks that minimize the average download time of any object in a scene.
相关工作
viewport optimization for web pages [Butkiewicz et al. 2015; Netravali et al. 2016]
360◦ videos [Corbillon et al. 2017; Guan et al. 2019; Qian et al.2018; Zhou et al. 2018]
本文与已有工作的区别
Firstly,web page optimizations that prioritize resources “above the fold” rely on the DOM tree, which does not contain information about WebXR objects and their dependencies;furthermore, the implementation is very different as WebXR involves working with WebGL and Javascript, rather than mainly HTML/Javascript/CSS
Secondly,360◦ video optimizations prioritize 2D tiles in the user’s FoV from a single user location, whereas our solution prioritizes 3D objects and their dependencies, for any user location and orientation.
贡献
We measure latency issues with web-based VR, with the time until all content is rendered in the user’s FoV being up to 41.26 seconds under a 10 Mbps connection, for example. However, the objects in the FoV of that scene consumed only 64% of the total objects requested, indicating there are opportunities for latency savings.
We propose a scoring method to determine which objects should receive priority downloads. The score is a combination of visibility and orientation from the center of the user’s FoV. Given these scores, the WebGL metadata, geometry data, and and Javascript code are minimally modified to request the objects and their dependencies in the appropriate order. To enable fine-grained retrieval of object dependencies, we group relevant geometry data together on the server,enabling efficient download any combination of objects.
To implement the above techniques, we develop a parser to determine WebXR dependencies. We experiment with various WebXR scenes under different network conditions.Our results show that our method reduces page load times by up to 50.3%, compared to the default WebXR implementation.Furthermore, our method works for different user FoVs and is robust to mis-estimation of the user FoV.

数据分组
问题公式
我们将这个问题形式化如下。
场景中存在𝑁数据缓冲区和𝐽对象。
我们试图将数据缓冲区分组到文件中。
来自WebXR元数据的场景构造告诉我们𝑅𝑖𝑗∈{0,1}，对象𝑗是否需要数据缓冲区𝑖。
问题是要确定整数变量𝑋𝑖𝑘(是否应该将databuffer𝑖包含在文件𝑘中)和𝑌𝑘𝑗（对象𝑗是否需要文件𝑘）。
目的是尽量减少视场内可见的一组对象O的下载时间：