VR开发基础（二）一文详解Oculus环境helloxr的openxr核心接口流程

注：Oculus使用的是开源的hello_xr示例，但要使用自家的loader；在hello_xr上篇侧重分析了入口和图形的基本流程，此篇将侧重分析XR相关的流程

一，Setup and Build hello_xr

1，下载hello_xr官方示例、Oculus的sdk，获取loader

The OpenXR™ Software Development Kit (SDK) Sources Project contains the hello_xr

GitHub - KhronosGroup/OpenXR-SDK-Source: Sources for OpenXR loader, basic API layers, and example code.

Oculus不支持标准的openxr的loader，需要下载ovr_openxr_mobile_sdk，取出对应的loader，/OpenXR/Libs/Android/ in SDK package;

修改hello_xr的cmake，将loader替换为Oculus提供的。

download the Oculus OpenXR Mobile SDK  Downloads page.

2，修改配置文件，直接引用已下载的loader

replace this line:

option(BUILD_LOADER "Build loader" ON)

with these:

option(BUILD_LOADER "Build loader" OFF)
add_library(openxr_loader SHARED IMPORTED)
set_property(
 TARGET
 openxr_loader
 PROPERTY
 IMPORTED_LOCATION
 C:/<path_to_Oculus_SDK_folder>/OpenXRMobileSDK/OpenXR/Libs/Android/${ANDROID_ABI}/${CMAKE_BUILD_TYPE}/libopenxr_loader.so
)

修改externalNativeBuild

 externalNativeBuild {
    cmake {
        arguments '-DANDROID_STL=c++_shared',
                '-DBUILD_API_LAYERS=OFF',
                '-DBUILD_TESTS=ON',
                '-DBUILD_LOADER=ON',
                '-DBUILD_CONFORMANCE_TESTS=OFF',
                '-DBUILD_ALL_EXTENSIONS=ON'
        targets "openxr_loader", "hello_xr"
    }
 }

更新

 externalNativeBuild {
    ndk {
        abiFilters 'arm64-v8a', 'armeabi-v7a'
    }
    cmake {
        cppFlags ''
    }
 }

• Manifest增加Oculus项

src\tests\hello_xr\AndroidManifest.xml两处

<application>
<meta-data android:name="com.oculus.intent.category.VR" android:value="vr_only"/>
<meta-data android:name="com.oculus.supportedDevices" android:value="quest|quest2"/>

<intent-filter>
    <action android:name="android.intent.action.MAIN" />
    <category android:name="com.oculus.intent.category.VR" />
    <category android:name="android.intent.category.LAUNCHER" />
 </intent-filter>

二，重要概念

OpenXR 核心概念和openXR官方文档：

study the OpenXR 1.0 Specification at the Khronos Group site.

更多核心概念参考：spec1.0中Fundamentals部分

 The site offers API reference documentation and a PDF reference guide that provides a detailed overview of the API.

 

Important: To get started with OpenXR core concepts, review Core Concepts and the rest of the topics under this doc set. These cover concepts and examples for:

在此面向应用开发，基于sample和框架的分析，review一些重要概念和流程

1 系统System

XR系统中定义了各种子系统，提供特定领域的功能，并有设备和硬件各子模块来实现，比如我们可以通过各个独立的设备获取不同的input输入、display frames显示等等；

——参考unity的XR plugin systems架构，即可理解unity的这一设计背景。

这些设备和模块在XR系统加载起来后，由运行时runtime来控制其激活和响应；

——参考unity新input action system，可理解这一软硬件解耦的框架。

 

2 对象概念和原子定义Handles and Atoms

Handles概念

代表运行时中的各object，有相应的创建销毁声明周期行为，应用可以调用对应的方法进行控制；这些对象在创建后，形成 父子管理 结构（也可以理解为 容器管理 结构），父级Handle往往管理子级的声明周期；

比如

要创建XrActionSet  （handle），调用xrCreateActionSet 方法，传入XrInstance （handle）参数，那么XrInstance 就是 XrActionSet 的父级句柄，持有着ActionSet对象；

要销毁 XrSession， XrSpace and XrSwapchain各子级对象都会被一同销毁；

应用中几个重要的句柄类型如下：

 

Handle type	Description
XrInstance	Enables apps to use the OpenXR runtime. These handles are the only ones without a parent.
XrSession	Enables interaction with a specific XR device, allows for user input, rendering, managing events, and more. Generally, it is a graphics interaction with a particular XrSystem using a particular graphics binding.
XrAction	Refers to individual actions when retrieving input data or sending haptic events. Apps receive input action states without directly receiving input from the hardware.
XrActionSet	Refers to groups of actions that are valid in a certain context (for example, all actions to interact with a menu UI). Action sets can be enabled or disabled.
XrSwapchain	Manages display of rendered images as a series of frame buffers. Images can be organized in multiple swapchains (queues of images to be displayed to the user).
XrSpace	Represents spaces that the app can reason about through xrLocateSpace.

 

OpenXR extensions通过创建新的handle对象，及其struct和functions来定义和扩展更多的api。

atoms 概念

定义为静态常量，在运行时中预定义了相关的数值，代表了对象无关的特征，比如XrPath and XrSystemID;

XrPath （atom）本质上是一个固定的数值，指向一个string，即语义化的设备路径如“/interaction_profiles/oculus/touch_controller”；XrSystemID制定了支持OpenXR的子系统的id；

3 传参封装structs和struct-chaining extension

openXR方法大量使用struct的结构传参，有更好的封装和扩展性，使得新的extension扩展可以增加新的参数而不影响原来的api；

另一个特性就是对extension扩展使用的struct成员构建了参数指针链，要求是扩展的这部分extension必须在创建instance时使能enable：

In order to use a structure type defined by an extension in a next chain, the proper extension must have been previously enabled during xrCreateInstance

 

Read-only的 struct pointer chain常常定义为

typedef struct XrBaseInStructure {
    XrStructureType                    type;
    const struct XrBaseInStructure*    next;
} XrBaseInStructure;

例如：xrCreateInstance 调用时，传入的参数结构体：

typedef struct XrInstanceCreateInfo {

    XrStructureType          type;    //for example XR_TYPE_INSTANCE_CREATE_INFO 用于对象创建

    const void*              next;        //next is NULL or a pointer to the next struct in a struct-chain.

    ...

    const char* const*       enabledExtensionNames;  //pointer to a string array with the names of extensions to enable

} XrInstanceCreateInfo;

 

XrInstanceCreateInfo createInfo{XR_TYPE_INSTANCE_CREATE_INFO};
createInfo.next = m_platformPlugin->GetInstanceCreateExtension();
...
createInfo.enabledExtensionNames = extensions.data();
...
CHECK_XRCMD(xrCreateInstance(&createInfo, &m_instance));

 

 

4 复调函数Two-call

open XR的某些方法需要先接收buffer size的参数再返回具体数据，使用一个方法，通过二次调用来达成（一次传参，二次返回）。

常见风格：

XrResult xrFunction(uint32_t capacityInput, uint32_t* countOutput, float* elements);

第一次调用传入

a valid countOutput pointer

a nullptr as elements

0 as capacityInput

// the function sets the value pointed to by countOutput to the required size in number of elements

二次调用传入

a buffer at least as large as the value that countOutput points to

a pointer to the allocated buffer as elements

the length of the buffer as capacityInput

//  the elements that have been written to the elements buffer.

Hello_xr示例：

uint32_t count;
CHECK_XRCMD(xrEnumerateEnvironmentBlendModes(m_instance, m_systemId, type, 0, &count, nullptr));
...
std::vector<XrEnvironmentBlendMode> blendModes(count);
CHECK_XRCMD(xrEnumerateEnvironmentBlendModes(m_instance, m_systemId, type, count, &count, blendModes.data()));

// count stores the number of supported blend modes

// 复调返回vector blendModes stores the supported environment blend modes.

 

三，Creating Instances and Sessions

下面进入核心API和基础流程，openxr的整体会话流程图上一篇有介绍过，结合helloxr的示例，在此抽象出应用的核心调用的执行关系：

1，入口void android_main(struct android_app* app)方法

总体来说，主要进行三大块

loader init、CreateInstance和InitializeSession、跑while{事件和渲染}

因此首先进行loader、平台等实例和系统会话的处理；

使用loader extensions, 方法定义为xrInitializeLoaderKHR,  xrGetInstanceProcAddress获取方法指针initializeLoader；

参数定义为XrLoaderInitInfoBaseHeaderKHR结构体（如上介绍的，XrStructureType风格，type为XR_TYPE_LOADER_INIT_INFO_ANDROID_KHR）

//Initializetheloaderforthisplatform

PFN_xrInitializeLoaderKHRinitializeLoader=nullptr;

if(XR_SUCCEEDED(

xrGetInstanceProcAddr(XR_NULL_HANDLE,"xrInitializeLoaderKHR",(PFN_xrVoidFunction*)(&initializeLoader)))){

XrLoaderInitInfoAndroidKHR  loaderInitInfoAndroid;

memset(&loaderInitInfoAndroid,0,sizeof(loaderInitInfoAndroid));

loaderInitInfoAndroid.type=XR_TYPE_LOADER_INIT_INFO_ANDROID_KHR;

loaderInitInfoAndroid.next=NULL;

loaderInitInfoAndroid.applicationVM=app->activity->vm;

loaderInitInfoAndroid.applicationContext=app->activity->clazz;

initializeLoader((constXrLoaderInitInfoBaseHeaderKHR*)&loaderInitInfoAndroid);

}

2，平台与图形实例：

此处设备操作，即涉及到创建session：

• Create XrInstance.
• Use XrInstance to get XrSystemId.
• Create a session using XrSystemId.

在设备上跑openXR，首先需要设置Graphic api，确定platform信息，各种平台和图形api extension事先进行定义；Oculus上的flag定义：

Graphics API Header Control:

XR_USE_GRAPHICS_API_D3D11, XR_USE_GRAPHICS_API_D3D12,

XR_USE_GRAPHICS_API_OPENGL or XR_USE_GRAPHICS_API_VULKAN

Window System Header Control:

XR_USE_PLATFORM_WIN32

首先：

Hello_xr设计了IOpenXrProgram、IGraphicsPlugin和IPlatformPlugin抽象层，封装了platform-specific等实例及其创建方法和参数结构，让hello_xr可以支持多平台；

//Createplatform-specificimplementation.

std::shared_ptr<IPlatformPlugin>platformPlugin=CreatePlatformPlugin(options,data);

//CreategraphicsAPIimplementation.

std::shared_ptr<IGraphicsPlugin>graphicsPlugin=CreateGraphicsPlugin(options,platformPlugin);

//InitializetheOpenXRprogram.

std::shared_ptr<IOpenXrProgram>program=CreateOpenXrProgram(options,platformPlugin,graphicsPlugin);

然后AndroidPlatformPlugin等 的实现：

struct AndroidPlatformPlugin : public IPlatformPlugin {

    AndroidPlatformPlugin(const std::shared_ptr<Options>& /*unused*/, const std::shared_ptr<PlatformData>& data) {

        instanceCreateInfoAndroid = {XR_TYPE_INSTANCE_CREATE_INFO_ANDROID_KHR};

        instanceCreateInfoAndroid.applicationVM = data->applicationVM;

        instanceCreateInfoAndroid.applicationActivity = data->applicationActivity;

    }

    std::vector<std::string> GetInstanceExtensions() const override { return {XR_KHR_ANDROID_CREATE_INSTANCE_EXTENSION_NAME}; }

    XrBaseInStructure* GetInstanceCreateExtension() const override { return (XrBaseInStructure*)&instanceCreateInfoAndroid; }

    XrInstanceCreateInfoAndroidKHR instanceCreateInfoAndroid;// 初始化先设置vm、activity

};

在然后：

program->CreateInstance(); // 调用loader的xrCreateInstance(&createInfo, &m_instance)

program->InitializeSystem();// 调用graphicsPlugin：m_graphicsPlugin->InitializeDevice(m_instance, m_systemId);获取systemId

program->InitializeSession();  // 调用xrCreateSession(m_instance, &createInfo, &m_session)，然后InitializeActions->xrAttachSessionActionSets(m_session, &attachInfo)

program->CreateSwapchains();// 调用xrCreateSwapchain(m_session, &swapchainCreateInfo, &swapchain.handle)

 

While(){

program->RenderFrame()//RenderFrame()->RenderLayer->

m_graphicsPlugin->RenderView

}

3，Create Instance

如上，接着进入CreateInstance

createInstanceinternal()前，还看到可以遍历openXR接口，此时也是自定义extensions接口的时机；

在此使能了graphic和platform plugin的extension；

先遍历了apilayer，再使用layerName遍历extensions，extensions是openxr的接口扩展机制可以定义新的方法，也可以修改现有方法。

两者在instance创建时使能，instance之后接口确定下来；

// Write out extension properties for a given layer.

const auto logExtensions = [](const char* layerName, int indent = 0) {

    uint32_t instanceExtensionCount;

    CHECK_XRCMD(xrEnumerateInstanceExtensionProperties(layerName, 0, &instanceExtensionCount, nullptr));

    std::vector<XrExtensionProperties> extensions(instanceExtensionCount);

    for (XrExtensionProperties& extension : extensions) {

        extension.type = XR_TYPE_EXTENSION_PROPERTIES;

    }

// 复调函数

    CHECK_XRCMD(xrEnumerateInstanceExtensionProperties(layerName, (uint32_t)extensions.size(), &instanceExtensionCount, extensions.data()));

    ...

};

    void CreateInstanceInternal() {

        CHECK(m_instance == XR_NULL_HANDLE);

        // Create union of extensions required by platform and graphics plugins.

        std::vector<const char*> extensions;

        // Transform platform and graphics extension std::strings to C strings.

        const std::vector<std::string> platformExtensions = m_platformPlugin->GetInstanceExtensions();

        std::transform(platformExtensions.begin(), platformExtensions.end(), std::back_inserter(extensions),

                       [](const std::string& ext) { return ext.c_str(); });

        const std::vector<std::string> graphicsExtensions = m_graphicsPlugin->GetInstanceExtensions();

        std::transform(graphicsExtensions.begin(), graphicsExtensions.end(), std::back_inserter(extensions),

                       [](const std::string& ext) { return ext.c_str(); });

        XrInstanceCreateInfo createInfo{XR_TYPE_INSTANCE_CREATE_INFO};

        createInfo.next = m_platformPlugin->GetInstanceCreateExtension();// 改为XR_TYPE_INSTANCE_CREATE_INFO_ANDROID_KHR

        createInfo.enabledExtensionCount = (uint32_t)extensions.size();

        createInfo.enabledExtensionNames = extensions.data();

        strcpy(createInfo.applicationInfo.applicationName, "HelloXR");

        createInfo.applicationInfo.apiVersion = XR_CURRENT_API_VERSION;

        CHECK_XRCMD(xrCreateInstance(&createInfo, &m_instance));

    }

將platform and graphics插件提供的方法合入extensions；

使用结构体参数XrInstanceCreateInfo ，創建instance

4，System Initialization

如上进入program->InitializeSystem();获取systemId。

XrSystemGetInfo systemInfo{XR_TYPE_SYSTEM_GET_INFO};

systemInfo.formFactor = m_formFactor;

CHECK_XRCMD(xrGetSystem(m_instance, &systemInfo, &m_systemId));

5，Session Creation

实例和会话的区别：apps使用一个openxr instance来与runtime通信，使用session来绑定一个具体的VR系统和graphicbinding扩展；

如上進入program->InitializeSession();获取session

XrSessionCreateInfo createInfo{XR_TYPE_SESSION_CREATE_INFO};

createInfo.next = m_graphicsPlugin->GetGraphicsBinding();

createInfo.systemId = m_systemId;

CHECK_XRCMD(xrCreateSession(m_instance, &createInfo, &m_session));

createInfo通过graphic插件提供，传入graphics binding header，可见session与图形系统、窗口上下文绑定；

Graphicsplugin_opengles.cpp

#ifdefXR_USE_PLATFORM_ANDROID

XrGraphicsBindingOpenGLESAndroidKHRm_graphicsBinding{XR_TYPE_GRAPHICS_BINDING_OPENGL_ES_ANDROID_KHR};

#endif

constXrBaseInStructure*GetGraphicsBinding()constoverride{

returnreinterpret_cast<constXrBaseInStructure*>(&m_graphicsBinding);

}

上述graphicsBinding来自初始systemId和初始设备时：

Void InitializeDevice(XrInstance  instance,XrSystemId  systemId)override{

#ifdefined(XR_USE_PLATFORM_ANDROID)

m_graphicsBinding.display=window.display;

m_graphicsBinding.config=(EGLConfig)0;

m_graphicsBinding.context=window.context.context;

#endif

}

在program->InitializeSession()中得到session后，接着将完善获取 actions（后续交互）和appSpace（后续物体渲染坐标系）：

• 绑定actions InitializeActions();
• 并创建全局app space：CHECK_XRCMD(xrCreateReferenceSpace(m_session, &referenceSpaceCreateInfo, &m_appSpace));
• 最后回到渲染迭代循环时，每次pollEvent，获取到session状态改变时，需要HandleSessionStateChangedEvent（）{ready: xrBeginSession()}

6，Collecting System Information

一，SystemID的意义

运行时在xrCreateInstance获取实例之后跑起来，但是session会话还没有启动，按需設定条件FormFactor来调用xrGetSystem，从运行时获取一个XrSystemId （atom）；

一个有效的XrSystemId 意味着系统已经激活和支持，系统条件指向了该系统配置信息；

支持的系统与FormFactor

OpenXR当前只支持枚举XR_FORM_FACTOR_HANDHELD_DISPLAY and XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY（一般用于枚举XR system）；

文档注：如果有多个系统在一个runtime同时连接，一般不暴露给app做选择，openXR也没有api来list 可用系统，一般交给runtime来设置，Oculus是由用户在desktop app上选择。

2，系统相关信息：

获取XrSystemId ，需要处理一些相关信息

支持的环境混合模式，用于获取systemID。这是指系统支持的混合模式。其中包括的枚举值：不透明XR_ENVIRONMENT_BLEND_MODE_OPAQUE，相加混合XR_ENVIRONMENT_BLEND_MODE_ADDITIVE，合成层与现实世界进行alpha 混合XR_ENVIRONMENT_BLEND_MODE_ALPHA_BLEND

文档注：XR_ENVIRONMENT_BLEND_MODE_OPAQUE on Quest, Quest 2, and Rift.

系统view数量等配置，获得systemID后遍历取得。

例如，单屏或立体，the number of views

XR_VIEW_CONFIGURATION_PRIMARY_MONO for single screen and XR_VIEW_CONFIGURATION_PRIMARY_STEREO for dual screen or one screen per eye)，调用xrBeginSession需要传入选择的view config;

二，Form Factor和configurations

构造条件，使用XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY类型的formFactor的systeminfo，获取systemID

1，hello_xr的systemId获取示例：

XrFormFactor m_formFactor{XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY};

...

XrSystemId m_systemId{XR_NULL_SYSTEM_ID};

...

XrSystemGetInfo systemInfo{XR_TYPE_SYSTEM_GET_INFO};

systemInfo.formFactor = m_formFactor;

CHECK_XRCMD(xrGetSystem(m_instance, &systemInfo, &m_systemId));

后续将使用m_systemId and the instance来创建session和初始化graphic api

2，遍历view configurations

复调方法：

uint32_t viewConfigTypeCount;

CHECK_XRCMD(xrEnumerateViewConfigurations(m_instance, m_systemId, 0, &viewConfigTypeCount, nullptr));

std::vector<XrViewConfigurationType> viewConfigTypes(viewConfigTypeCount);

CHECK_XRCMD(xrEnumerateViewConfigurations(m_instance, m_systemId, viewConfigTypeCount, &viewConfigTypeCount,

            viewConfigTypes.data()));

CHECK((uint32_t)viewConfigTypes.size() == viewConfigTypeCount);

2）viewConfigProperties

for (XrViewConfigurationType viewConfigType : viewConfigTypes) {

    ...

    XrViewConfigurationProperties viewConfigProperties{XR_TYPE_VIEW_CONFIGURATION_PROPERTIES};

    CHECK_XRCMD(xrGetViewConfigurationProperties(m_instance, m_systemId, viewConfigType, &viewConfigProperties));

XrViewConfigurationView

uint32_t viewCount;

CHECK_XRCMD(xrEnumerateViewConfigurationViews(m_instance, m_systemId, viewConfigType, 0, &viewCount, nullptr));

if (viewCount > 0) {

    std::vector<XrViewConfigurationView> views(viewCount, {XR_TYPE_VIEW_CONFIGURATION_VIEW});

    CHECK_XRCMD(xrEnumerateViewConfigurationViews(m_instance, m_systemId, viewConfigType, viewCount, &viewCount, views.data()));

4）XrEnvironmentBlendMode

uint32_t count;

CHECK_XRCMD(xrEnumerateEnvironmentBlendModes(m_instance, m_systemId, type, 0, &count, nullptr));

...

std::vector<XrEnvironmentBlendMode> blendModes(count);

CHECK_XRCMD(xrEnumerateEnvironmentBlendModes(m_instance, m_systemId, type, count, &count, blendModes.data()));

获取blend modes和XrViewConfigurationType

7，Input API

按照openxr标准的input，组织成action set，绑定到session

xrAttachSessionActionSets

General Flow

To sum up, managing input occurs through this flow:

// For each ActionSet:
xrCreateActionSet(...)

// For each Action
xrCreateAction(...)

// For each ActionSpace
xrCreateActionSpace(...)

// For each supported and tested device:
xrSuggestInteractionProfileBindings(...)

xrAttachSessionActionSets(...)

while (frameLoop) {
    // Only once per frame:
    xrSyncActions(...)

    // As many times as needed, can be multiple times for the same action
    xrGetActionStateTYPE(...)
    xrLocateSpace(...)
}

8，Creating and Using Swapchains

Enabling graphics pipelining relies on swapchains, which are queues of images to be displayed to the user.

一，Enumerating Swapchain Formats

说明一下swapchain的formats

也即OpenGL internal formats, GL_RGB10_A2 (10 bits for Red, 10 bits for Green, 10 bits for Blue, 2 bits for Alpha) for OpenGL-based graphics,

or, for Vulkan, formats such as VK_FORMAT_B8G8R8A8_SRGB (8 bits for Red, 8 bits for Green, 8 bits for Blue, 8 bits for Alpha).

// Select a swapchain format.

uint32_t swapchainFormatCount;

CHECK_XRCMD(xrEnumerateSwapchainFormats(m_session, 0, &swapchainFormatCount, nullptr));

std::vector<int64_t> swapchainFormats(swapchainFormatCount);

CHECK_XRCMD(xrEnumerateSwapchainFormats(m_session, (uint32_t)swapchainFormats.size(), &swapchainFormatCount, swapchainFormats.data()));

swapchainFormats使用vector保存data，返回image的formats

二，Creating Swapchains

进入openxr_program->CreateSwapchains()

1，获取系統信息xrGetSystemProperties

XrSystemProperties systemProperties{XR_TYPE_SYSTEM_PROPERTIES};
CHECK_XRCMD(xrGetSystemProperties(m_instance, m_systemId, &systemProperties));

2，将system view configurations配置遍历出来

uint32_t viewCount;
CHECK_XRCMD(xrEnumerateViewConfigurationViews(m_instance, m_systemId, m_viewConfigType, 0, &viewCount, nullptr));
m_configViews.resize(viewCount, {XR_TYPE_VIEW_CONFIGURATION_VIEW});
CHECK_XRCMD(xrEnumerateViewConfigurationViews(m_instance, m_systemId, m_viewConfigType, viewCount, &viewCount m_configViews.data()));

3，create a swapchain per view

正式调用XR创建swapchain，以swapchain.handle为key保存到m_swapchains

// Create a swapchain for each view.
for (uint32_t i = 0; i < viewCount; i++) {
    const XrViewConfigurationView& vp = m_configViews[i];
    ...
}

 

using an XrSwapchainCreateInfo struct for info such as width, height, face count (that is number of faces, meaning either 6 textures for a swapchain per view for cubemaps, or 1)

for (uint32_t i = 0; i < viewCount; i++) {

    // Create the swapchain.
    XrSwapchainCreateInfo swapchainCreateInfo{XR_TYPE_SWAPCHAIN_CREATE_INFO};
    swapchainCreateInfo.arraySize = 1;
    swapchainCreateInfo.format = m_colorSwapchainFormat;     //xrEnumerateSwapchainFormats枚举值

   swapchainCreateInfo.width = vp.recommendedImageRectWidth;
    swapchainCreateInfo.height = vp.recommendedImageRectHeight;
    swapchainCreateInfo.mipCount = 1;
    swapchainCreateInfo.faceCount = 1;
    swapchainCreateInfo.sampleCount = m_graphicsPlugin->GetSupportedSwapchainSampleCount(vp);
    swapchainCreateInfo.usageFlags = XR_SWAPCHAIN_USAGE_SAMPLED_BIT | XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT;
    Swapchain swapchain;
    swapchain.width = swapchainCreateInfo.width;
    swapchain.height = swapchainCreateInfo.height;
    CHECK_XRCMD(xrCreateSwapchain(m_session, &swapchainCreateInfo, &swapchain.handle));

m_swapchains.push_back(swapchain);   // For each view there is a separate swapchain

4，每个swapchain再拿SwapchainImages

为swapchain创建images（实际后续xrAcquire只调出使用一个index的image）， 一个swapchain多个image。

使用图形系统m_graphicsPlugin开辟OpenGL类型的swapchainImages 结构体数组（类似取纹理单元最大数量），调用xrEnumerateSwapchainImages 填充数组（指针vector，一般用.data()取出整个指针，这里取第0个元素作为数组指针传递），最后使用XREnum接口遍历image出来保存；

  uint32_t imageCount;
    CHECK_XRCMD(xrEnumerateSwapchainImages(swapchain.handle, 0, &imageCount, nullptr));

 

//It allocates the buffer, initializes the buffer, and returns an array of pointers to each swapchain image struct.

    std::vector<XrSwapchainImageBaseHeader*> swapchainImages =
        m_graphicsPlugin->AllocateSwapchainImageStructs(imageCount, swapchainCreateInfo);
    CHECK_XRCMD(xrEnumerateSwapchainImages(swapchain.handle, imageCount, &imageCount, swapchainImages[0]));//images array指针

    m_swapchainImages.insert(std::make_pair(swapchain.handle, std::move(swapchainImages)));

到此，结束openxr_program->CreateSwapchains()；

 

四，迭代循环Render View to Parts of Swapchain Images

进入迭代循环：

openxr_program->RenderFrame()->RenderLayer()->

物体准备：

for (XrSpace visualizedSpace : m_visualizedSpaces) { //每个物体使用一个space准备数据->cubes

最终进入

swapchain和projectionLayerViews准备：

// Render view to the appropriate part of the swapchain image.

for (uint32_t i = 0; i < viewCountOutput; i++) {

 1，xrAcquireSwapchainImage

使用acquireInfo，获取初始化时xrEnum枚举出来的image的可用index

XrSwapchainImageAcquireInfo acquireInfo{XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO};

uint32_t swapchainImageIndex;
CHECK_XRCMD(xrAcquireSwapchainImage(viewSwapchain.handle, &acquireInfo, &swapchainImageIndex));

 

2，waitInfo

等待上一帧完成，使用waitInfo（类似的有renderFrame一开始的xrWaitFrame）；

设置超时，Apps must wait on the compositor to finish reading from any image

XrSwapchainImageWaitInfo waitInfo{XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO};
waitInfo.timeout = XR_INFINITE_DURATION;
CHECK_XRCMD(xrWaitSwapchainImage(viewSwapchain.handle, &waitInfo));

 

3，projectionLayerViews

XrCompositionLayerProjection作为layer，装载LayerViews；

XrCompositionLayerProjectionView作为单眼view，装载location等projection  element信息

projectionLayerViews[i] = {XR_TYPE_COMPOSITION_LAYER_PROJECTION_VIEW};
projectionLayerViews[i].pose = m_views[i].pose;
projectionLayerViews[i].fov = m_views[i].fov;
projectionLayerViews[i].subImage.swapchain = viewSwapchain.handle;
projectionLayerViews[i].subImage.imageRect.offset = {0, 0};
projectionLayerViews[i].subImage.imageRect.extent = {viewSwapchain.width, viewSwapchain.height};

4，转入图形插件：

传入物体数据和image作为纹理

const XrSwapchainImageBaseHeader* const swapchainImage = m_swapchainImages[viewSwapchain.handle][swapchainImageIndex];
m_graphicsPlugin->RenderView(projectionLayerViews[i], swapchainImage, m_colorSwapchainFormat, cubes);

graphicplugin中，

设置帧缓存，绑定swapchainImage附件，glDrawElements；

5，releases

the swapchain image through xrReleaseSwapchainImage

XrSwapchainImageReleaseInfo releaseInfo{XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
CHECK_XRCMD(xrReleaseSwapchainImage(viewSwapchain.handle, &releaseInfo));

6，返回最后的layers汇总提交frame：

Calling xrEndFrame attempts  to submit the frame by submitting all composition layers.

openxr_program->RenderFrame() -> CHECK_XRCMD(xrEndFrame(m_session, &frameEndInfo));

五，同步提交机制

openxr使用swapchains及其image，应用无法控制显示顺序，无法控制帧数，

只能设置些帧处理时间，提交机制包括调用 xrWaitFrame, xrBeginFrame, and xrEndFrame functions

同步等待提交：xrWaitFrame

must call xrWaitFrame before rendering the frame，this is a blocking call, meaning that a subsequent xrWaitFrame call will block until the previous frame is marked as ready-to-render by calling xrBeginFrame；

设置帧开始状态：xrBeginFrame

call xrBeginFrame before the start of frame rendering and all rendering work must follow because xrBeginFrame marks the start of the rendering process for that frame.

真正的提交：xrEndFrame

Calling xrEndFrame attempts to submit the frame by submitting all composition layers.

// The wait-begin-end Cycle：

XrFrameWaitInfo frameWaitInfo{XR_TYPE_FRAME_WAIT_INFO};
XrFrameState frameState{XR_TYPE_FRAME_STATE};
CHECK_XRCMD(xrWaitFrame(m_session, &frameWaitInfo, &frameState));

XrFrameBeginInfo frameBeginInfo{XR_TYPE_FRAME_BEGIN_INFO};
CHECK_XRCMD(xrBeginFrame(m_session, &frameBeginInfo));

std::vector<XrCompositionLayerBaseHeader*> layers;
...

XrFrameEndInfo frameEndInfo{XR_TYPE_FRAME_END_INFO};
frameEndInfo.displayTime = frameState.predictedDisplayTime;
frameEndInfo.environmentBlendMode = m_environmentBlendMode;
frameEndInfo.layerCount = (uint32_t)layers.size();
frameEndInfo.layers = layers.data();
CHECK_XRCMD(xrEndFrame(m_session, &frameEndInfo));

中间图形插件进行帧绘制，颜色附件来自image，image在swapchain，projectionLayerViews[i].subImage.swapchain最后的views都交给layer.views

六，Rendering Layers

如上，apps draw frames by submitting layers，合成器连接layers和GPU来绘制frame。

1，使用xrLocateViews 找layer位置

xrLocateViews function，XrViewLocateInfo struct：

 return the viewer pose and projection parameters for rendering each view to use in a projection layer.

获取layer级别的m_views.data（pose/fov等），给到projectionLayerViews

XrViewLocateInfo viewLocateInfo{XR_TYPE_VIEW_LOCATE_INFO};

viewLocateInfo.viewConfigurationType = m_viewConfigType;

viewLocateInfo.displayTime = predictedDisplayTime;

viewLocateInfo.space = m_appSpace;

 

res = xrLocateViews(m_session, &viewLocateInfo, &viewState, viewCapacityInput, &viewCountOutput, m_views.data());

CHECK_XRRESULT(res, "xrLocateViews");

2，使用xrLocateSpace找cubes位置

Hello_xr预定义了cubes的space

// For each locatable space that we want to visualize, render a 25cm cube.
std::vector<Cube> cubes;

for (XrSpace visualizedSpace : m_visualizedSpaces) {
    XrSpaceLocation spaceLocation{XR_TYPE_SPACE_LOCATION};
    res = xrLocateSpace(visualizedSpace, m_appSpace, predictedDisplayTime, &spaceLocation);
    CHECK_XRRESULT(res, "xrLocateSpace");
    if (XR_UNQUALIFIED_SUCCESS(res)) {
        if ((spaceLocation.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) != 0 &&
            (spaceLocation.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT) != 0) {
            cubes.push_back(Cube{spaceLocation.pose, {0.25f, 0.25f, 0.25f}});
        }
    } else {
        Log::Write(Log::Level::Verbose, Fmt("Unable to locate a visualized reference space in app space: %d", res));
    }
}