Adaptive Virtual Texture （Ka Chen）

本文摘自GPU Pro 7 Apdative Virtual Texture。 By WangYuHan 2022 05.28

Introduce：

Apdative Virtual Texture(AVT) are an improvement upon Procedural Virtual Textures. This technique can be applied to a large open world and can achieve a much higher texture resolution when needed. With AVT, the artist can place thousands of projected decals with high-resolution texture on the terrain surface These decals will be baked together with terrain materials into virtual textures at runtime. Once baked,the rendering engine will directly display the virtual textures instead of rendering terrain materials and decals.Thus, the render performance is greatly improved.

Procedural Virtual Texture Basics

Procedural Virtual Texture are mimapped texture caches that store the recent rendering result of terrain material blending and projected decals. In a deferred rendering engine. these virtual texture store the composite using the G-Buffer's format ,which can then be used directly when rendering subsequent frames. It is a powerful optimization technique because the rendering engine can simply skip the expensive terrain material blending once it has been cached in the virtual texture.

Adaptive Virtual Texture

Goals

The standard virtual texture technqiue allocates a large mipmapped texture.which is then uniformly applied onto the whole terrain world.The actual texture resolution can be calculated as :

texel ratio = TextureSize.xy / WorldSize.xy

Although usually a very large virtual texture(such as 512K * 512K) is allocated, sometimes the texture resolution is not hight enough when it is apllied on a large open world such as 10KM * 10KM world.In this case the texel ratio is only 0.5 texel/cm. Such a low texture resolution limits the potential usage of prodcedural virtual texture in next-generation(下一代) games. In oreder to prevent the look of low-resolution terrain. an additional detail material layer has to be aplied on top of procedurla virtual textured terrain.

Overview

By using a 512K *512K resolution virtual texture. we would like to achieve a high texture resolution such as 10 texels/cm in a 10KM * 10KM world. in a 10KM * 10KM world.such a high texel ratio is only needed for terrain surface that may appear very close to the camera

For terrain surface that are a bit farther away , a lower texel ratio sucha as 5 texels/cm is sufficient. We would require even less texture resolution for surface that are much farther away.The Key is to find a solution to apply the virtual texture based on the distance from the camera to the terrain surface.

We divide the game world into multiple sectors and each sector has a predefined size(such as 64 * 64 meters) as shown in Figure 1.1. Every sector is allocated with a virtual image inside the virtual texture. The size of the virtual image is calculated based on the distance from the camera.The closet sectors are allocated with the maximum virtual image size, for example 64K * 64K. Farther sectors are allocated with smaller sizes, such as a 32K *32K or 16K * 16K, The minimum size can be 1K * 1k.

texel ratio = 64K / 64 meters = 10 texels / cm.

Farther sectors have lower resolution such as 5 texels/cm and so on.It is important to note that these virtual image size will be adjusted dynamically at runtime based on camera distance whenever the camera moves.

Allocate virtual Texture Atlas

As mentiond at the end of section 1.3.2 we need to adjust the size of each image dynameically at runtime. It is important to have a good texture altasing algorthm so that the performance of AVT is maximized and space fragmentation.is kept to a minimum.We use a quad-tree scheme to allocate the virtual images. This scheme make sure that the image is alsways allocated at the location where the UV address is aligned with image size.Figure 1.2 shows the allocation of three virtual images inside the virtual texture atlas.

Adjust Virtual Image Size at Runtime

In each frame we calculate a target virtual image size based on its distance from the camera.If the target size si different than the current size of the sector, we will insert a new image with the size into the atlas and remove the old one.Listings 1.1 and 1.2 show the code to calcaulte and adjust, respectively, the size of virtual image for sectors in the world.

Remap the Indirection Texture.

Indirection texture Our virtual textures uses an indirection texture to translate from the virtual UV address to the physical UV address.(For the basic virtual texture technique and information about indirection texture, please refer to [Mittring 08]). When virutal image are moved scaled, we have to update the indirection texture so that the new virtual UV address will reuse the existing pages in the physical texture caches.The format of our Indirection is 32 bit and defined as in Figure 1.4.

Figure 1.1 Calculating the virtual image size

PageOffsetX and PageOffsetY are the UV offsets of the physical page pointed to by the current virtual page.Mip describle the available mipmap level for the current virtual page.The final physical texture UV address is calculated as showns in Listiong 1.3

Remap the indirection texture when the image is up-scaled The update of the indirection texture for Sector B is shown in Figure 1.5

The work involved in updating the indirection texture is shown in Listing 1.4.

The remapping of an up-scaled virtual image can be viewed in Figure 1.6. In this case the image size is up-scale form 32K to 64K.We can conclude that the indirection entries for the new image should point to the exact same entries of one mip level higher in the old image,as shown by the red arrows.As the mip level 0 of the new image doesn't exist in the old image, we will set the entries of mip level 0 in the new image to lower mip level physical texture pages to prevent visual popping in the current frame.

Remap the indirection texture when image is down-scaled Remapping a down-scaled virtual image is just to reverse the step of up-scaling a virtual image,as shown in Figure 1.7.

Virtual Texture Best Practices

In the previous section we discussed the key features of AVT and how to handle updating the virtual texture altas and indirection texture when adjust-ments of the image size happens in real time.In this section we will talk about some practical implementation detail of AVT.

Output Page IDs

Virtual textures are divided into multiple 256 * 256 sized virtual pages.For every pixel on the screen,we calculate to which virtual page it is related and ouput the page information into a read/write buffer during the G-buffer rendering pass .We call this buffer the Page ID buffer.

The page information for every pixel is written as a 32-bit integer as shown in Figure 1.8.

PageID and Size are calculated as

PageID.xy = Virtual UV.xy / Virtual Page Size,

Size = LOG2(Virtual Page Size)

The size of the Page ID buffer is 1/8 of the G-buffer textue in both directions in order to reduce the memory overhead.Listing 1.5 is the fragment shader used to output the page ID to the small read/write buffer.

Note that g_VirtualSectorsInfo.VirtualDither.xy is the shader parameter passed in from the render engine, and its value range from 0 to 7 and changes according to a predefined pattern in every frame.Since we only pick one pixel out of 64 to output,if a page ID is missed in one frame,it will be processed during a subsequent frame.

Physical Textures and Compressing

We allocate three physical texture for deferred rendering purposes.These are the albedo texture.normal map texture,and specular texture. Table 1.1 provides the color channel information for thest texture.

Every physical texture page is compressed at runtime by a compute shader on the GPU.We modified the sample code provide By Microsoft for Xbox One to compress three different types if physical texture page in one pass. For the normal map texture.we choose the BC5 format to compress only the X and Y channels of the normal map into a separated 4 bits-per-pixel block. This give a much less blocky result than with the BC1 format.In some situations the normal vector saved in the virtual texture is not unit length.For example,when the pixel is on a big slope on the terrain surface, the final noramlvector mightbe scaled by the slope angle or a mask texture. We save the scale of the normal vector in the Z channel of the Specular Physical Texture during compression. Later on,when we fetch the virtual texture. we reapply the scale to the normal vector coming from the physcal texture.Listing 1.6 shows the HLSL shader for calculating the final G-buffer colors.

Improve Performance by Distributed Rendering

The Performance of virtual texture rendering may very depending on how many virtual pages are visible in a given frame.When the camera is moving or turn-ing very fast,it could take significant time to cache the physcial textures.We can spread the rendering of virtual pages into multiple frames to alleviate this problem. We call this method distributed rendering.

On the GPU we read the PageID bufffer that is output from the GPU, collect the visible virtual pages,and remove the duplicated pages. We then sort the visible pages according to their image sizes scaled by mipmap level, as shown in Listing 1.7.

For each sorted virtual page starting from the first page, we first search for its physcial page in the physical texture cache and allocate one if it is not already there;then, we render the fully composited terrain material into the page.At the same time we record the actual rendering time for virtual textures.If the accumulated rendering time for the current frame is longer than a threshold,we skip the rendering of the remaining page in the list.The skipped pages will be rendered during the next frame.

We always sort the virtual pages such that we render the page with the smallest image size first. This guarantees that the terrain is always displayed on screen even if some page have been skipped.Some part of the image may appear blurry in the current frame if they have been skipped, but these areas will become sharper later.once they are updated. In practice this is generally not noticeable because it happens very quicky from one frame to the next.

Virtual Texture Filtering

Anisotropic filtering. Our Adaptive Virtual Textures support 8X anisotropic filtering. This means that the pixel shader may access neghtboring pixels up to 4 pixels away If the shader accesses a pixel lying on the border of the page,its neighboring pixel could reside in another physical page and it might not be the correct neghboring pixel in the world space.This would cause color bleeding problems between pages.

To fix this problem, we allocate 4 -pixel-wide border on each side of the physical texture page.For a 256 * 256 virtual page,its physical page size becomes 264 * 264.When rendering into the physical page,the viewport is also enlarged to 264 * 264 so that the neighboring pixel at the border are rendered.(see Figure 1.9)

The linear filtering Software tr-linear filtering is also supported by simply fetching the indirection texture twice with a higher mipmap level and a lower mipmap level to get two sets of physical UV addresses, then fetching from the physical textures twice and blending between them according to the calculated ratio between the mipmap levels.

Another approach is to use hardwaretri-linear filtering with virtual textures.For every physical texture we have,we can create an additional quarter size physical texture as mipmap level 1 and render into this mipmap level 1 page.whenever the mipmap 0 page is rendered.This method required 25% more video memory for the physical texture caches. It also increase the GPU overhead because the mipmap level 1 cache must be synced with the mimap level 0 all the time.

Figure 1.10 shows a comparsion between bi-blinear filtering, anisotropic filtering and tri-linear anisotropic filtering when looking at the ground with a sharp view angle. The image looks blurred with bi-linear filtering. With anisotropic filtering alone,the image looks much sharper but there is a visible seam where mipmap levels change.With tr-linear anisotropic filtering both problem are solved.

Combine with Projected Decals

Apdptive Virtual Textures become very powerful when they are combined with projected decals on terrain.The G-buffer properties(albedo,normal,and specular) of the decals can be baked into the same virtual page where terrain surface are rendered.Thus rendering of projected decals becomes almost free with Adaptive Virtual Textures.Since AVT supports very higher virtual texture resolution (10 texels/cm), level artists can put thousands of hight-detail projected decals on the ground, and this vastly improves the visual appearance for next-generation games.

The rendering pipeline for projected decals in adaptive virtual textures is quite straight forward:

For every visible page collected and sorted from the PageID buffer,

1. find or allocate a physical texture page

2. render terrain surface material into the physical page

3.find decals belonging to this page in the world space

4.render decals into the physical page by projecting them into the virtual texture.

Figure 1.11 shows the steps to render terrain material and decals into a virtual texture page.

Conclusion

This chapter describle a new technique called Adaptive Virtual Textures for rendering terrain and decals with a high resolution in an open world. AVT is an improvement upon Procedural Virtual Textures. The main contribution of this technique is that it support a very high virtual texture resolution at close range and can be used to render thousands of decals very efficiently.

Bibliography

[Mittring 08] Martin Mittring. “Advanced Virtual Texture Topics.” In ACMSIGGRAPH 2008 Games, pp. 23–51. New York: ACM, 2008.

[Widmark 12] Mattias Widmark. “Terrain in Battlefield 3.” Paper presented atGame Developers Conference, San Francisco, CA, March 5–9, 2012.