Compressed Texture Formats in Metal
http://metalbyexample.com/compressed-textures/
In this article, we will consider several GPU-friendly compressed texture formats. These formats allow us to trade some image quality for substantial improvements in disk usage and performance. In particular, we will look at the ETC2, PVRTC, and ASTC formats.
The sample app showcases a variety of compressed texture formats
Download the sample code for this article here.
Why Use Compressed Textures?
In previous articles, we created textures by loading an image into memory, drawing it into a bitmap context, and copying the bits into a Metal texture. These textures used uncompressed pixel formats such as MTLPixelFormatRGBA8Unorm.
Metal also supports several compressed formats. These formats allow us to copy image data directly into a texture without decompressing it. These formats are designed to be decompressed on-demand by the GPU, when the texture is sampled rather than when it is loaded.
The formats we will consider below are lossy, meaning they do not perfectly preserve the image contents, as a lossless format like PNG does. Instead, they trade a reduction in image quality for much smaller memory usage. Using compressed texture data therefore greatly reduces the memory bandwidth consumed when sampling textures, and is also more cache-friendly.
A Brief Overview of Compressed Texture Formats
S3TC
S3TC, also known as DXT, was the first compressed format to gain broad adoption, due to its inclusion in DirectX 6.0 and OpenGL 1.3 in 1998 and 2001, respectively. Although S3TC is broadly supported on desktop GPUs, it is not available on iOS devices.
PVRTC
The PVRTC image format was introduced by Imagination Technologies, creators of the PowerVR series of GPUs at the heart of every iOS device. It was first described in a 2003 paper by Simon Fenney.
PVRTC operates by downsampling the source image into two smaller images, which are upscaled and blended to reconstruct an approximation of the original. It considers blocks of 4×4 or 4×8 pixels at a time, which are packed into one 64-bit quantity. Thus, each pixel occupies 4 bits or 2 bits, respectively.
One significant limitation of using PVRTC format on iOS is that textures must be square, and each dimension must be a power of two. Fortunately, game textures are most commonly produced in dimensions that are compatible with this limitation.
ETC
Ericsson Texture Compression (ETC) debuted in 2005. Similarly to PVRTC’s 4-bit-per-pixel mode, it compresses each 4×4 block of pixels into a single 64-bit quantity, but lacks support for an alpha channel. A subsequent version, ETC2, adds support for 1-bit and 8-bit alpha channels. ETC2 is supported by all Metal-compatible hardware, but PVRTC often offers better quality at comparable file sizes.
ASTC
Advanced Scalable Texture Compression (ASTC) is the most recent compressed texture format supported by Metal. Developed by AMD and supported fairly widely on OpenGL ES 3.0-class hardware, this format incorporates a selectable block size (from 4×4 to 12×12) that determines the compression ratio of the image. This unprecedented flexibility makes ASTC a very attractive choice, but it requires an A8 processor at a minimum, making it unusable on devices such as the iPhone 5s.
Container Formats
The compression format used by a texture is only half of the picture. In order to load a texture from disk, we need to know its dimensions (width and height) and its pixel format. This metadata is often written into the file itself, in the form of a header that precedes the image data. The layout of the header is described by the container format of the image. In this article, we will use two container formats: PVR and ASTC.
The PVR Container Format
Not to be confused with the PVRTC compression format, the PVR container format describes how the data in a texture file should be interpreted. A PVR file can contain uncompressed data or compressed data in numerous formats (including S3TC, ETC, and PVRTC).
There are a few different, mutually-incompatible versions of the PVR format. This means that each version of PVR requires different code to parse correctly. For example, the header structure for the “legacy” format (PVRv2) looks like this:
|
|
The header structure for the most recent version (PVRv3) looks like this:
|
|
There are more similarities than differences between the headers. The various fields describe the contained data, including the texture’s dimensions and whether the file contains a set of mipmaps or a single image. If the file contains mipmaps, they are simply concatenated together with no padding. The program reading the file is responsible for calculating the expected length of each image.
Note that the header also has surfaceCount and (in the case of PVRv3) faceCount fields, which can be used when writing texture arrays or cube maps into a single file. We will not use these fields in this article.
The ASTC Container Format
The ASTC container format is custom-tailored to the ASTC compression format. The header indicates the compression block size, which allows you to select the correct pixel format when loading the data into texture memory and calculate the length of the texture data. Here is the layout of the ASTC header:
|
|
The xSize, ySize, and zSize fields are 24-bit quantities, each encoded as three bytes. The compressed image data immediately follows the header.
Creating Compressed Textures
There are various tools for creating compressed textures, both on the command line and with a GUI. Below, we’ll look at two tools, which allow us to create textures in the formats discussed above.
Creating Textures with texturetool
Apple includes a command-line utility with Xcode called texturetool that can convert images into compressed texture formats such as PVRTC and ASTC. At the time of this writing, it does not support the PVRv3 container format. Rather, it writes PVR files using the legacy (PVRv2) format, described above. With the -e flag, you specify your desired container format: PVR, ASTC, or RAW (no header). With the -f format you specify the compression type: PVRTC or ASTC. The -m flag signifies that all mipmaps should be generated and written sequentially into the file.
Each compression format has its own flags for selecting the degree of compression. For example, to create a 4 bit-per-pixel PVRTC texture with a PVR legacy header, including all mipmap levels, use the following invocation of texturetool:
texturetool -m -e PVRTC -f PVR --bits-per-pixel-4 -o output.pvr input.png
To create an ASTC-compressed texture in an ASTC container with an 8×8 block size using the “thorough” compression mode:
texturetool -e ASTC -f ASTC --block-width-8 --block-height-8 --compression-mode-thorough -o output.astc input.png
The -m flag for generating mipmaps is ignored when writing a texture with an ASTC container.
Creating Textures with PVRTexToolGUI
Imagination Technologies includes a tool in their PowerVR SDK that allows creation of compressed textures with a GUI. PVRTexToolGUI allows you to import a PNG and select a variety of compression formats, including ASTC, PVRTC, and ETC2. Unlike texturetool, this application uses the PVRv3 container format by default, so you should use code that expects the appropriate header format if you compress textures with this tool.
The PVRTexToolGUI from Imagination Technologies allows the user to compress images in a variety of formats
PVRTexToolGUI has a command-line counterpart, PVRTexToolCLI, that exposes all of the parameters available in the GUI and allows you to write scripts for rapidly batch-converting textures.
Loading Compressed Texture Data into Metal
Loading compressed textures is a two-fold process. First, we read the header of the container format. Then, we read the image data described by the header and hand it to Metal. Regardless of the compression algorithm used, we do not need to do any parsing of the image data ourselves, since Metal is able to understand and decode compressed textures in hardware.
Creating a texture with compressed data is exactly the same as with uncompressed data. First, we create a texture descriptor with the appropriate pixel format and dimensions. If the texture has mipmaps, we indicate so with the mipmapped parameter. For example, here is how we create a texture descriptor for a PVRTC texture with 4 bits per pixel for which we have mipmaps:
|
|
We can then ask a device to produce a texture that matches this descriptor:
|
|
For each mipmap level in the image, we need to make a separate call to -replaceRegion:mipmapLevel:withBytes:bytesPerRow:, as follows:
|
|
Where levelWidth and levelHeight are the dimensions of the current mip level, level is the index of the current mip level (the base level is index 0), and levelData is a pointer to the mip level’s image data. levelBytesPerRow can be computed as the length of the image data divided by the level’s height.
The Sample App
The sample app includes a demonstration of all of the texture formats mentioned in this article. Tapping the “Switch Textures” button brings up a menu that allows the user to select among the various textures. The menu options for the ASTC options will not be available if the device on which the app is running does not support ASTC compression.
The sample app allows you to select among several compression formats supported by Metal
Download the sample code for this article here.
Conclusion
In this article, we have discussed the merits of compressed textures and discussed a few of the available formats. We looked at how to create and load textures in the ETC2, PVRTC, and ASTC formats. Generally speaking, ASTC offers the best quality-to-size ratio on hardware where it is available. The PVRTC format should generally be preferred when targeting devices with the A7 processor.
Acknowledgements
Conversations with Shannon Potter provoked my interest in this topic and encouraged me to write this article.
Share this:
- Click to email this to a friend (Opens in new window)
- Click to print (Opens in new window)
- Click to share on Facebook (Opens in new window)
- Click to share on Twitter (Opens in new window)
Categories Intermediate, TexturesPost navigation
Rendering Text in Metal with Signed-Distance Fields
13 thoughts on “Compressed Texture Formats in Metal”
Andy
November 17, 2016 at 3:28 am | Reply
Can you share the swift variant?
Warren Moore
November 27, 2016 at 2:51 pm | Reply
I never ported this code to Swift, nor am I aware of anyone doing so, but it should be pretty straightforward.
Cheney
December 14, 2016 at 12:41 am | Reply
Hi，Is there a method in the metal as same as glViewport in OpenGL, set the viewport ,to achieve split-screen？
Warren Moore
December 17, 2016 at 5:13 pm | Reply
You can use the
setViewport:method on yourMTLRenderCommandEncoderto set the viewport (NDC-to-window transform). It takes aMTLViewportstruct.
Sarbartha
January 8, 2017 at 6:58 am | Reply
can you post the swift version too?
Warren Moore
January 8, 2017 at 7:29 pm | Reply
I don’t generally post Swift ports, since the actual API usage is the same between Objective-C and Swift. Having said that, some of the lower-level parts of this particular example might be a little tricky, due to Swift’s lack of specificity around alignment. In any case, my friend Marius’ post on textures covers how to load textures with
MTKTextureLoaderin Swift, which is the recommended approach these days. It can handle all of the formats mentioned here and more.
May 23, 2017 at 10:53 am | Reply
Since they share the same storage, any changes to the pixels of the new texture object are reflected in the calling texture object, and vice versa.
Peter Molnar
November 14, 2017 at 4:58 pm | Reply
Hello.
How can I load PNG image as compressed texture on Mac? I was expecting the same behavior as in OpenGL but when I changed PixelFormat to BC1 it was not working correctly.Warren Moore
November 15, 2017 at 4:42 pm | Reply
I’m not familiar with how OpenGL handles BCn textures, but I wouldn’t expect that loading RGBA data straight into a BC1 texture in Metal would work. You’d need to compress it offline with a tool like img2ktx, then load that data.
icalic
November 22, 2017 at 1:24 am | Reply
Hello Warren,
is it possible in metal programming to compute peak GFLOPS on ios and mac? something like clpeak(https://github.com/krrishnarraj/clpeak) on opencl, i curious about peak GFLOPS latest chip iphone and ipad(a11 and 10x).
latest snapdragon 835 gpu on oneplus 5:
Global memory bandwidth (GBPS)
float : 17.82
float2 : 19.30
float4 : 19.43
float8 : 20.38
float16 : 20.45Single-precision compute (GFLOPS)
float : 294.65
float2 : 285.81
float4 : 311.02
float8 : 265.02
float16 : 308.34half-precision compute (GFLOPS)
half : 570.72
half2 : 539.62
half4 : 610.79
half8 : 314.82
half16 : 313.73No double precision support! Skipped
Integer compute (GIOPS)
int : 65.98
int2 : 68.05
int4 : 80.68
int8 : 79.25
int16 : 77.79Transfer bandwidth (GBPS)
enqueueWriteBuffer : 8.71
enqueueReadBuffer : 8.98
enqueueMapBuffer(for read) : 3228.33
memcpy from mapped ptr : 8.99
enqueueUnmap(after write) : 1016.22
memcpy to mapped ptr : 8.97Kernel launch latency : 214.91 us
source: https://forum.beyond3d.com/threads/clpeak-compute-gflops-with-opencl-on-android.60374/#post-2011570Volodymyr
March 26, 2018 at 9:11 am | Reply
why ASTC no need mipmaps?
Warren Moore
March 26, 2018 at 9:35 am | Reply
It’s not that ASTC doesn’t need mipmaps; rather, that
texturetooldoesn’t allow the creation of mipmap chains for ASTC images. It’s a limitation of the tool.
December 18, 2018 at 5:34 pm | Reply
Looks like ETC2 loading was broken for NPOT textures. Assertion was failing and texture was corrupt.
This line fixed it for me
uint32_t widthInBlocks = (int)ceilf((float)width / blockWidth);
/…….
(int)ceilf((float)width / blockWidth)
1209
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
