Color Grading Playing with Colors

https://catlikecoding.com/unity/tutorials/custom-srp/color-grading/

1 color adjustments
currently we only apply tone mapping to the final image, to bring HDR colors in visible LDR range.
but this is not the only reason to adjust the colors of an image.
there are roughly three steps of color adjustments for video, photos, and digital images.
first comes color correction, which aims to make the image match what we would see if we observed the scene, compensating for the limitations of the medium.

second comes color grading, which is about achieving a desired look or feel that does not match the original scene and need not be realistic.
these two steps are often merged into one color grading step.
after that comes tone mapping, to map HDR colors to the display range.

with only tone mapping applied the image tends to become less colorful except when it is very bright.
ACESincreases the contrast of dark colors a bit, but it is no substitute for color grading. this tutorial uses neutral tone mapping as a basis.

1.1 Color Grading Before Tone Mapping

Color grading happens before tone mapping. Add a function for it to PostFXStackPasses, before the tone mapping passes. Initially only have it limit the color components to 60.

float3 ColorGrade (float3 color) 
{
	color = min(color, 60.0);
	return color;
}

Invoke this function in the tone mapping passes instead of limiting the color there. Also add a new pass for no tone mapping, but with color grading.

float4 ToneMappingNonePassFragment (Varyings input) : SV_TARGET {
	float4 color = GetSource(input.fxUV);
	color.rgb = ColorGrade(color.rgb);
	return color;
}

float4 ToneMappingACESPassFragment (Varyings input) : SV_TARGET {
	float4 color = GetSource(input.fxUV);
	color.rgb = ColorGrade(color.rgb);
	color.rgb = AcesTonemap(unity_to_ACES(color.rgb));
	return color;
}

float4 ToneMappingNeutralPassFragment (Varyings input) : SV_TARGET {
	float4 color = GetSource(input.fxUV);
	color.rgb = ColorGrade(color.rgb);
	color.rgb = NeutralTonemap(color.rgb);
	return color;
}

float4 ToneMappingReinhardPassFragment (Varyings input) : SV_TARGET {
	float4 color = GetSource(input.fxUV);
	color.rgb = ColorGrade(color.rgb);
	color.rgb /= color.rgb + 1.0;
	return color;
}

1.2 Settings

We’re going to copy the functionality of the Color Adjustments post-processing tool of URP and HDRP. The first step is to add a configuration struct for it to PostFXSettings. I added using System because we’ll need to add the Serializable attribute a bunch more times.

using System;
using UnityEngine;

[CreateAssetMenu(menuName = "Rendering/Custom Post FX Settings")]
public class PostFXSettings : ScriptableObject 
{
	…

	[Serializable]
	public struct ColorAdjustmentsSettings {}

	[SerializeField]
	ColorAdjustmentsSettings colorAdjustments = default;

	public ColorAdjustmentsSettings ColorAdjustments => colorAdjustments;

	…
}

The color grading functionality of URP and HDRP is identical. We’ll add the same configuration options for color grading, in the same order. First is Post Exposure, an unconstrained float. After that comes Contrast, a slider going from −100 to 100. The next option is Color Filter, which is an HDR color without alpha. Next up is Hue Shift, another slider but going from −180° to +180°. The last option is Saturation, again a slider from −100 to 100.

	public struct ColorAdjustmentsSettings {

		public float postExposure;

		[Range(-100f, 100f)]
		public float contrast;

		[ColorUsage(false, true)]
		public Color colorFilter;

		[Range(-180f, 180f)]
		public float hueShift;

		[Range(-100f, 100f)]
		public float saturation;
	}

The default values are all zero, except the color filter should be white. These settings won’t change the image.

ColorAdjustmentsSettings colorAdjustments = new ColorAdjustmentsSettings {colorFilter = Color.white};

we are doing color grading and tone mapping at the same time, so refactor rename PostFXStack.DoToneMapping to DoColorGradingAndToneMapping. we will also be accessing the inner types of PostFXSettings a lot here, so let us add using static PostFXSettings to keep the code shorter. then add a ConfigureColorAdjustments method in which we grab the color adjustment settings and invoke it at the start of DoColorGradingAndToneMapping.

using UnityEngine;
using UnityEngine.Rendering;
using static PostFXSettings;

public partial class PostFXStack 
{	
	…	
	void ConfigureColorAdjustments () {
		ColorAdjustmentsSettings colorAdjustments = settings.ColorAdjustments;
	}

	void DoColorGradingAndToneMapping (int sourceId) {
		ConfigureColorAdjustments();

		ToneMappingSettings.Mode mode = settings.ToneMapping.mode;
		Pass pass = Pass.ToneMappingNone + (int)mode;
		Draw(sourceId, BuiltinRenderTextureType.CameraTarget, pass);
	}
	
	…
}

What does using static do?
It’s similar to using a namespace, but for a type. It makes all constant, static, and type members of a class or struct directly accessible without fully qualifying them.

we can suffice with setting a shader vector and color for the color adjustments. the color adjustments vector components are the exposure曝光度, contrast对比度, hue shift色调调整, and saturation饱和度.
exposure is measured in stops, which means that we have to raise 2 to the power of the configured exposure value.
also convert contrast and saturation to the 0-2 range and
hue shift to -1—1.
the filter must be in linear color space.

ColorAdjustmentsSettings colorAdjustments = settings.ColorAdjustments;
		buffer.SetGlobalVector(colorAdjustmentsId, new Vector4(
			Mathf.Pow(2f, colorAdjustments.postExposure),
			colorAdjustments.contrast * 0.01f + 1f,
			colorAdjustments.hueShift * (1f / 360f),
			colorAdjustments.saturation * 0.01f + 1f
		));
		buffer.SetGlobalColor(colorFilterId, colorAdjustments.colorFilter.linear);

1.3 post exposure 后曝光
on the shader side, add the vector and color. we will put every adjustment in its own function and start with the post exposure. create a ColorGradePostExposure function that multiplies the color with the exposure value. then apply exposure in ColorGrade after limiting the color.

float4 _ColorAdjustments;
float4 _ColorFilter;

float3 ColorGradePostExposure (float3 color) {
	return color * _ColorAdjustments.x; //xyzw 曝光度、对比度、色调、饱和度
}

float3 ColorGrade (float3 color) {
	color = min(color, 60.0);
	color =
	ColorGradePostExposure(color);
	return color;
}

the idea of post-exposure is that it mimics a camera’s exposure, but is applied after all other posteffects, immediately before all other color grading. 在所有后处理之后，但是在所有校色之前。

it is a nonrealistic artistic tool that can be used to tweak exposure without influencing other effects, like bloom.

1.4 contrast 对比度
the second adjustment is constrast. we apply it by substracting uniform mid gray from the color, then scaling by the contrast, and adding mid gray to that. use ACEScc_MIDGRAY for the gray color.

What’s ACEScc?
ACEScc is a logarithmic subset of ACES color space. The mid gray value is 0.4135884.

float3 ColorGradingContrast (float3 color) {
	return (color - ACEScc_MIDGRAY) * _ColorAdjustments.y + ACEScc_MIDGRAY;
}

float3 ColorGrade (float3 color) {
	color = min(color, 60.0);
	color = ColorGradePostExposure(color);
	color = ColorGradingContrast(color);
	return color;
}

for best results this conversion is done in Log C instead of linear color space.
we can convert from linear to Log C with the LinearToLogC function from the Color Core Library file and back with the LogCToLinear function.

float3 ColorGradingContrast (float3 color) 
{
	color = LinearToLogC(color);
	color = (color - ACEScc_MIDGRAY) * _ColorAdjustments.y + ACEScc_MIDGRAY;
	return LogCToLinear(color);
}

Linear and Log C.

when contrast is increased this can lead to negative color components, which can mess up later adjustments. 后面的调整会乱掉
so eliminate negative values after adjusting contrast in ColorGrade.

color = ColorGradingContrast(color);
color = max(color, 0.0);

1.5 color filter 颜色过滤，滤镜
the color filter comes next, simply multiply it with the color. it works fine with for negative values, so we can apply it before eliminating them. 颜色滤镜其原则就是用一个颜色去乘

float3 ColorGradeColorFilter (float3 color) {
	return color * _ColorFilter.rgb;
}

float3 ColorGrade (float3 color) {
	color = min(color, 60.0);
	color = ColorGradePostExposure(color);
	color = ColorGradingContrast(color);
	color = ColorGradeColorFilter(color);
	color = max(color, 0.0);
	return color;
}

1.6 Hue Shift
颜色色调的调整，只是对hsv的h进行调整，所以要将rgb转化为hsv以后在对其h进行缩放处理。然后再将调整之后的hsv转化为rgb，这就是hue shift的过程。
可以参考：https://zhuanlan.zhihu.com/p/67930839
URP and HDRP perform the hue shift after the color filter and we will use the same adjustment order. 色调的调整在颜色滤镜之后处理
the color’s hue is adjusted by converting the color format from RGB to HSV via RgbToHsv, adding the hue shift to H, and converting back via HsvToRgb.
because hue is defined on a 0-1 color wheel we have to wrap it around if it goes out of range.
we can use RotateHue for that, passing it the adjusted hue, zero, and 1 as arguments. this must happen after negative values are eliminated.

float3 ColorGradingHueShift (float3 color) {
	color = RgbToHsv(color);
	float hue = color.x + _ColorAdjustments.z;
	color.x = RotateHue(hue, 0.0, 1.0);
	return HsvToRgb(color);
}

float3 ColorGrade (float3 color) {
	color = min(color, 60.0);
	color = ColorGradePostExposure(color);
	color = ColorGradingContrast(color);
	color = ColorGradeColorFilter(color);
	color = max(color, 0.0);
	color = ColorGradingHueShift(color);
	return color;
}

1.7 saturation
the last adjustment is saturation.
first get the color’s luminance with the help of the Luminance function.
the result is then calculated like contrast, except with luminance instead of mid gray and not in Log C.
this can again produce negative values, so remove those from the final result of ColorGrade.

饱和度的调整类似于对比度的调整，但是是先减去灰度值，再缩放，最后再加上灰度值。

float3 ColorGradingSaturation (float3 color) {
	float luminance = Luminance(color);
	return (color - luminance) * _ColorAdjustments.w + luminance;
}

float3 ColorGrade (float3 color) {
	color = min(color, 60.0);
	color = ColorGradePostExposure(color);
	color = ColorGradingContrast(color);
	color = ColorGradeColorFilter(color);
	color = max(color, 0.0);
	color = ColorGradingHueShift(color);
	color = ColorGradingSaturation(color);
	return max(color, 0.0);
}

2 more controls
the color adjustments tool is not the only color grading option offered by URP and HDRP.
we will add support for a few more, once again copying unity’s approach.

2.1 white balance
the white balance tool makes it possible to adjust the perceived temperature of the image.
it has two sliders for the -100~100 range.
the first is temperature, for making the image cooler or warmer.
the second is Tint, used for tweaking the temperature-shifted color.
add a settings struct for it to PostFXSettings, with zeros as defaults.

	[Serializable]
	public struct WhiteBalanceSettings 
	{

		[Range(-100f, 100f)]
		public float temperature, tint;
	}
	[SerializeField]
	WhiteBalanceSettings whiteBalance = default;
	public WhiteBalanceSettings WhiteBalance => whiteBalance;

White balance settings.

3 LUT
performing all color grading steps per pixel is a lot of work.
we could make may variants that only apply the steps that alter something, but that would requires lots of keywords or passes.
what we can do instead is bake color grading into a lookup table——LUT for short——and sample it to convert colors.
the LUT is a 3D texture, typically 32x32x32.
filling that texture and sampling it later is much less work than performing color grading directly on the entire image. URP and HDRP use the same approach.

3.1 LUT resolution
typically a color LUT resolution of 32 is enough, but let us make it configurable.
this is a quality setting that we will add to CustomRenderPipelineAsset and then use for all color grading.
we will use an enum to offer 16, 32 and 64 as options, then pass it to the pipeline constructor as an integer.

public enum ColorLUTResolution { _16 = 16, _32 = 32, _64 = 64 }

	[SerializeField]
	ColorLUTResolution colorLUTResolution = ColorLUTResolution._32;

	protected override RenderPipeline CreatePipeline () {
		return new CustomRenderPipeline(
			allowHDR, useDynamicBatching, useGPUInstancing, useSRPBatcher,
			useLightsPerObject, shadows, postFXSettings, (int)colorLUTResolution
		);
	}

Keep track of the color LUT resolution in CustomRenderPipeline and pass it to the CameraRenderer.Render method.

int colorLUTResolution;

	public CustomRenderPipeline (
		…
		PostFXSettings postFXSettings, int colorLUTResolution
	) {
		this.colorLUTResolution = colorLUTResolution;
		…
	}

	protected override void Render (ScriptableRenderContext context, Camera[] cameras) {
		foreach (Camera camera in cameras) {
			renderer.Render(
				context, camera, allowHDR,
				useDynamicBatching, useGPUInstancing, useLightsPerObject,
				shadowSettings, postFXSettings, colorLUTResolution
			);
		}
	}

which passes it to PostFXStack.Setup.

public void Render (
		ScriptableRenderContext context, Camera camera, bool allowHDR,
		bool useDynamicBatching, bool useGPUInstancing, bool useLightsPerObject,
		ShadowSettings shadowSettings, PostFXSettings postFXSettings,
		int colorLUTResolution
	) {
		…
		postFXStack.Setup(context, camera, postFXSettings, useHDR, colorLUTResolution);
		…
	}

And PostFXStack keeps track of it.

int colorLUTResolution;

	…

	public void Setup (
		ScriptableRenderContext context, Camera camera, PostFXSettings settings,
		bool useHDR, int colorLUTResolution
	) {
		this.colorLUTResolution = colorLUTResolution;
		…
	}

3.2 rendering to a 2D lut texture

the LUT is 3D, but a regular shader can not render to a 3D texture.
so we will use a wide 2d texture instead to simulate a 3D texture,
by placing 2D slices in a row.
thus the LUT texture’s height is equal to the configured resolution and its width is equal to the resolution squared.
get a temporary render texture with that size, using the default HDR format.
do this after configuring color grading in DoColorGradingAndToneMapping.

ConfigureShadowsMidtonesHighlights();

		int lutHeight = colorLUTResolution;
		int lutWidth = lutHeight * lutHeight;
		buffer.GetTemporaryRT(
			colorGradingLUTId, lutWidth, lutHeight, 0,
			FilterMode.Bilinear, RenderTextureFormat.DefaultHDR
		);

from now on we will render both color grading and tone mapping to the LUT.
rename the existing tone-mapping passes accordingly, so ToneMappingNone becomes ColorGradingNone and so on. then draw to the LUT instead of the camera target, using the appropriate pass.
afterwards copy the source to the camera target to get the unadjusted image as the final result and release the LUT.

ToneMappingSettings.Mode mode = settings.ToneMapping.mode;
Pass pass = Pass.ColorGradingNone + (int)mode;
Draw(sourceId, colorGradingLUTId, pass);
Draw(sourceId, BuiltinRenderTextureType.CameraTarget, Pass.Copy);
buffer.ReleaseTemporaryRT(colorGradingLUTId);

we are now bypassing color grading and tone mapping, but the frame debugger reveals that we draw a flattened version of the image before the final copy.

3.3 LUT color matrix

to create an appropriate LUT we need to fill it with a color conversion matrix.
we do that by adjusting the color grading pass functions to use a color derived from the UV coordinates instead of sampling from the source texture.
add a GetColorGradedLUT, which gets the color and also immediately performs color grading.
the the pass functions only have to apply tone mapping on top of that.

float3 GetColorGradedLUT (float2 uv, bool useACES = false) {
	float3 color = float3(uv, 0.0);
	return ColorGrade(color, useACES);
}

float4 ColorGradingNonePassFragment (Varyings input) : SV_TARGET {
	float3 color = GetColorGradedLUT(input.fxUV);
	return float4(color, 1.0);
}

float4 ColorGradingACESPassFragment (Varyings input) : SV_TARGET {
	float3 color = GetColorGradedLUT(input.fxUV, true);
	color = AcesTonemap(color);
	return float4(color, 1.0);
}

float4 ColorGradingNeutralPassFragment (Varyings input) : SV_TARGET {
	float3 color = GetColorGradedLUT(input.fxUV);
	color = NeutralTonemap(color);
	return float4(color, 1.0);
}

float4 ColorGradingReinhardPassFragment (Varyings input) : SV_TARGET {
	float3 color = GetColorGradedLUT(input.fxUV);
	color /= color + 1.0;
	return float4(color, 1.0);
}

在执行tone mapping之前进行校色

we can find the LUT input color via the GetLutStripValue function. it requires the UV coordinates and a color grading lut parameters vector that we need to send to the GPU.

float4 _ColorGradingLUTParameters;

float3 GetColorGradedLUT (float2 uv, bool useACES = false) {
	float3 color = GetLutStripValue(uv, _ColorGradingLUTParameters);
	return ColorGrade(color, useACES);
}

the four vector parameter values are the LUT height, 0.5 divided by the width, 0.5 divided by the height, and the height divided by itself minus one.

buffer.GetTemporaryRT(
			colorGradingLUTId, lutWidth, lutHeight, 0,
			FilterMode.Bilinear, RenderTextureFormat.DefaultHDR
		);
		buffer.SetGlobalVector(colorGradingLUTParametersId, new Vector4(
			lutHeight, 0.5f / lutWidth, 0.5f / lutHeight, lutHeight / (lutHeight - 1f)
		));

3.4 Log C LUT
the LUT matrix that we get is in linear color space and only covers the 0-1 range.
to support HDR we have to extend this range. we can do this by interpreting the input color
as being in Log C space. that extends the range to just below 59.

Stored linear and Log C intensities.

float3 GetColorGradedLUT (float2 uv, bool useACES = false) {
	float3 color = GetLutStripValue(uv, _ColorGradingLUTParameters);
	return ColorGrade(LogCToLinear(color), useACES);
}