Python+OpenCV：高动态范围（High Dynamic Range, HDR）

Python+OpenCV：高动态范围（High Dynamic Range, HDR）

目标

• Learn how to generate and display HDR image from an exposure sequence.
• Use exposure fusion to merge an exposure sequence.

理论

High-dynamic-range imaging (HDRI or HDR) is a technique used in imaging and photography to reproduce a greater dynamic range of luminosity than is possible with standard digital imaging or photographic techniques.

While the human eye can adjust to a wide range of light conditions, most imaging devices use 8-bits per channel, so we are limited to only 256 levels.

When we take photographs of a real world scene, bright regions may be overexposed, while the dark ones may be underexposed, so we can’t capture all details using a single exposure.

HDR imaging works with images that use more than 8 bits per channel (usually 32-bit float values), allowing much wider dynamic range.

There are different ways to obtain HDR images, but the most common one is to use photographs of the scene taken with different exposure values.

To combine these exposures it is useful to know your camera’s response function and there are algorithms to estimate it.

After the HDR image has been merged, it has to be converted back to 8-bit to view it on usual displays. This process is called tonemapping.

Additional complexities arise when objects of the scene or camera move between shots, since images with different exposures should be registered and aligned.

In this tutorial we show 2 algorithms (Debevec, Robertson) to generate and display HDR image from an exposure sequence, and demonstrate an alternative approach called exposure fusion (Mertens),

that produces low dynamic range image and does not need the exposure times data.

Furthermore, we estimate the camera response function (CRF) which is of great value for many computer vision algorithms.

Each step of HDR pipeline can be implemented using different algorithms and parameters, so take a look at the reference manual to see them all.

Exposure sequence HDR in OpenCV

####################################################################################################
# 图像高动态范围(High Dynamic Range, HDR)
def lmc_cv_hdr_exposure_sequence():
    """
        函数功能: 图像高动态范围(High Dynamic Range, HDR).
    """

    # Loading exposure images into a list
    stacking_images = []
    images = glob.glob('D:/99-Research/TestData/cv/hdr/exposures/memorial*.png')
    image_list = [lmc_cv.imread(image_name) for image_name in images]
    image_list = [lmc_cv.cvtColor(image, lmc_cv.COLOR_BGR2RGB) for image in image_list]
    exposure_times = np.array([1024, 512, 256, 128, 64, 32, 16, 8, 4, 2, 1, 0.5, 0.25, 0.125, 0.0625, 0.03125],
                              dtype=np.float32)
    # exposure_times = 1.00/exposure_times

    # Merge exposures to HDR image
    merge_debevec = lmc_cv.createMergeDebevec()
    hdr_debevec = merge_debevec.process(image_list, times=exposure_times.copy())
    merge_robertson = lmc_cv.createMergeRobertson()
    hdr_robertson = merge_robertson.process(image_list, times=exposure_times.copy())

    # Tonemap HDR image
    tonemap1 = lmc_cv.createTonemap(gamma=15.0)
    res_debevec = tonemap1.process(hdr_debevec.copy())
    tonemap2 = lmc_cv.createTonemap(gamma=15.0)
    res_robertson = tonemap2.process(hdr_robertson.copy())

    # Exposure fusion using Mertens
    merge_mertens = lmc_cv.createMergeMertens()
    res_mertens = merge_mertens.process(image_list)

    # Convert datatype to 8-bit and save
    res_debevec_8bit = np.clip(res_debevec * 255, 0, 255).astype('uint8')
    res_robertson_8bit = np.clip(res_robertson * 255, 0, 255).astype('uint8')
    res_mertens_8bit = np.clip(res_mertens * 255, 0, 255).astype('uint8')

    # 显示结果
    images = [image_list[0], res_debevec_8bit, res_robertson_8bit, res_mertens_8bit]
    titles = ['Original Image', 'Debevec', 'Robertson', 'Mertenes Fusion']
    pyplot.figure('Image Inpainting %d' % (0 + 1), figsize=(12, 9))
    for i in range(len(images)):
        pyplot.subplot(2, 2, i + 1)
        pyplot.imshow(images[i], 'gray')
        pyplot.title(titles[i])
        pyplot.xticks([])
        pyplot.yticks([])
        pyplot.savefig('High Dynamic Range.png')
    pyplot.show()

Estimating Camera Response Function

The camera response function (CRF) gives us the connection between the scene radiance to the measured intensity values.

The CRF if of great importance in some computer vision algorithms, including HDR algorithms. Here we estimate the inverse camera response function and use it for the HDR merge.

####################################################################################################
# 图像高动态范围(High Dynamic Range, HDR)
def lmc_cv_hdr_exposure_sequence():
    """
        函数功能: 图像高动态范围(High Dynamic Range, HDR).
    """

    # Loading exposure images into a list
    stacking_images = []
    images = glob.glob('D:/99-Research/TestData/cv/hdr/exposures/memorial*.png')
    image_list = [lmc_cv.imread(image_name) for image_name in images]
    image_list = [lmc_cv.cvtColor(image, lmc_cv.COLOR_BGR2RGB) for image in image_list]
    exposure_times = np.array([1024, 512, 256, 128, 64, 32, 16, 8, 4, 2, 1, 0.5, 0.25, 0.125, 0.0625, 0.03125],
                              dtype=np.float32)
    # exposure_times = 1.00/exposure_times

    # Merge exposures to HDR image
    merge_debevec = lmc_cv.createMergeDebevec()
    hdr_debevec = merge_debevec.process(image_list, times=exposure_times.copy())
    merge_robertson = lmc_cv.createMergeRobertson()
    hdr_robertson = merge_robertson.process(image_list, times=exposure_times.copy())

    # Tonemap HDR image
    tonemap1 = lmc_cv.createTonemap(gamma=15.0)
    res_debevec = tonemap1.process(hdr_debevec.copy())
    tonemap2 = lmc_cv.createTonemap(gamma=15.0)
    res_robertson = tonemap2.process(hdr_robertson.copy())

    # Exposure fusion using Mertens
    merge_mertens = lmc_cv.createMergeMertens()
    res_mertens = merge_mertens.process(image_list)

    # Convert datatype to 8-bit and save
    res_debevec_8bit = np.clip(res_debevec * 255, 0, 255).astype('uint8')
    res_robertson_8bit = np.clip(res_robertson * 255, 0, 255).astype('uint8')
    res_mertens_8bit = np.clip(res_mertens * 255, 0, 255).astype('uint8')

    # 显示结果
    images = [image_list[0], res_debevec_8bit, res_robertson_8bit, res_mertens_8bit]
    titles = ['Original Image', 'Debevec', 'Robertson', 'Mertenes Fusion']
    pyplot.figure('Image Inpainting %d' % (0 + 1), figsize=(12, 9))
    for i in range(len(images)):
        pyplot.subplot(2, 2, i + 1)
        pyplot.imshow(images[i], 'gray')
        pyplot.title(titles[i])
        pyplot.xticks([])
        pyplot.yticks([])
        pyplot.savefig('E:/WHL/High Dynamic Range.png')
    pyplot.show()

    # Estimate camera response function (CRF)
    cal_debevec = lmc_cv.createCalibrateDebevec()
    crf_debevec = cal_debevec.process(image_list, times=exposure_times)
    hdr_debevec = merge_debevec.process(image_list, times=exposure_times.copy(), response=crf_debevec.copy())
    cal_robertson = lmc_cv.createCalibrateRobertson()
    crf_robertson = cal_robertson.process(image_list, times=exposure_times)
    hdr_robertson = merge_robertson.process(image_list, times=exposure_times.copy(), response=crf_robertson.copy())

    # 创建窗口
    pyplot.figure('Inverse Camera Response Function')
    # 显示直方图
    x1 = range(254)
    x2 = range(254)
    pyplot.subplot(1, 2, 1)
    # pyplot.plot(x1, crf_debevec[:, :, 0], 'go-', x1, crf_debevec[:, :, 1], 'rx-', x1, crf_debevec[:, :, 2], 'b*-')
    pyplot.plot(x1, crf_debevec[x1, :, 0], 'r', x1, crf_debevec[x1, :, 1], 'g', x1, crf_debevec[x1, :, 2], 'b')
    pyplot.xlabel('Measured Intensity')
    pyplot.ylabel('Calibrated Intensity')
    pyplot.title('Debevec Inverse Camera Response Function')
    pyplot.subplot(1, 2, 2)
    pyplot.plot(x2, crf_robertson[x2, :, 0], 'r', x2, crf_robertson[x2, :, 1], 'g', x2, crf_robertson[x2, :, 2], 'b')
    pyplot.xlabel('Measured Intensity')
    pyplot.ylabel('Calibrated Intensity')
    pyplot.title('Robertson Inverse Camera Response Function')