PCA-Based Spatially Adaptive Denoising of CFA Images for Single-Sensor Digital Cameras

Abstract

Single-sensor digital color cameras use a process called color demosaicking to produce full color images from the data captured by a color filter array (CFA). The quality of demosaicked images is degraded due to the sensor noise introduced during the image acquisition process. The conventional solution to combating CFA sensor noise is demosaicking first, followed by a separate denoising processing. This strategy will generate many noise-caused color artifacts in the demosaicking process, which are hard to remove in the denoising process. Few denoising schemes that work directly on the CFA images have been presented because of the difficulties arisen from the red, green and blue interlaced mosaic pattern, yet a well designed denoising first and demosaicking later scheme can have advantages such as less noise-caused color artifacts and cost-effective implementation. This paper presents a principle component analysis (PCA) based spatially-adaptive denoising algorithm, which works directly on the CFA data using a supporting window to analyze the local image statistics. By exploiting the spatial and spectral correlations existed in the CFA image, the proposed method can effectively suppress noise while preserving color edges and details. Experiments using both simulated and real CFA images indicate that the proposed scheme outperforms many existing approaches, including those sophisticated demosaicking and denoising schemes, in terms of both objective measurement and visual evaluation.

单传感器数字彩色相机使用称为彩色去马赛克的过程从由滤色器阵列（CFA）捕获的数据产生全色图像。由于在图像采集过程中引入的传感器噪声，去马赛克图像的质量降低。解决CFA传感器噪声的传统解决方案首先是去马赛克，然后是单独的去噪处理。该策略将在去马赛克过程中产生许多噪声引起的颜色伪影，这在去噪过程中难以去除。由于红色，绿色和蓝色交错镶嵌图案的困难，很少有直接在CFA图像上工作的去噪方案，但是设计良好的去噪第一和去马赛克后期方案可以具有诸如噪声引起的颜色伪影较少的优点。 并且具有成本效益的实施。本文提出了一种基于主成分分析（PCA）的空间自适应去噪算法，该算法使用支持窗口直接处理CFA数据，以分析局部图像统计。通过利用CFA图像中存在的空间和光谱相关性，所提出的方法可以有效地抑制噪声，同时保留颜色边缘和细节。使用模拟和真实CFA图像的实验表明，在客观测量和视觉评估方面，所提出的方案优于许多现有方法，包括那些复杂的去马赛克和去噪方案。

I. INTRODUCTION

Most existing digital color cameras use a single sensor with a color filter array (CFA) [11] to capture visual scenes in color. Since each sensor cell can record only one color value, the other two missing color components at each position need to be interpolated from the available CFA sensor readings to reconstruct the full-color image. The color interpolation process is usually called color demosaicking (CDM). Many CDM algorithms [2] [11], [14] [17] proposed in the past are based on the unrealistic assumption of noise-free CFA data. The presence of noise in CFA data not only deteriorates the visual quality of captured images, but also often causes serious demosaicking artifacts which can be extremely difficult to remove using a subsequent denoising process. Note that many advanced denoising algorithms [19] [26], which are designed for monochromatic (or full color) images, are not directly applicable to CFA images due to the underlying mosaic structure of CFAs. To overcome the problem, we will propose a principle component analysis (PCA)-based denoising scheme which directly operates on the CFA domain of captured images.

大多数现有的数码彩色摄像机使用带有滤色镜阵列（CFA）的单个传感器[11]来捕捉彩色视觉场景。由于每个传感器单元只能记录一个颜色值，因此每个位置的另外两个缺失的颜色分量需要从可用的CFA传感器读数进行插值以重建全色图像。颜色插值过程通常称为颜色去马赛克（CDM）。过去提出的许多CDM算法[2] [11]，[14] [17]都基于无噪声CFA数据的不切实际的假设。CFA数据中的噪声的存在不仅恶化了所捕获图像的视觉质量，而且经常导致严重的去马赛克伪像，使用随后的去噪处理非常难以去除。 请注意，许多先进的去噪算法[19] [26]，设计用于单色（或全色）图像，由于CFA的基础镶嵌结构，不能直接应用于CFA图像。 为了克服这个问题，我们将提出一种基于主成分分析（PCA）的去噪方案，该方案直接在捕获图像的CFA域上运行。

Though most existing CDM techniques [3] [10], [14] [17] assume noise-free CFA data, this assumption does not hold well in practice. For almost all kinds of color imaging devices, ranging from the low-cost and/or resource-constrained ones such as wireless camera phones to the high-end ones such as digital cinema cameras, image corruptive noise is inherent and can be severe; thus, the restoration of color images from noisy CFA data is a challenging problem. To suppress the effect of noise on the demosaicked image, three strategies are possible: denoising after demosaicking; denoising before demosaicking; and joint demosaicking-denoising.

虽然大多数现有的CDM技术[3] [10]，[14] [17]假设无噪声CFA数据，但这种假设在实践中并不适用。对于几乎所有类型的彩色成像设备，从低成本和/或资源受限的彩色成像设备，如无线照相手机，到数字电影摄像机等高端彩色成像设备，图像腐败噪声是固有的，可能是严重的; 因此，从嘈杂的CFA数据中恢复彩色图像是一个具有挑战性的问题。为了抑制噪声对去镶嵌图像的影响，可能有三种策略：去马赛克后进行去噪; 在去马赛克之前进行去噪; 和联合去马赛克去噪。

An intuitive and convenient strategy to remove noise is to denoise the demosaicked images. Algorithms developed for gray-scale imaging, for example [19] [26], can be applied to each channel of the demosaicked color image separately whereas some color image filtering techniques [12], [13] process color pixels as vectors. The problem of this strategy is that noisy sensor readings are roots of many color artifacts in demosaicked images and those artifacts are difficult to remove by denoising the demosaicked full-color data. In general the CFA readings corresponding to different color components have different noise statistics. The CDM process blends the noise contributions across channels, thus producing compound noise that is difficult to characterize. This makes the design of denoising algorithms for single-sensor color imaging very difficult.

消除噪声的直观且方便的策略是对去镶嵌的图像进行去噪。为灰度成像开发的算法，例如[19] [26]，可以分别应用于去马赛克彩色图像的每个通道，而一些彩色图像过滤技术[12]，[13]将彩色像素处理为矢量。该策略的问题在于，噪声传感器读数是去镶嵌图像中的许多颜色伪影的根源，并且通过对去镶嵌的全色数据进行去噪来难以去除那些伪像。通常，对应于不同颜色分量的CFA读数具有不同的噪声统计。CDM过程混合了跨通道的噪声贡献，从而产生难以表征的复合噪声。这使得用于单传感器彩色成像的去噪算法的设计非常困难。

Recently, some schemes that perform demosaicking and denoising jointly have been proposed [27] [34]. In [33], Trussell and Hartig presented a mathematical model for color demosaicking using minimum mean square error (MMSE) estimator. The additive white noise is considered in the modeling. Ramanath and Snyder [34] proposed a bilateral filter based demosaicking method. Since bilateral filtering exploits the similarity in both spatial and intensity spaces, this scheme can handle light noise corrupted in the CFA image. Hirakawa and Parks [27] developed a joint demosaicking-denoising algorithm by using the total least square (TLS) technique where both demosaicking and denoising are treated as an estimation problem with the estimates being produced from the available neighboring pixels. The filter used for joint demosaicking-denoising is determined adaptively using the TLS technique under some constraints of the CFA pattern. In [28] and [29], Hirakawa et al. proposed two wavelet based schemes that can perform CDM simultaneously with denoising. The joint demosaicking-denoising scheme developed by Zhang et al. [30] first performs demosaicking-denoising on the green channel. The restored green channel is then used to estimate the noise statistics in order to restore the red and blue channels. In implementing the algorithm, Zhang et al. estimated the red-green and blue-green color difference images rather than directly recovering the missing color samples by using a linear model of the color difference signals. Inspired by the directional linear minimum mean square-error estimation (DLMMSE) based CDM scheme in [9], Paliy et al. [31], [32] proposed an effective nonlinear and spatially adaptive filter by using local polynomial approximation to remove the demosaicking noise generated in the CDM process and then adapted this scheme to noisy CFA inputs for joint demosaicking-denoising.

最近，已经提出了一些联合执行去马赛克和去噪的方案[27] [34]。在[33]中，Trussell和Hartig提出了一种使用最小均方误差（MMSE）估计器进行颜色去马赛克的数学模型。 在建模中考虑加性白噪声。Ramanath和Snyder [34]提出了一种基于双边滤波器的去马赛克方法。 由于双边滤波利用空间和强度空间中的相似性，该方案可以处理CFA图像中被破坏的光噪声。Hirakawa和Parks [27]通过使用总体最小二乘（TLS）技术开发了一种联合去马赛克去噪算法，其中去马赛克和去噪被视为估计问题，估计是从可用的相邻像素产生的。在CFA模式的某些约束下，使用TLS技术自适应地确定用于联合去马赛克去噪的滤波器。 在[28]和[29]中，Hirakawa等人提出了两种基于小波的方案，它们可以在去噪的同时执行CDM。Zhang等人[30]开发的联合去马赛克去噪方案首先在绿色通道上进行去马赛克去噪。 然后使用恢复的绿色通道估计噪声统计数据，以恢复红色和蓝色通道。 在实现该算法时，Zhang等人估计了红绿和蓝绿色差图像，而不是通过使用色差信号的线性模型直接恢复丢失的颜色样本。受[9]中基于方向线性最小均方误差估计（DLMMSE）的CDM方案的启发，Paliy等[31]，[32]提出了一种有效的非线性和空间自适应滤波器，通过使用局部多项式近似来消除CDM过程中产生的去马赛克噪声，然后将该方案适用于噪声CFA输入以进行联合去马赛克去噪。

The third way to remove noise from CFA data is to implement denoising before demosaicking. However, due to the underlying mosaic structure of CFAs, many existing effective monochromatic image denoising methods can not be applied to the CFA data directly. To overcome the problem, the CFA image can be divided into several sub-images using the approach known from the CFA image compression literature, e.g., [37]. Since each of the sub-images constitutes a gray-scale image, it can be enhanced using denoising algorithms from gray-scale imaging. The desired CFA image is obtained by restoring it from the enhanced sub-images. Nonetheless, such a scheme does not exploit the interchannel correlation which is essential to reduce various color shifts and artifacts in the final image [12], [13]. Since the volume of CFA images is three times less than that of the demosaicked images, there is a demand to develop new denoising algorithms which can fully exploit the interchannel correlations and operate directly on CFA images, thus achieving higher processing rates.

消除CFA数据噪声的第三种方法是在去马赛克之前实现去噪。 然而，由于CFA的基础镶嵌结构，许多现有的有效单色图像去噪方法不能直接应用于CFA数据。为了克服这个问题，可以使用CFA图像压缩文献中已知的方法将CFA图像分成几个子图像，例如[37]。由于每个子图像构成灰度图像，因此可以使用来自灰度成像的去噪算法来增强它。通过从增强的子图像恢复来获得期望的CFA图像。 尽管如此，这种方案并没有利用声道间相关性，这对于减少最终图像中的各种色彩偏移和伪影至关重要[12]，[13]。 由于CFA图像的体积比去马赛克图像的体积小三倍，因此需要开发新的去噪算法，其能够充分利用信道间相关性并直接在CFA图像上操作，从而实现更高的处理速率。

This paper presents a new and efficient scheme for denoising CFA images. The technique of principle component analysis (PCA) [38], [39] is employed to analyze the local structure of each CFA variable block, which contains color components from different channels. In [26], a PCA-based monochromatic image denoising scheme was proposed and here we improve the algorithm and extend it to CFA mosaic images. By adaptively computing the co-variance matrix of each variable block, the PCA could transform the noisy signal into another space, in which the signal energy is better clustered and the noise can be more effectively removed. Since there can be different and varying structures in each local training window, to improve the estimation accuracy of PCA transformation matrix we select the similar blocks to the underlying one and use them only, instead of all blocks, for PCA training. Such a training sample selection procedure can better preserve the image local structures. The proposed spatially adaptive PCA denoising scheme works directly on the CFA image and it can effectively exploit the spatial and spectral correlation simultaneously.

本文提出了一种新的有效的CFA图像去噪方案。 主成分分析技术[PC] [38]，[39]用于分析每个CFA可变块的局部结构，其中包含来自不同通道的颜色成分。 在[26]中，提出了一种基于PCA的单色图像去噪方案，在这里我们改进算法并将其扩展到CFA马赛克图像。 通过自适应地计算每个可变块的协方差矩阵，PCA可以将噪声信号转换成另一个空间，其中信号能量被更好地聚类并且可以更有效地去除噪声。由于在每个局部训练窗口中可以存在不同且变化的结构，为了提高PCA变换矩阵的估计精度，我们选择与底层相似的块，并且仅使用它们而不是所有块来进行PCA训练。 这种训练样本选择过程可以更好地保留图像局部结构。 所提出的空间自适应PCA去噪方案直接在CFA图像上工作，它可以同时有效地利用空间和光谱相关性。

The rest of the paper is structured as follows. Section II briefly reviews the concept of PCA. Section III presents the PCA-based denoising algorithm for CFA images. The motivation and design characteristics are described in detail. In Section IV, experimental results are provided to demonstrate the efficiency of the proposed method for single-sensor CFA image denoising. Finally, conclusions are drawn in Section V.

本文的其余部分的结构如下。第二节简要回顾了PCA的概念。第三节介绍了基于PCA的CFA图像去噪算法。详细描述了动机和设计特征。在第IV节中，提供了实验结果以证明所提出的单传感器CFA图像去噪方法的效率。最后，结论在第五节中得出。

II. PRINCIPAL COMPONENT ANALYSIS(PCA)

PCA [38], [39] is a classical de-correlation technique which has been widely used for dimensionality reduction with direct applications in pattern recognition, data compression and noise reduction. Denote by  an m-component vector variable and denote by

PCA [38]，[39]是一种经典的去相关技术，已被广泛用于降维，直接应用于模式识别，数据压缩和降噪。 用表示m维向量变量并表示为

(2-1)

the sample matrix of X, where , is the discrete sample of variable . The ith row of sample matrix X, denoted by , is the sample vector of . The mean value of can be estimated as , and thus, the mean value vector of X is

X的样本矩阵，其中，是变量的离散样本。 由表示的第i行样本矩阵X是的样本向量。 的平均值可以估计为，因此，平均值向量X为

(2-2)

Subtracting  from  results in the centralized vector . The element of  is  and the sample vector of  is , where . Accordingly, the centralized matrix  of  can be expressed as follows

从中减去导致集中向量。 的元素是，的样本矢量是，其中。因此，的集中矩阵可以表示如下

(2-3)

The co-variance matrix of  is calculated as 

的协方差矩阵计算为

The goal of PCA is to find an orthonormal transformation matrix P to decorrelate , i.e.,  and the co-variance matrix of  is diagonal. Since  is symmetrical, its singular value decomposition (SVD) can be written as

PCA的目标是找到正交变换矩阵P以解相关，即并且的协方差矩阵是对角线。 由于是对称的，因此可以将其奇异值分解（SVD）写为

(2-4)

where  is the mxm orthonormal eigenvector matrix and  is the diagonal eigenvalue matrix with . The terms  and  are the eigenvectors and eigenvalues of . By setting

其中是标准正交特征向量矩阵，是具有的对角特征值矩阵。术语和是的特征向量和特征值。通过设置

(2-5)

 can be decorrelated, i.e.,  and .

可以去相关的，即和。

Besides decorrelation, another important property of PCA is that it is optimal by using a subset of its principal components to represent the original signal. For example, one could use the first k most important eigenvectors to form the transformation matrix as , k <m. Then the transformed dataset  will be of dimension kxn. Compared with the original dataset , which is mxn, the dimension of  is reduced while preserving most of the energy of . This property is also known as optimal dimensionality reduction [39].

除去相关之外，PCA的另一个重要特性是通过使用其主要分量的子集来表示原始信号是最佳的。例如，可以使用前k个最重要的特征向量将形成变换矩阵为，k <m。然后，变换数据集将具有维度kxn。与原始数据集（mxn）相比，的维数减小，同时保留了的大部分能量。 这种性质也称为最佳降维[39]。

The optimal dimensionality reduction property of PCA can be used for noise removal. Generally speaking, the energy of a signal will concentrate on a small subset of the PCA transformed dataset, while the energy of noise will evenly spread over the whole dataset. Therefore, by preserving only the most important subset of the transformed dataset and then conducting the inverse PCA transform, the noise could be significantly reduced while the signal being well recovered. In Section III, this idea will be used to build the proposed direct CFA image denoising solution.

PCA的最佳降维性能可用于去除噪声。一般而言，信号的能量将集中在PCA变换数据集的一小部分上，而噪声能量将均匀地分布在整个数据集上。因此，通过仅保留变换数据集的最重要子集然后进行逆PCA变换，可以在信号被良好恢复的同时显着降低噪声。 在第三节中，这个想法将用于构建所提出的直接CFA图像去噪解决方案。

III. PCA-BASED DENOISING OF CFA MOSAIC IMAGES

A. CFA Sensor Noise

Without loss of generality, the widely used Bayer pattern [1] is considered in this paper and the algorithm can be easily extended to other CFAs. In the Bayer pattern [Fig. 1(a)], the red (r), green (g) and blue (b) samples are interlaced, with the double sampling frequency of the green channel compared to the red and blue channels. The purpose of CDM is to interpolate the two missing color components at each pixel location in the CFA image, thus restoring the full-color image from the CFA sensor readings. Fig. 1(b) shows a CFA image and Fig. 1(c) depicts the corresponding demosaicked image.

在不失一般性的情况下，本文考虑了广泛使用的拜耳模式[1]，并且该算法可以很容易地扩展到其他CFA。在拜耳模式[图1(a)中，红色(r)，绿色(g)和蓝色(b)样本是交错的，绿色通道与红色和蓝色通道相比较为双采样频率。CDM的目的是在CFA图像中的每个像素位置处插入两个缺失的颜色分量，从而从CFA传感器读数恢复全色图像。 图1(b)显示了CFA图像，图1(c)显示了相应的去马赛克图像。

Most CDM algorithms [2] [10], [14] [17] operate on the assumption of noise-free CFA data. This assumption, however, is invalid in practice. Digital color imaging devices, ranging from the low cost and/or resource constrained ones (e.g., wireless camera phones) to the high-end ones (e.g., digital cinema cameras) produce images with modest to severe noise, making digital photographs not always visually pleasing. It is accepted that the corrupted noise in charge-coupled device (CCD) and complementary-symmetry/metal-oxide semiconductor (CMOS) sensors is signal-dependent [27], [35], [36]. Foi et al. [35] pointed out that the noise variance depends on the signal magnitude, while Poisson, film-grain, multiplicative and speckle models can be used to model the noise. In [27], Hirakawa modeled the raw sensor output as , where x is the desired noiseless signal,  is unit Gaussian white noise and k0 and k1 are sensor dependent parameters. Although this noise model may fit some sensors better, the design of denoising algorithms may be complex and the computational cost may be very expensive.

大多数CDM算法[2] [10]，[14] [17]在无噪声CFA数据的假设下运行。然而，这种假设在实践中是无效的。从低成本和/或资源受限（例如，无线照相手机）到高端（例如，数字电影摄像机）的数字彩色成像设备产生具有适度到严重噪声的图像，使得数字照片不总是在视觉上赏心悦目。可以接受的是，电荷耦合器件（CCD）和互补对称/金属氧化物半导体（CMOS）传感器中的损坏噪声依赖于信号[27]，[35]，[36]。Foi等[35]指出噪声方差取决于信号幅度，而泊松，胶片粒子，乘法和散斑模型可用于模拟噪声。在[27]中，平川将原始传感器输出建模为，x其中是所需的无噪声信号，是单位高斯白噪声，k0和k1是传感器相关参数。尽管该噪声模型可以更好地适合某些传感器，但是去噪算法的设计可能是复杂的并且计算成本可能非常昂贵。

One simple and widely used noise model is the signal-independent additive noise model y=x+v. It is a special case of the signal-dependent noise model with k1=0 and commonly used to approximate the Poisson noise in CCD/CMOS sensors. Since the additive noise model is simple to use in the design and analysis of denoising algorithms, it has been widely used in the literature [19] [26]. The signal-dependent noise characteristic can be compensated by estimating the noise variance adaptively in each local area [33].

一种简单且广泛使用的噪声模型是与信号无关的加性噪声模型y=x+v。这是信号相关噪声模型的一种特殊情况k1=0，通常用于近似CCD / CMOS传感器中的泊松噪声。由于加性噪声模型在去噪算法的设计和分析中很容易使用，因此它已被广泛应用于文献[19] [26]。 可以通过在每个局部区域自适应地估计噪声方差来补偿信号相关的噪声特性[33]。

In [30], Zhang et al. proposed a channel-dependent additive noise model, which is a tradeoff between the signal-dependent noise model and the signal-independent additive noise model, by considering the different types of color filters in the CFA

在[30]中，Zhang等人通过考虑CFA中不同类型的滤色器，提出了一种依赖于通道的加性噪声模型，它是信号相关噪声模型和信号无关加性噪声模型之间的权衡

(3-1)

where , , and  are the noise signals in the red, green and blue locations of the CFA image (referring to Fig. 1). The terms r, g, and b are the desired sample values to be recovered from their noisy versions , , and . The second order statistics of , , and , i.e., the corresponding standard deviations ,  and , may be different but the noises are assumed to be mutually uncorrelated.

其中，和是CFA图像的红色，绿色和蓝色位置的噪声信号（参见图1）。术语r，g，b是从它们的有噪声版本，和中恢复的所需样本值。，和的二阶统计量，即相应的标准偏差，和可以不同，但假设噪声是相互不相关的。

We adopt the channel-dependent model in the design of the proposed CFA image denoising algorithm. It allows the noise statistics to vary in different channels because a given type of sensors behaves differently in different wavelengths. On the other hand, in channel-dependent noise model the sensor noise is independent of signal within each channel to simplify the denoising algorithm. This simplification does not materially degrade the visual quality of denoised images because the signal to noise ratio (SNR) is high anyway when the signal amplitude is high.

我们在拟议的CFA图像去噪算法的设计中采用了信道相关模型。它允许噪声统计在不同的通道中变化，因为给定类型的传感器在不同波长中表现不同。另一方面，在信道相关噪声模型中，传感器噪声与每个信道内的信号无关，以简化去噪算法。这种简化不会显着降低去噪图像的视觉质量，因为当信号幅度高时，信噪比（SNR）无论如何都很高。

B. CFA Block-Based Spatially Adaptive PCA

A conventional solution to removing the effect of noise on the demosaicked full color image is to denoise after CDM. However, if noise is untreated in the demosaicking step, the noise caused color artifacts can be very hard to remove in the subsequent denoising process. The CDM process will complicate the noise characteristics by blending the noise across channels. Some joint demosaicking-denoising schemes have been reported [27], [30]. Since both demosaicking and denoising can be viewed as to estimate a sample from its neighbors, Hirakawa and Parks [27] adaptively computed a filter to accomplish the two tasks simultaneously by using the TLS technique under some constraints of the CFA pattern. Zhang et al. [30] first estimate the color difference signal from the noisy CFA image and then reconstruct the green channel by using a specific wavelet denoising algorithm. The red/blue channel is then readily recovered. Both the two schemes perform better than many demosaicking first and denoising later methods.

消除去马赛克全彩色图像上的噪声影响的传统解决方案是在CDM之后去噪。然而，如果在去马赛克步骤中未对噪声进行处理，则在随后的去噪处理中可能很难去除引起颜色伪影的噪声。CDM过程会通过混合通道中的噪声来使噪声特性复杂化。 已经报道了一些联合去马赛克去噪方案[27]，[30]。由于去马赛克和去噪都可以被视为估计其邻居的样本，Hirakawa和Parks [27]自适应地计算了一个过滤器，通过在CFA模式的某些约束下使用TLS技术同时完成两个任务。Zhang等[30]首先从噪声CFA图像估计色差信号，然后使用特定的小波去噪算法重建绿色通道。然后可以很容易地恢复红/蓝通道。这两种方案都比先前的许多去马赛克和后来的去噪方法都要好。

The other strategy to remove noise from CFA data is to implement denoising before demosaicking. Some apparent advantages of this pipeline are that it can reduce the noise-caused color artifacts, and the denoising and demosaicking algorithms can be independently designed. The difficulty of this strategy lies in the red, green and blue interlaced CFA pattern, which blocks the application of many existing effective monochromatic image denoising methods. One simple solution is to partition, for example, the big Bayer pattern CFA image into one red, one blue and two greens sub-images and then denoise them separately as gray level images. This solution, however, does not exploit the spectral correlation within red, green and blue channels.

从CFA数据中去除噪声的另一个策略是在去马赛克之前实现去噪。该流水线的一些明显优点是它可以减少噪声引起的颜色伪影，并且可以独立设计去噪和去马赛克算法。这种策略的难点在于红色，绿色和蓝色交错的CFA图案，这阻碍了许多现有的有效单色图像去噪方法的应用。一个简单的解决方案是将例如大拜耳模式CFA图像分割成一个红色，一个蓝色和两个绿色子图像，然后将它们分别作为灰度图像去噪。然而，该解决方案没有利用红色，绿色和蓝色通道内的光谱相关性。

To fully exploit the correlation among the three color channels, we propose a spatially adaptive denoising algorithm that works directly on CFA mosaic images using the PCA technique introduced in Section II. In [26], Muresan and Parks proposed a PCA-based denoising algorithm for monochromatic images. In this paper, we improve the algorithm and extend the PCA-based denoising to mosaic CFA images by considering the special structures of CFA patterns. Following the underlying layout of the CFA, such as the Bayer pattern considered throughout this paper, we define a block which consists of at least one red, one green and one blue sample. We call this block the variable block because the elements in this block will be used as the variables in PCA training. For example, Fig. 2 shows the variable block with four elements: one red, one blue and two green samples. We stretch the block to a column vector and denote it as

为了充分利用三个颜色通道之间的相关性，我们提出了一种空间自适应去噪算法，该算法使用第二节中介绍的PCA技术直接在CFA马赛克图像上工作。在[26]中，Muresan和Parks提出了一种基于PCA的单色图像去噪算法。在本文中，我们通过考虑CFA模式的特殊结构，改进算法并将基于PCA的去噪扩展到拼接CFA图像。遵循CFA的基础布局，例如本文中考虑的拜耳模式，我们定义了一个由至少一个红色，一个绿色和一个蓝色样本组成的块。我们将此块称为变量块，因为此块中的元素将用作PCA训练中的变量。例如，图2显示了具有四个元素的可变块：一个红色，一个蓝色和两个绿色样本。我们将块拉伸到列向量并将其表示为

(3-2)

Please note that this four-element variable block is used here for the sake of simplicity of the discussion to be followed. In practical implementations, this block can be bigger, for example, 4 4, 6 6, etc.

请注意，这里使用这个四元素可变块是为了简化下面的讨论。在实际实现中，该块可以更大，例如，4x4,6x6等。

Variable vector will be associated with a block which contains enough samples for training. This training block should be much bigger than the variable block in order to ensure that the statistics of the variables can be reasonably calculated. Wherever any part of the training block can match the variable block, the pixels of that part will be taken as the samples of the variable vector. For instance, in Fig. 2, there are nine samples for each element of X. Specifically, variable g1 is associated with green samples at locations (1,1), (1,3), (1,5), (3,1), (3,3), (3,5), (5,1), (5,3), and (5,5). The samples of other variables can be obtained correspondingly. It is reasonable to assume that those samples are independent draws of the variable because they are acquired at different spatial locations. Let G1 be the row vector containing all the samples associated with g1 whereas R2, B3, and G4 denote similar row vectors for r2, r3 and g4, respectively. The whole dataset of X will be .

变量向量将与包含足够训练样本的块相关联。该训练块应比变量块大得多，以确保可以合理地计算变量的统计量。只要训练块的任何部分与可变块匹配，该部分的像素将被视为变量向量的样本。例如，在图2中，的每个元素有九个样本。具体来说，变量g1与位置（1,1），（1,3），（1,5），（3,1），（3,3），（3,5），（5， 1），（5,3）和（5,5）。可以相应地获得其他变量的样本。可以合理地假设这些样本是变量的独立绘制，因为它们是在不同的空间位置获取的。设是包含与相关的所有样本的行向量，而，和分别表示，和的相似行向量。的整个数据集将是。

Denote by , , , and  the mean values of variables , ,  and , respectively. Those mean values can be estimated as the average of all the samples in , ,  and , respectively. The mean vector of X can be denoted as .X is centralized as  and  is accordingly the centralized dataset of X.

用，，和分别表示变量，，和的平均值。这些平均值可以分别估计为，，和中所有样品的平均值。的平均向量可以表示为。被集中为而因此是的集中数据集。

With the additive noise model, the variable of noisy observation of X can be expressed as follows:

利用加性噪声模型，噪声观测的变量可表示如下：

(3-3)

where  is the noise variable vector. Zero mean noise characteristics dictate that the mean vector of  is the same as that of , i.e., . Note that, in practice, is calculated from the samples of , but not . The centralized vector of  is then .

其中是噪声变量向量。 零平均噪声特性决定的平均矢量与的平均矢量相同，即。 注意，在实践中，是根据的样本计算的，而不是的样本。然后的集中矢量是。

Similar to X, we denote by  the dataset of additive channel-dependent noise V, where  and  come from green channel noise ,  comes from red channel noise  and  from blue channel noise . The available measurements of noiseless dataset X is then . We subtract the mean value  from  to get the centralized dataset of .

与类似，我们用表示加性通道相关噪声的数据集，其中和来自绿色通道噪声，来自红色通道噪声和来自蓝色通道噪声。然后，无噪声数据集的可用测量值为。我们从中减去平均值以获得的集中数据集

(3-4)

Now the problem transforms to estimate  from the noisy measurement . Suppose we have got the estimated dataset of , denoted by , then the samples in the training block are denoised. The central part of the training block can be extracted as the denoising block because boundary samples do not usually contribute to the denoising performance as much as the samples from the central part. The whole CFA image can be denoised by moving the denoising block from top left to bottom right. The way of removing the noise from  using PCA techniques will be discussed in the following.

现在问题转变为从噪声测量估计。假设我们得到的估计数据集，用表示，然后训练块中的样本被去噪。可以提取训练块的中心部分作为去噪块，因为边界样本通常不像中心部分的样本那样对去噪性能有贡献。通过将去噪块从左上角移动到右下角，可以对整个CFA图像进行去噪。下面将讨论使用PCA技术从中去除噪声的方法。

C.Denoising in PCA Domain

As mentioned in Section II, the optimal dimension reduction property of PCA can be used to reduce noise. By computing the covariance matrix  of , the optimal PCA transformation matrix  for  can be obtained by using (2-4) and (2-5). However, the available dataset  is noise corrupted so that  cannot be directly computed. Fortunately,  can be estimated using the linear noise model .

如第II部分所述，PCA的最佳降维特性可用于降低噪音。通过计算的协方差矩阵，可以通过使用（2-4）和（2-5）获得的最佳PCA变换矩阵。但是，可用的数据集是噪声损坏的，因此无法直接计算。幸运的是，可以使用线性噪声模型来估计。

Assuming that training samples are available for each element of , the covariance matrix of  can be estimated using maximal likelihood estimation (MLE)

假设训练样本可用于的每个元素，则可以使用最大似然估计（MLE）来估计的协方差矩阵

(3-5)

Since the signal  and noise  are uncorrelated, items  and  will be nearly zero matrices which reduces the above expression of  to the following:

由于信号和噪声是不相关的，因此项和将几乎为零矩阵，这将的上述表达式减少到以下：

(3-6)

where  and  are the covariance matrices of  and , respectively.

其中和分别是和的协方差矩阵。

Estimating  from  requires  is known. With the noise vector  and the fact that the four elements of V are uncorrelated with each other, we have

估计来自需要是已知的。利用噪声向量和V的四个元素彼此不相关的事实，我们有

(3-7)

where , , and  are the standard deviations of the channeldependent noise ,  and  in (3-1). With (3-6) and (3-7), the covariance of  can be calculated as . Thus, there can be some negative values of  in the diagonal positions. In the implementation, we will replace the negative values with zero or a small positive number, e.g., 0.0001.

其中，和是（3-1）中信道相关噪声，和的标准偏差。对于（3-6）和（3-7），协方差可以计算为。因此，在对角线位置可能存在一些的负值。在实现中，我们将用零或小的正数替换负值，例如0.0001。

Now we can decompose by using (2-4) as

现在我们可以使用（2-4）分解为

(3-8)

where  is the 4x4 orthonormal eigenvector matrix and  is the diagonal eigenvalue matrix with . The orthonormal PCA transformation matrix for  is then

其中是4x4正交特征向量矩阵，是具有的对角特征值矩阵。然后是的正交PCA变换矩阵

(3-9)

It should be noted that if the noise levels of , , and  are the same, i.e., ,  then  will be an identity matrix with a scaling factor . In this case, it can be proved that the SVD of  and  will give the same eigenvector matrix  and, hence, the same PCA transformation matrix . However, when , , and  are different,  can not be scaled to an identity matrix and, therefore,  and  will yield different eigenvector matrices by SVD.

应当注意，如果，和的噪声水平相同，即，则将是具有缩放因子的单位矩阵。在这种情况下，可以证明和的SVD将给出相同的特征向量矩阵，因此，给出相同的PCA变换矩阵。然而，当，和不同时，不能缩放到单位矩阵，因此，和将通过SVD产生不同的特征向量矩阵。

Applying  to the noisy dataset  resulting in the following:

将应用于噪声数据集，导致以下结果：

(3-10)

where  is the decorrelated dataset for signal and  is the transformed noise dataset for noise. Since signal  and noise  are uncorrelated, the covariance matrix of  is

其中是信号的去相关数据集，是噪声的变换噪声数据集。 由于信号和噪声是不相关的，因此的协方差矩阵是

(3-11)

where

其中

(3-12)

(3-13)

are the covariance matrices of  and , respectively.

是和的协方差矩阵。

In the PCA transformed domain  in (3-10), most energy of  concentrates on the several most important components, i.e., the first several rows of  whereas the energy of noise  is distributed in  much more evenly. Therefore, resetting the last several rows (the least important components) of  as zeros will preserve well the signal  while removing the noise . This operation is actually based on the optimal dimension reduction property of PCA, as discussed in Section II. We denote by  the dimension reduced (by resetting the last several rows as zeros) dataset of , and  can be written as , where  and  represent, respectively, the dimension reduced datasets of  and . Similarly, the corresponding covariance matrices, denoted as , , and , are related as .

在（3-10）中的PCA变换域中，的大部分能量集中在几个最重要的分量上，即的前几行，而噪声的能量更均匀地分布在中。因此，将的最后几行（最不重要的分量）重置为零将保留信号，同时消除噪声。 该操作实际上基于PCA的最佳降维特性，如第II部分所述。我们用表示减小的维度（通过将最后几行重置为零）的数据集，并且可以写为，其中和分别表示和的降维数据集。类似地，对应的协方差矩阵，表示为，和，与相关。

The noise in dimension reduced dataset  can be further suppressed by using linear minimum mean squared-error estimation (LMMSE). The LMMSE of , i.e., the ith row of , is obtained as

通过使用线性最小均方误差估计（LMMSE），可以进一步抑制尺寸减小的数据集中的噪声。的LMMSE，即的第i行，获得为

(3-14)

where . Applying (3-14) to each nonzero row of  yields the full denoised dataset . Now, the denoised result of the original dataset , i.e., the estimation of unknown noiseless dataset , can be obtained by transforming back  from PCA domain to time domain as follows:

其中。 将（3-14）应用于的每个非零行产生完整的去噪数据集。现在，原始数据集的去噪结果，即未知无噪声数据集的估计，可以通过如下将从PCA域转换回时域来获得：

(3-15)

Reformatting  results in the denoised CFA block.

重新格式化会导致去噪的CFA块。

In the proposed PCA-based CFA image denoising algorithm, we need to set two parameters: the sizes of variable block and training block. (The denoising block can be set as the same as or smaller than the variable block.) Different settings of the two parameters lead to different results and have different complexity. If the resolution of the image is low, the size of the variable block should be relatively small because the spatial correlation of low resolution images will also be low. Empirically, we found that setting the variable block as 4x4, 6x6 or 8x8 can achieve good results for most of the testing images. The size of training block can be 16 (or higher) times that of the variable block, e.g., 24x24 or 30x30 for a 6x6 variable block.

在提出的基于PCA的CFA图像去噪算法中，我们需要设置两个参数：可变块和训练块的大小。 （去噪块可以设置为与可变块相同或更小。）两个参数的不同设置会导致不同的结果并具有不同的复杂性。 如果图像的分辨率低，则可变块的大小应该相对较小，因为低分辨率图像的空间相关性也将较低。根据经验，我们发现将变量块设置为4x4,6x6或8x8可以为大多数测试图像获得良好的结果。 训练块的大小可以是可变块的大小的16倍（或更高），例如，对于6×6可变块，24x24或30x30。

D. CFA Image Decomposition and Training Sample Selection

The PCA-based denoising scheme described in the previous sections can be directly applied to the noisy CFA image for noise suppression. Although the noise can be effectively removed, in the experiments we found two problems of the proposed method. First there can be some visible noise residual in the smooth areas. Second there can be some phantom artifacts along edge boundaries with smooth background. Fig. 3(a) and (b) shows two examples respectively. In the first row, Fig. 3 (a-1) (a-4) shows the cropped smooth area of an original image, the noisy CFA image of it, the denoised CFA image by the proposed PCA-based denoising scheme and the demosaicking result of the denoised CFA (the demosaicking scheme [9] is used here), respectively. In the second row, Fig. 3 (b-1) (b-4) shows an example of the phantom artifacts along the boundary between edges and smooth background. Such noise residual and phantom artifacts make the reconstructed color image visually unpleased in some areas.

前面部分中描述的基于PCA的去噪方案可以直接应用于噪声抑制的噪声CFA图像。 尽管可以有效地去除噪声，但在实验中我们发现了所提出方法的两个问题。 首先，在平滑区域中可能存在一些可见的噪声残留。 其次，边缘边界可能存在一些具有平滑背景的幻像。 图3（a）和（b）分别显示了两个实例。在第一行中，图3（a-1）（a-4）显示了原始图像的裁剪平滑区域，它的嘈杂CFA图像，通过提出的基于PCA的去噪方案的去噪CFA图像和去噪CFA的去马赛克结果（这里使用去马赛克方案[9]）。在第二行中，图3（b-1）（b-4）示出了沿边缘和平滑背景之间的边界的虚影伪影的示例。 这种噪声残留和幻像伪像使得重建的彩色图像在某些区域中在视觉上不令人满意。

The noise residual in the smooth areas is mainly caused by the relatively low local signal to noise ratio or contrast. Since there are no strong edge structures in the smooth areas, there are fewer significant principal components in the PCA domain, and, hence, the discrimination between noise and signals is not as effective as that in the areas with strong edges. In addition, the training samples for the red, green and blue variables are collected from the local window according to the Bayer pattern. The mean values of the variables are computed as the average of the samples and then subtracted from the sample matrix for covariance calculation. However, in this way, the neighboring red and green pixels will not contribute to the mean value estimation of the blue variable, and so do for red and green variables. The biases in mean value estimation will lead to estimation bias of co-variance matrix and, hence, the PCA transformation matrix.

平滑区域中的噪声残留主要是由相对较低的局部信噪比或对比度引起的。由于在平滑区域中没有强边缘结构，因此PCA域中的重要主成分较少，因此，噪声和信号之间的区分不如具有强边缘的区域那样有效。此外，根据拜耳模式从局部窗口收集红色，绿色和蓝色变量的训练样本。将变量的平均值计算为样本的平均值，然后从样本矩阵中减去以进行协方差计算。然而，以这种方式，相邻的红色和绿色像素将不会有助于蓝色变量的平均值估计，红色和绿色变量也是如此。平均值估计中的偏差将导致协方差矩阵的估计偏差，并因此导致PCA变换矩阵。

The reason for the phantom artifacts along edge boundaries with smooth background is as follows. The proposed PCA-based denoising algorithm will use a local training block to estimate the PCA transformation matrix. All the possible samples in the training block are used in the calculation. However, sample structures may change within a block, especially if the block contains object boundaries with smooth background. Involving such samples in the PCA training may lead to much bias in the estimation of PCA transformation matrix and consequently reduce the denoising performance, e.g., generating many phantom artifacts.

沿着具有平滑背景的边缘边界的幻象伪像的原因如下。 所提出的基于PCA的去噪算法将使用局部训练块来估计PCA变换矩阵。 训练块中的所有可能样本都用于计算。 但是，样本结构可能会在块内发生更改，尤其是在块包含具有平滑背景的对象边界时。 在PCA训练中涉及这样的样本可能导致PCA变换矩阵的估计中的很大偏差并因此降低去噪性能，例如，产生许多幻像伪像。

To overcome the above two problems, we propose two preprocessing steps before applying the PCA-based denoising. First, we decompose the noisy CFA image into two parts: the low-pass smooth image and the high-pass image. Denote by  the noisy CFA image. We use a 2-D Gaussian low-pass filter  to smooth 

为了克服上述两个问题，我们在应用基于PCA的去噪之前提出了两个预处理步骤。首先，我们将噪声CFA图像分解为两部分：低通平滑图像和高通图像。 用表示嘈杂的CFA图像。我们使用二维高斯低通滤波器来平滑

(3-16)

The high-pass image is then obtained as

然后获得高通图像

(3-17)

With a suitable scale parameter s in the Gaussian filter, the low-pass image   will be almost noiseless and most of the noise is contained in the high-pass output , which also contains the image edge structures to be preserved. Since  is almost noiseless, we do not make further processing on it. The PCA-based CFA denoising scheme will be applied to the high-pass image , where the noise will be dominant in the smooth areas and they can then be better suppressed by LMMSE filtering in the PCA domain. Denote by  the denoised image of , the final denoised CFA image is obtained as . It can be validated that in a local window of , the mean value of red, green or blue variable will be nearly zero for smooth areas. In some sense, the Gaussian smoothing operation can be viewed as a procedure to better estimate the mean values of red, green and blue variables so that the noise residual in smooth areas can be reduced effectively.

利用高斯滤波器中的合适的比例参数s，低通图像几乎是无噪声的，并且大部分噪声包含在高通输出中，其还包含要保留的图像边缘结构。 由于几乎没有噪音，我们不对其进行进一步处理。基于PCA的CFA去噪方案将应用于高通图像，其中噪声将在平滑区域中占主导地位，然后可以通过PCA域中的LMMSE滤波更好地抑制它们。由表示的去噪图像，最终去噪的CFA图像获得为。可以验证在的局部窗口中，对于平滑区域，红色，绿色或蓝色变量的平均值几乎为零。在某种意义上，高斯平滑操作可以被视为更好地估计红色，绿色和蓝色变量的平均值的过程，从而可以有效地减少平滑区域中的噪声残余。

 

现在让我们集中讨论如何使用平滑背景减少边缘边界附近的幻像伪影。如前所述，这些伪像是由训练块中不适当的训练样本引起的。直观地，这个问题的一个解决方案是选择与底层变量块类似的块并仅使用它们，但不是所有块用于PCA训练。这样的训练样本选择过程可以更好地估计可变块的协方差矩阵，并因此导致更准确的PCA变换矩阵。最后，通过去除幻影伪像可以更好地保留图像局部边缘结构。

用表示中的变量块，用表示从以为中心的训练块生成的相关训练数据集（参见第III-B节）。矩阵的每列可以是的样本向量。在第III-C节中描述的方法中，整个数据集用于训练。在这里，我们从中选择最佳样本进行PCA转换。 由表示，k = 1,2 ..，n，的第k列，并且用表示包含可变块处的样本的样本向量。和的无噪声对应物分别表示为和。 向量的长度是m。和之间的l2距离可以计算为

(3-18)

其中。显然，距离越小，与的相似性越大。

因此，我们根据其相关距离选择训练样本。如果

(3-19)

然后将选择作为的一个训练样本，其中T是预设阈值，并且在实现中可以将其设置为大约5。 假设选择了完全K个样本。 在实践中，我们需要足够大的K来保证的协方差矩阵的合理估计。因此，如果K <100，我们将使用给出最小距离的前100作为训练样本。 由表示由所选择的K个样本向量组成的数据集。然后将第III-C节中描述的算法应用于，而不是原始数据集。

图4比较了使用和不使用CFA图像分解和训练样本选择程序的方法的去噪结果。第一行显示平滑区域的结果，第二行显示边缘和平滑背景之间边界的结果。我们可以清楚地看到，图像分解和训练样本选择的过程可以显着提高去噪和去马赛克图像的视觉质量。

E. Summary of the Algorithm

所提出的基于空间自适应PCA的CFA去噪算法总结如下。

1.估算红色，绿色和蓝色通道的噪声标准偏差，和。（关于CFA图像的噪声估计，参见第IV-C节。）

2.使用（3-16）和（3-17）将嘈杂的CFA图像分解为和。将以下去噪步骤3和4应用于。

3.设置可变块和训练块的大小。然后可以确定噪声协方差矩阵。

4.对于每个训练块：
执行训练样本选择程序（参见第III-D节）。
用表示所选的训练数据集。
使用（3-5）计算协方差矩阵;
估计信号的协方差矩阵为;
使用（3-8）分解并设置PCA变换矩阵;
将数据集转换为PCA域：;
通过将最后几行重置为零，将减少到（降维）;
使用（3-14）将的每一行收缩为;
将转换回时域为;
重新格式化以获得去噪的CFA块。
结束

5.用表示的去噪输出，最终的去噪图像是。

IV. EXPERIMENTAL RESULTS

为了全面测试所提出的基于PCA的CFA去噪算法的性能，我们在本节中进行了大量实验。进行了三种不同类型的实验。首先，我们评估其去噪输出与通过对CFA图像应用其他去噪方案进行比较。其次，与具有去马赛克方案的CFA去噪方案的联合评估是与许多去马赛克和去噪以及联合去噪和去马赛克算法相比较而进行的。最后，使用真实的原始CFA图像来说明所提出方法的性能。在下文中，我们将详细报告这些实验结果。

A. Assessment on CFA Denoising Result

我们首先评估所提出的关于去噪CFA图像的方法。两种复杂的基于小波的去噪技术[24]，[25]用于比较。由于两种去噪方案是针对单色图像而设计的，而不是针对CFA马赛克图像，我们在实验中通过两种情况对它们进行了测试。在第一种情况下，我们将CFA图像视为单色图像，并将去噪方案[24]，[25]直接应用于它们。在第二种情况下，我们将CFA图像划分为四个子图像（即，一个红色，一个蓝色和两个绿色通道子图像）并将它们用于去噪。在第二种情况下，每个子图像是真正的单色图像。在实验中，所提出的方法使用6×6像素的可变和去噪块以及30×30像素的训练块。在CFA图像分解和训练样本选择中，高斯平滑滤波器的比例设置为s = 3，阈值设置为T = 5。

已经使用各种测试图像检查了上述解决方案的性能，其中的示例在图5（a）和（b）中示出。这些分辨率为512x768像素的测试图像被广泛用于评估去马赛克算法的文献中以及众所周知的柯达数据集中最难的算法。按照标准实践，通过使用拜耳模式对测试图像进行采样来获得马赛克CFA图像。为了模拟具有通道相关传感器噪声的CFA数据，将高斯白噪声分别添加到马赛克图像的红色，蓝色和绿色通道中。为了对拟议方案与其他去噪方案进行公平评估，我们假设这些算法在标准偏差的等效信道无关高斯白噪声上工作，即在第一种情况下依赖于信道的噪声的平均能量。此外，我们评估静止和非平稳噪声情况，即当vr，vg和vb相等以及它们不相等时。 在后一种情况下，我们使每个通道的噪声水平与该通道的信号能量成比例。

由于原始CFA图像可用，因此可以使用相对于原始图像的去噪图像的峰值信噪比（PSNR）客观地评估不同去噪解决方案的性能。表I列出了在两个测试图像上获得的PSNR值。我们看到，所提出的基于空间自适应PCA的CFA去噪方案实现了最高的PSNR值，并且优于其他基于小波的去噪解决方案。还观察到，通过将整个大CFA图像分成四个单色子图像，可以改进[24]和[25]中的方案的去噪性能。图6通过显示去噪CFA图像和原始CFA图像之间的裁剪差异图像来说明CFA图像栅栏的去噪结果。

B. Joint Assessment on Denoising and Demosaicking Using Simulated CFA Images

有必要评估提出的与去马赛克联合的CFA去噪方案。由于所提出的方法在去马赛克之前执行去噪，因此将其与另外两种方法进行比较：去马赛克和联合去噪 - 去马赛克之后的去噪，其构成了用于抑制单传感器相机图像中的噪声的最先进方案。即，通过结合四种强大的去马赛克技术[4]，[9]，[10]，[15]和两种复杂的基于小波的去噪技术[24]，[25]的八种解决方案，从CFA数据中恢复全彩色信息顺序地，两个最近开发的联合去马赛克去噪方案[27]，[30]被用来进行比较。最后，竞赛由四名去马赛克去噪方法的代表完成，通过结合提出的CFA图像去噪方法和去马赛克技术[4]，[9]，[10]，[15]得到。

如在部分IV-A中，图5（a）和（b）中的示例图像用于实验中。将有噪声的CFA图像输入到竞争解决方案中以产生最终恢复的图像（即，具有抑制的噪声的去镶嵌的全色数据）。表II和表III列出了通过不同方案获得的PSNR值。对结果的详细检查表明，所提出的方法产生了非常好的结果并且优于许多现有方法。当使用四种去马赛克算法中的每一种时，所提出的基于PCA的CFA去噪方法几乎总是比具有相应去马赛克算法的基于小波的去噪方案提供更好的结果。提出的CFA去噪方法遵循去马赛克方法[9]比[27]和[30]中最近提出的两个联合去马赛克去噪方案表现更好。

除了良好的PSNR结果之外，所提出的方法的一个重要优点的特征在于最终恢复图像的高视觉质量。为了在展示使用各种解决方案恢复的图像之间的视觉差异的同时节省空间，仅在纸张中显示最终全色图像的裁剪部分。原始分辨率的恢复图像可在http://www.comp.polyu.edu.hk/~cslzhang/PCA-CFA-Denoisin g.htm获得。由于[25]的去噪方案比[24]具有更高的视觉质量，
除了[25]中的去噪方法以及[9]，[10]和[15]中提出的强大的去马赛克技术之外，还进行了去马赛克方法后的去噪输出的视觉比较。
为了显示去马赛克方法之前去噪的输出，提出的CFA去噪方案与[9]的去马赛克解决方案相结合，因为它在去马赛克方案中给出了最好的结果[4]，[9]，[10]，[15] ]。联合去马赛克去噪解决方案[27]，[30]的输出都用于视觉比较。

图7（a）示出了测试图像栅栏的部分，图7（b）示出了模拟的噪声CFA图像。噪声水平为，，。图7（c） - （e）分别显示了通过将[25]的去噪方法应用于[9]，[10]和[15]的去马赛克方法的输出而恢复的图像。图7（f）和（g）分别对应于[27]和[30]的联合去马赛 - 去噪方法的输出。图7（h）是通过提出的基于PCA的自适应CFA去噪方法的重建图像，其后是去马赛克方法[9]。图8（a） - （f）示出了图7（c） - （h）中的恢复图像与图7（a）中的原始图像之间的色差图像。 类似地，图9示出了由房屋图像上的同一组方法产生的结果。

对图7-9中所示的恢复图像的详细检查表明，即使是[9]，[10]和[15]中复杂的去马赛克方法，通常也会产生许多噪声引起的颜色伪影，这些伪像抵抗随后的去噪处理。此外，对去镶嵌图像进行去噪会使边缘模糊，并且可以在抑制颜色伪影的同时去除细节。 当通过联合去马赛克去噪恢复图像时，[30]中的方法比[27]表现得更好，但这两种方法都遭受了许多颜色伪影。与现有解决方案相比，通过直接对CFA数据执行去噪操作，所提出的基于空间自适应PCA的去噪方法在去除噪声方面更有效，因此显着减少了恢复图像中传感器噪声引起的颜色伪影的量。除了改进的降噪性能外，所提出的方法还能很好地保留精细的图像结构，并在测试的解决方案中实现最佳的视觉质量。 这可以在图7和图8中的栅栏图像的草部分以及图9中的房屋图像的窗板部分中清楚地看到。
请注意，这些详细结构在对抗噪声时会被其他方案过度平滑，但在使用建议的基于自适应PCA分析的方法时会保留。

C. Experiments on Real CFA Image

除了测试图像之外，所考虑的方法也应用于真实的单传感器捕获图像。图5（c）示出了在相对弱的照明下从数字电影摄像机捕获的帧裁剪的具有500×800像素大小的示例CFA图像分辨率。使用所提出的方法对这样的真实CFA图像进行去噪需要根据CFA数据计算每个通道的噪声能量。具体地，我们将NxM CFA图像划分为四个N / 2xM / 2子图像（两个绿色子图像，一个红色子图像和一个蓝色子图像）。然后，我们将一阶段正交小波变换[18]应用于每个子图像。噪声水平可以估计为[19]或，其中W是第一阶段的对角子带。 （这里使用第二种方法。）对于绿色通道，噪声水平是来自两个绿色子图像的的平均值。图10显示了与先前实验相同的实验设置的结果。不难看出，在现实成像场景中，所提出的方法在视觉质量方面优于竞争对手。这是因为所提出的方法有效地使用空间和光谱图像特性来同时抑制噪声并保留边缘和精细细节。在所提出的“去噪第一和去马赛克”方案中删除了“去马赛克第一和去噪后”方案中的许多噪声引起的颜色伪影以及联合去马赛克和去噪方案，而在所提出的方案中保留了在其他方案中平滑的许多边缘。

V. CONCLUSION

本文介绍了一种基于PCA的CFA图像去噪方案，用于单传感器数码相机成像应用。 为了在去噪过程中充分利用CFA传感器读数的空间和光谱相关性，通过使用支撑窗口来构建来自不同颜色通道的马赛克样本来构建向量变量，其统计量被计算以找到PCA变换矩阵。 尺寸减小和LMMSE在PCA转化域中进行以抑制CFA图像噪声。通过反转PCA变换以产生增强的CFA图像来完成去噪过程。 为了进一步降低平滑区域内的噪声残差和边缘与背景边界周围的幻象伪影，在PCA变换之前引入了CFA图像分解和训练样本选择的过程。 所提出的直接CFA图像去噪方案，随后是后续去马赛克方案，显着地减少了去马赛克图像中的噪声引起的颜色伪影。 这些伪像经常出现在许多去马赛克第一和去噪后期方案的输出全色图像以及一些联合去马赛克去噪方案中。 在抑制噪声的同时，所提出的方案很好地保留了图像中的精细结构，这些结构通常被其他去噪方案平滑。