A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance论文总结及翻译

A Comprehensive Tool for Modeling CMOS Image-Sensor-Noise Performance

Author: Ryan D. Gow
Link: https://ieeexplore.ieee.org/document/4215175/metrics#metrics
Select: ⭐️⭐️⭐️⭐️
Type: Academic Journal
备注: CMOS图像传感器噪声性能建模的综合工具

总结

该文章对CMOS传感器噪声进行了分析和建模，将其建模为一下几种噪声的叠加。

1. 光子散粒噪声（Photon-Shot-Noise）

   由于光的量子性质，光子在光电二极管中收集的数量存在不可避免的不确定性，这被称为光子噪声。

   遵循泊松分布，方差等于平均到达率

2. 光响应不均匀性（Photoresponse Non-Uniformity，PRNU）

   PRNU（Photoresponse Non-Uniformity）描述的是由于工艺变异，不同像素对光的响应不同的事实[2]。这通常被认为是一个像素增益不匹配的现象，因此PRNU被归类为信号相关的固定模式噪声。

   正态分布可以很好地近似实际数据，除非在极低或者极高的光强度下，否则该噪声可以近似为信号无关的。

3. 像素间串扰(crosstalk)

   像素间串扰(crosstalk)由两种不同的机制引起：光学和扩散串扰。由于我们的模型不旨在模拟光学效应，因此只考虑传感器中由扩散引起的串扰。这种现象会导致颜色混合、图像模糊，并在颜色重建后降低信噪比(SNR)

   传感器中的交叉干扰会导致不同通道之间的颜色混合。因此，颜色重建过程必须通过应用适当的白平衡和颜色矩阵进行补偿.

4. 暗电流（DARK CURRENT）

   暗电流生成中心在硅晶中随机分布（如图5）。换句话说，不同像素产生不同量的暗电流，这被称为暗电流非均匀性（DCNU）。这会表现为固定模式的曝光相关噪声。还有一个与时间相关的噪声，称为暗电流shot噪声（DCSN）

   • DCNU：通过向logistic分布中加入均匀分布实现建模
   • DCSN：与光子散粒噪声建模一致

5. 像素电路噪声（IXEL CIRCUIT NOISE）

   光电二极管外部存在的噪声取决于电路设计和处理技术的使用。这包括热噪声、闪烁噪声、随机电报信号（RTS）、行噪声和列噪声。这些类型的噪声特别重要进行建模：热噪声设置了基本的时间噪声底线，随着像素尺寸的缩小，闪烁和RTS变得越来越重要，而在图像中出现的行或列的伪影特别明显。

   • 热噪声：零均值高斯分布，因此建模热噪声只需要生成具有适当标准差的正态分布即可。
   • 低频“闪烁”噪声（称为1/f）和RTS：
     • 这是由于硅晶格中陷阱的作用，这些陷阱随机地捕获和发射载流子[28]。这调制了通道电导，导致噪声信号的产生。
     • 使用Fisher–Tippet–Gumbel分布建模，并叠加均匀分布（类似暗电流）
   • 行噪声：遵循高斯分布。
   • 列噪声：
     • 时间列噪声：采用FTG分布来模拟时间列噪声
     • 垂直固定模式噪声：高斯分布，为了创建固定模式噪声，列值在多个帧上保持不变
   • ADC量化噪声：适当的缩放和量化进行处理可以模拟

全文

Abstract—Accurate modeling of image noise is important in understanding the relative contributions of multiple-noise mechanisms in the sensing, readout, and reconstruction phases of image formation. There is a lack of high-level image-sensor system modeling tools that enable engineers to see realistic visual effects of noise and change-specific design or process parameters to quickly see the resulting effects on image quality. This paper reports a comprehensive tool, written in MATLAB, for modeling noise in CMOS image sensors and showing the effect in images. The tool uses accepted theoretical/empirical noise models with parameters from measured process-data distributions. Output images from the tool are used to demonstrate the effectiveness of this approach in determining the effects of various noise sources on image quality.

摘要—准确建模图像噪声对于了解多种噪声机制在感测、读出和重建图像形成阶段的相对贡献非常重要。目前缺乏高级图像传感器系统建模工具，使工程师能够看到噪声的真实视觉效果，并更改特定的设计或工艺参数以快速查看对图像质量的影响。本文报道了一个全面的工具，使用MATLAB编写，用于建模CMOS图像传感器中的噪声并展示其对图像的影响。该工具使用经验和理论噪声模型，参数来自于测量的工艺数据分布。该工具输出的图像用于展示该方法在确定各种噪声源对图像质量的影响方面的有效性。

I. INTRODUCTION

AN ACCURATE model of CMOS image-sensor-noise performance is useful in optimizing sensor design and setting targets, ultimately leading to improvements in image quality. Many attempts at modeling CMOS image-sensor noise are based on over-simplistic assumptions and may not incorporate a sufficient variety of noise sources to yield a realistic simulator [1]–[4]. Although noise can be quantified using various metrics, such as the signal-to-noise ratio (SNR), these are not necessarily a meaningful reflection of how camera users will perceive the image quality. These issues inspire the creation of a unified and accurate sensor-noise model capable of displaying noisy end-images. In this paper, we present such a tool.

准确的CMOS图像传感器噪声性能模型有助于优化传感器设计和设置目标，最终导致图像质量的改进。许多建模CMOS图像传感器噪声的尝试基于过度简化的假设，可能没有包含足够多的噪声源以产生真实的模拟器[1]–[4]。虽然噪声可以使用各种指标进行量化，例如信噪比（SNR），但这些指标不一定是相机用户感知图像质量的有意义的反映。这些问题启发了创建一个统一而准确的传感器噪声模型，能够显示带有噪声的最终图像。本文介绍了这样的工具。

First, CMOS image-sensor noise is introduced along with the general modeling approach used in this paper. Next, we discuss the techniques used to model each source of noise associated with charge collection. This includes photon shot noise, photoresponse nonuniformity (PRNU), crosstalk, and dark current [5]. Emphasis is given to the treatment of non-Gaussian random noise: Instead of simply assuming a Gaussian distribution for all random noise, we offer more representative distributions where appropriate. Then, the modeling of circuit noise is discussed. This includes column, row, low-frequency, thermal, and quantization noises. Finally, an overview of the complete tool is presented, and its usefulness is demonstrated with some sample applications.

首先介绍了CMOS图像传感器噪声以及本文使用的一般建模方法。接下来，我们讨论了用于模拟与电荷收集相关的每个噪声源的技术。这包括光子散粒噪声、光响应不均匀性（PRNU）、串扰和暗电流[5]。重点放在非高斯随机噪声的处理上：我们不仅对所有随机噪声简单地假设高斯分布，而是在适当的情况下提供更具代表性的分布。然后讨论了电路噪声的建模，包括列、行、低频、热和量化噪声。最后，介绍了完整工具的概述，并展示了一些示例应用程序来证明其有用性。

II. NOISE SOURCES AND TYPES

The imperfections or “errors” found in real images result from the optics, sensor, and color processing chain. In this paper, only artefacts associated with the CMOS sensor are considered. These may be in the form of fixed offsets or may be random noise, in which pixel values exhibit statistical variation with respect to time and/or location. Clearly, the temporal and time invariant components—known as fixed-pattern noise (FPN)—are indistinguishable if the system is sampled at only one instant in time, as in still images. Depending on its nature, the noise may also vary with temperature and signal level [2], [6], [7]. Our model accounts for these dependencies where necessary. Fig. 1 shows where the main noise sources occur in the pipeline of the four-transistor (4T) active pixel sensor on which this paper is based [8].

实际图像中发现的不完美或“错误”来自于光学、传感器和颜色处理链。本文只考虑与CMOS传感器相关的伪影。这些可能采取固定的偏移量形式，也可能是随机噪声，其中像素值相对于时间和/或位置呈现统计变化。显然，如果系统只在一个时间点采样，例如在静止图像中，那么这些随时间不变的固定模式噪声（FPN）无法区分。根据其性质，噪声也可能随着温度和信号水平的变化而变化[2]，[6]，[7]。我们的模型在必要时考虑了这些依赖关系。图1显示了基于这篇论文的四晶体管（4T）有源像素传感器管道中主要的噪声源的位置[8]。

Fig. 1. Main noise sources in the CMOS imager pipeline (adapted from [5]). Note that photodiode kTC noise is not modeled in this paper as our model refers to 4T pixels, which allow the effect of this form of noise to be eliminated.
图1。CMOS成像器管线中的主要噪声源（改编自[5]）。注意，本文中没有对光电二极管kTC噪声进行建模，因为我们的模型涉及4T像素，这允许消除这种形式的噪声的影响。

III. PERFECT IMAGES

To accurately evaluate the impact of noise on image quality, it is essential that the noise is added to initially noiseless images. Lifelike scenes are needed for evaluating the perceptual impact of noise in end-applications. Hence, we utilize raytracing software [9] to generate noiseless realistic scenes. In addition, photographs obtained using a very high-quality digital camera may provide a sufficiently high SNR to be considered “perfect” for practical purposes. The model is capable of handling monochrome and color images. For the latter, we assume that the sensor utilizes a Red-Green-Blue Bayer-patterned color filter array (CFA) [10].

为了准确评估噪声对图像质量的影响，必须将噪声添加到最初无噪声的图像中。对于评估噪声在最终应用中的感知影响，需要真实场景。因此，我们利用光线追踪软件[9]生成无噪声的逼真场景。此外，使用高质量数字相机拍摄的照片可能具有足够高的信噪比，被视为在实际应用中“完美”的图像。该模型能够处理黑白和彩色图像。对于后者，我们假设传感器采用红绿蓝 Bayer-patterned彩色滤光片阵列（CFA）[10]。

Whichever image is selected for use in the model, its pixel values are normalized to represent electrons in the photodiode. This is determined by the user specifying the peak number of photo-generated electrons per second and the integration time and full-well capacity of the pixel. We, therefore, have the ability to control a wide variety of input scenes to which noise can be added in order to achieve a realistic image.

无论选择哪种图像用于模型，其像素值都被归一化为代表光电二极管中的电子。这由用户指定每秒光产生的电子峰值数量、积分时间和像素的最大电荷容量来确定。因此，我们有能力控制各种输入场景，可以向其中添加噪声以获得逼真的图像。

IV. MODELING RANDOM NOISE(随机噪声建模)

An $m\times n$ noisy image for one type of random noise can variable (rv). Several sources of random noise, therefore, can generally be considered as $m\times n$ runs of a single random be modeled by superposing $m\times n$ realizations of appropriate stochastic processes. Each rv has a corresponding probability distribution function (pdf), which contains the most important information about the variable. Hence, modeling random noise in images entails using suitable probability distributions to generate matrices of noise values.

针对一种随机噪声变量（random noise can variable, rv），大小为 $m\times n$的噪声图像可以被建模为 $m\times n$个适当随机过程的实现叠加。因此，多种随机噪声源通常可以通过叠加适当随机过程的 $m\times n$个实现来进行建模。每个rv都有相应的概率分布函数（probability distribution function, pdf），其中包含了关于变量的最重要信息。因此，图像中的随机噪声建模需要使用合适的概率分布来生成噪声值矩阵。

V. CHARGE COLLECTION AND ASSOCIATED NOISE（电荷收集和相关噪声）

A. Photon-Shot-Noise Model

Due to the quantum nature of light, there is an in- escapable uncertainty in the number of photons collected in the photodiode—this is termed photon shot noise. The number of photons incident in a given observation window is known to follow the Poisson distribution, with the variance equal to the mean arrival rate [11], [12]. Thus, photon shot noise can be simulated by reading each pixel value in the perfect input image and generating a random number using the Poisson distribution (or normal distribution for large pixel values > 1000 [13]), with the mean arrival rate equal to that pixel value. The resulting matrix represents the image incorporating photon shot noise.

由于光的量子性质，光子在光电二极管中收集的数量存在不可避免的不确定性，这被称为光子噪声。在给定的观察时间窗口中入射的光子数据已知，其遵循泊松分布，方差等于平均到达率[11]，[12]。因此，可以通过读取完美输入图像中的每个像素值并使用泊松分布（或对于大于1000的像素值，使用正态分布[13]）生成随机数，其中平均到达率等于该像素值来模拟光子散粒噪声。生成的矩阵代表了具有光子散粒噪声的图像。

B. PRNU Model

PRNU describes the fact that not all pixels have the same response to light due to process variations [2]. This is usually considered as a pixel gain-mismatch phenomenon, so PRNU is classed as a signal-dependent FPN. It is typically quoted as a percentage of the mean signal value. Fig. 2 demonstrates the signal-dependent nature of PRNU for a recent color-image sensor. This form of noise is becoming increasingly significant as the trend toward smaller pixels makes fabricating identical pixels ever more difficult [14].

PRNU（Photoresponse Non-Uniformity）描述的是由于工艺变异，不同像素对光的响应不同的事实[2]。这通常被认为是一个像素增益不匹配的现象，因此PRNU被归类为信号相关的FPN。它通常以平均信号值的百分比为单位。图2展示了最近一款彩色图像传感器的PRNU的信号相关性质。随着越来越多的厂商采用更小的像素，制造相同像素变得越来越困难，这种噪声的重要性也越来越大[14]。

Fig. 2. PRNU at different light levels for a prototype sensor.原型传感器在不同光照水平下的PRNU。

In modeling PRNU, it is necessary to consider the variation in output pixel values across the image array. Thus, a histogram of pixel values from the difference frame used to measure PRNU was generated. By superposing a normal distribution with the same mean and standard deviation as the measured data, we were able to obtain a reasonable fit. Therefore, given that the normal distribution approximates the actual data reasonably well, this distribution was used in the model. A gain map is generated using a normal distribution with a mean of unity and a standard deviation equal to the measured PRNU. This gain map is then applied on a pixel-by-pixel basis to the input image. The gain map remains fixed over multiple frames.

为了对 PRNU 进行建模，有必要考虑图像阵列中输出像素值的变化。因此，首先生成了用于测量 PRNU 的差分帧像素值的直方图。通过在具有相同均值和标准差的正态分布上叠加测量数据，我们能够得到一个合理的拟合结果。因此，鉴于正态分布可以很好地近似实际数据，该分布被用于模型中。利用均值为 1，标准差等于测得的 PRNU 的正态分布生成增益映射。该增益映射会基于每个像素对输入图像进行调整。增益映射在多帧图像中保持不变。

The model is based on some assumptions. Notably, we ignore the signal dependence of the PRNU. Fig. 2 showed that the PRNU percentage variation with signal is approximately constant, except at extremely low or high light levels. It is, therefore, reasonable to treat the PRNU as being independent of signal. An example output is illustrated in Fig. 3, which shows the impact of 10% PRNU on an image of mean 1000 electrons.

该模型基于一些假设。特别地，我们忽略了PRNU的信号依赖性。图2显示，PRNU的百分比变化随信号近似恒定，除非在极低或极高的光强水平下。因此，将PRNU视为与信号无关是合理的。图3展示了10% PRNU对平均电子数为1000的图像的影响的示例输出。

C. Crosstalk Model

Crosstalk between pixels is caused by two different mechanisms: optical and diffusion crosstalks. Since our model does not aim to simulate optical effects, only the crosstalk due to diffusion in the sensor is considered. This phenomenon causes color mixing, image blur, and degrades the SNR after color reconstruction [15]. Like PRNU, crosstalk is becoming an increasingly significant problem, given the trend toward smaller pixel dimensions.

像素间串扰(crosstalk)由两种不同的机制引起：光学和扩散串扰。由于我们的模型不旨在模拟光学效应，因此只考虑传感器中由扩散引起的串扰。这种现象会导致颜色混合、图像模糊，并在颜色重建后降低信噪比(SNR) [15]。像PRNU一样，由于像素尺寸越来越小，串扰问题正在变得越来越严重。

To evaluate diffusion crosstalk, a single pixel should be exposed to light, and the resulting values for that pixel and its neighbors—which are shielded from light by a metallization layer—must be measured. The signal measured in each neighboring pixel can then be expressed as a percentage of the total signal, thus providing a measure of the crosstalk in each direction. The results of such a characterization are shown in Fig. 4 for a pixel implemented in 0.18-µm CMOS image-sensor technology.

为了评估扩散串扰，应该暴露单个像素光，并测量该像素及其邻居的结果值，这些邻居由金属化层遮挡，因此不受光照。然后，可以将测量的每个相邻像素中的信号表示为总信号的百分比，从而提供每个方向上串扰的度量。这样的表征结果如图4所示，对于实现在0.18微米CMOS图像传感器技术中的像素。

Fig. 4. Example measurement result for crosstalk under Green illumination.

Given these parameters, it is possible to simulate crosstalk as follows. 1) Determine the proportion of original pixel value remaining after experiencing crosstalk (i.e., by scaling by 78.4% in the example above). 2) Then, add the crosstalk contribution from the surrounding pixels. The process is repeated for each pixel in the input image to generate a corresponding output image with crosstalk.

根据这些参数，可以按照以下步骤模拟色彩串扰：1）确定经历色彩串扰后保留的原始像素值的比例（例如在上述示例中按78.4%进行缩放）；2）然后加上周围像素的色彩串扰贡献。对于输入图像中的每个像素，重复此过程以生成相应的带有色彩串扰的输出图像。

The crosstalk parameters are modeled as being constant for a given run of n frames resulting in scene-dependent crosstalk. It is assumed that the crosstalk is constant across the pixel array, which in reality will not be the case due to variation in chief ray angle, which changes the location of the photo generation inside the pixel. Furthermore, the model assumes that the number of crosstalk electrons is always linearly proportional to the pixel values. In reality, this may not be true, but real sensor data was unavailable to investigate this more thoroughly.

所使用的crosstalk参数被建模为在一次运行的n帧中保持不变，这导致了与场景有关的crosstalk。模型假设crosstalk在像素阵列上是恒定的，而实际上由于主光线角度的变化，导致光子生成的位置在像素内的变化，因此这并不是事实。此外，该模型假设crosstalk电子数始终与像素值成线性比例关系，但在实际情况下可能并不完全准确，但是由于缺乏真实的传感器数据，无法进行更深入的研究。

Crosstalk in sensors featuring a CFA results in color mixing between the channels. Hence, the color-reconstruction process (see Section IX) must compensate by applying an appropriate white balance and color matrix (details of which can be found elsewhere [16]–[18]).

传感器中的交叉干扰会导致不同通道之间的颜色混合。因此，颜色重建过程（详见第九节）必须通过应用适当的白平衡和颜色矩阵进行补偿（详细信息可以在其他地方找到[16]–[18]）。

VI. DARK CURRENT

Dark-current generation centers are randomly distributed spatially in the silicon (Fig. 5). In other words, different pixels produce differing amounts of dark current, and this is termed the dark-current nonuniformity (DCNU). This manifests itself as a fixed-pattern exposure-dependent noise. There is also a temporal noise associated with it known as the dark-current shot noise (DCSN) [19]–[22].

暗电流生成中心在硅晶中随机分布（如图5）。换句话说，不同像素产生不同量的暗电流，这被称为暗电流非均匀性（DCNU）。这会表现为固定模式的曝光相关噪声。还有一个与时间相关的噪声，称为暗电流shot噪声（DCSN）[19]-[22]。

Fig. 5. Shifted and scaled histograms illustrating the dark-current distribution at different temperatures (for a sensor taken at 1-s exposure). Note that the shape, as well as the location of the distribution, varies with temperature.
图5。显示不同温度下暗电流分布的移位和缩放直方图（对于曝光1秒的传感器）。注意，形状以及分布的位置随温度而变化。

The task of modeling dark current in images, therefore, requires generating DCNU arrays with realistic statistical properties and, then, given that the DCSN follows the Poisson distribution, applying a shot noise to each pixel.

因此，建模图像中的暗电流需要生成具有现实统计特性的DCNU阵列，然后由于DCSN遵循泊松分布，对每个像素应用shot噪声。

A. Modeling DCNU

Some previously published attempts assume that the DCNU is Gaussian distributed [2], [3], but examination of sensor- characterization data shows that this is a flawed assumption. Indeed, from Fig. 6, it is clear that a tiny proportion of pixels in the array show extremely high levels of dark current. Such pixels will rapidly saturate, appearing as white dots in a low- light scene, which considerably degrades the perceived-image quality. Accounting for these “white” pixels is, therefore, of paramount importance, particularly as when two white pixels are adjacent in an image a couplet results, which is difficult to remove by signal processing. Couplets are a key driver in the manufacturing yield of high-volume image sensors [21].

一些之前发表的研究尝试假设DCNU（dark current nonuniformity）符合高斯分布[2]，[3]，但传感器特性数据的检查表明这是一个错误的假设。事实上，从图5可以看出，阵列中极少数的像素显示出极高水平的暗电流。这些像素将迅速饱和，在低光场景中呈现为白点，这会严重降低感知图像的质量。因此，考虑到这些“白色”像素是非常重要的，特别是当图像中有两个相邻的白色像素时，会产生一个难以通过信号处理去除的couplet。Couplet是大规模图像传感器制造的一个关键因素[21]。

Fig. 6. Concept of using two distributions to achieve a more realistic end-histogram.

Furthermore, the generation of dark current is highly temperature-dependent, doubling approximately every 6 ◦C– 8 ◦C in crystalline silicon [20]. In order to reflect realistic operating conditions, characterization at multiple temperatures is needed (see Fig. 6). Very leaky pixels appear to have a lower temperature coefficient than less leaky pixels. This is likely due to different physical mechanisms behind the dark current [22].

此外，暗电流的产生高度依赖于温度，在晶体硅中，大约每6◦C-8◦C会翻倍[20]。为了反映实际的工作条件，需要在多个温度下对其进行特性化（参见图5）。非常泄漏的像素似乎比较不泄漏的像素具有较低的温度系数。这可能是由于暗电流背后的不同物理机制所致[22]。

1)Simulating DCNU at 25 ◦C: We initially sought to fit the entire dark-current histogram to one of the many available probability distributions. However, all of the “fits” were visibly poor. This is not surprising: Difficulty in accounting for both the extreme outliers, as well as accurately following the main body of the distribution, is undoubtedly the reason why some previous attempts have succumbed to using the normal distribution. In modeling the DCNU, we first propose a distribution for the main body region and account for the outliers separately.

1. 在 25℃ 下模拟 DCNU：我们最初试图将整个暗电流直方图拟合为许多可用概率分布之一。然而，所有“拟合”都明显不佳。这并不奇怪：难以同时考虑到极端异常值以及准确地跟随分布的主体部分，这无疑是先前某些尝试采用正态分布的原因。在模拟 DCNU 时，我们首先提出一个适用于主体区域的分布，并将异常值单独考虑。

When fitting a distribution to sample data, MATLAB returns the best fit parameters for that distribution and a log- likelihood value [23], [24]. Using this metric along with visual comparisons, the logistic distribution was selected to model the shape of the main body of the dark-current histogram at 25 ◦C. This distribution is an alternative to the normal distribution, where a higher proportion of the population is distributed in the tails [25].

当将一个分布拟合到样本数据时，MATLAB会返回该分布的最佳拟合参数和对数似然值。结合视觉比较，我们选用了logistic 分布来模拟25℃下暗电流直方图主体部分的形状。logistic 分布是正态分布的一种替代方案，其中更高比例的人群分布在尾部区域。

It is now necessary to consider how to model the ultra noisy white pixels. From a histogram perspective, this means finding a method to extend the right-hand tail. Since one all-encompassing distribution could not be found, a superposition of the distributions was considered. The uniform distribution [13] was considered the most intuitive for this purpose, since its rectangular-shaped pdf can add to the logistic pdf, as shown in Fig. 7.

现在需要考虑如何建模极高噪声的白色像素。从直方图的角度来看，这意味着找到一种方法来扩展右侧尾部。由于找不到一种全面的分布，因此考虑使用分布的叠加。均匀分布[13]被认为对于此目的最直观，因为其矩形形状的概率密度函数可以加到 logistic 概率密度函数中，如[图6]所示。

To avoid adding excessive noise to the main body, only a certain proportion of the uniformly distributed data must represent the high noise values and all other elements must be set to zero. In benchmarking sensors, an important metric is the parts per million (ppm) of pixels with the dark current greater than a given threshold, and we, therefore, incorporate this as an input parameter in the model. This technique was shown to produce realistic end-histograms.

为了避免向主体部分添加过多的噪声，只有一定比例的均匀分布数据代表高噪声值，而所有其他元素必须设置为零。在基准测试传感器时，一个重要的度量是每百万个像素中暗电流大于给定阈值的部分（ppm），因此我们将其作为模型的输入参数。这种技术被证明可以产生逼真的最终直方图。

2)Simulating DCNU at Other Temperatures: The DCNU at 45 ◦C and 65 ◦C was modeled using the same procedure as employed for 25 ◦C. However, we found that the inverse Gaussian [26] (also known as the Wald distribution) and log- normal [27] distributions were the best fits for the main body at these temperatures with a uniform distribution superimposed to model “white” pixels. Modeling performance at 25 ◦C, 45 ◦C, and 65 ◦C is adequate for most purposes, and the modular structure of our model allows for easy extension to other temperatures.

2）在其他温度下模拟DCNU：在45°C和65°C下，DCNU的建模使用与25°C相同的程序。但是，我们发现逆高斯分布（也称为Wald分布）和对数正态分布是这些温度下主体的最佳拟合分布，通过添加均匀分布来模拟“白点”像素。在25°C、45°C和65°C下的模拟表现已经足够满足大多数需求，而我们的模型的模块化结构使得可以轻松扩展到其他温度。

B. Modeling DCSN

Dark-current pixel values in any individual frame are affected by the associated shot-noise component. Therefore, to simulate an individual frame, it is necessary to apply shot noise to the DCNU array. This is exactly analogous to modeling photon shot noise on a perfect image array. Hence, we use the same algorithm as discussed in Section V-A.

任何单帧中的暗电流像素值都会受到相关的 shot-noise 组件的影响。因此，为了模拟单个帧，必须将 shot noise 应用于 DCNU 数组。这与在完美的图像阵列上建模光子 shot noise 完全类似。因此，我们使用与第 V-A 节中讨论的算法相同的算法。

VII. PIXEL CIRCUIT NOISE（像素电路噪声）

Noise present out with the photodiode depends on the circuit design and processing technology used. This includes thermal noise, flicker noise, random telegraph signal (RTS), row noise, and column noise. These types of noise are particularly important to model: Thermal noise sets the fundamental temporal noise floor, flicker and RTS are becoming increasingly significant as pixel dimensions shrink, and artefacts appearing as rows or columns in the image are particularly noticeable.

光电二极管外部存在的噪声取决于电路设计和处理技术的使用。这包括热噪声、闪烁噪声、随机电报信号（RTS）、行噪声和列噪声。这些类型的噪声特别重要进行建模：热噪声设置了基本的时间噪声底线，随着像素尺寸的缩小，闪烁和RTS变得越来越重要，而在图像中出现的行或列的伪影特别明显。

A. Modeling Thermal Noise

Thermal noise is present at the source follower, in the readout transistors, and in the analog-to-digital converter (ADC) of an imager. It is known to follow a zero-mean Gaussian distribution [20], so modeling thermal noise simply requires generating a normal distribution with appropriate standard deviation.

热噪声存在于图像传感器的源随器、读出晶体管和模数转换器中。它被认为是零均值高斯分布[20]，因此建模热噪声只需要生成具有适当标准差的正态分布即可。

B. Modeling Flicker/RTS Noise

Low-frequency “flicker” noise (known as 1/f) and RTS are crucial components of the source-follower noise. These are due to the action of traps in the silicon lattice which randomly capture and emit carriers [28]. This modulates the channel conductance, leading to a noise signal.

低频“闪烁”噪声（称为1/f）和RTS是源跟随器噪声的关键组成部分。这是由于硅晶格中陷阱的作用，这些陷阱随机地捕获和发射载流子[28]。这调制了通道电导，导致噪声信号的产生。

In analyzing the histogram of the temporal pixel noise for contemporary imagers, we observed that it appeared to have a “fat tailed” distribution. This is similar—but not nearly as extreme—as that observed with dark current, as detailed in Section VI of this paper. It is the extremely noisy pixels, which are most disturbing, that, in this case, manifest themselves as flickering dots in a video sequence. Since we know that the thermal component of pixel circuit noise is Gaussian distributed, completing the pixel-circuit-noise model thus requires finding a suitable distribution for the flicker and RTS components.

在分析当今图像传感器的时间像素噪声直方图时，我们观察到其具有“厚尾”分布。这类似于本文第 VI 部分所述的暗电流的分布，但远没有那么极端。在这种情况下，最令人不安的是极端嘈杂的像素，它们会在视频序列中显示为闪烁的点。由于我们知道像素电路噪声的热成分是高斯分布的，因此完成像素电路噪声模型需要找到适合闪烁和 RTS 成分的分布。

However, flicker and RTS are not fully understood phenomena, and the notion that flicker noise can be regarded as a superposition of single-carrier RTS has been debated in the literature [28]–[31]. No definitive understanding exists of how flicker noise or RTS values are distributed across an array of transistors. Nonetheless, a connection has been made between the fluctuation of flicker noise and the field of extreme value statistics. Specifically, it has been shown that voltage fluctuations in gallium arsenide (GaAs) films, which exhibit a 1/f power spectrum, following the Fisher–Tippet–Gumbel (FTG) distribution [32]–[35].

然而，脉动噪声和 RTS 并非完全理解的现象，而且在文献中有争议，认为脉动噪声可以被视为单载流子 RTS 的叠加 [28]–[31]。没有明确的了解如何在晶体管阵列中分布脉动噪声或 RTS 值的认识。尽管如此，脉动噪声的波动与极端值统计学领域有关。具体而言，已经表明，在展现 1/f 幅度谱的砷化镓 (GaAs) 膜中，电压波动遵循Fisher–Tippet–Gumbel (FTG) 分布 [32]–[35]。

This inspired the use of the FTG distribution (also known as the log-Weibull or type-1 extreme value) to simulate the long tails of the flicker-noise histogram. We found that the FTG distribution could not yield a good fit to both the main body region of the histogram and the extreme tail values. Therefore, as when modeling dark current (see Section VI), we make use of a superposition with the uniform distribution. This enables us to obtain appropriate tail values. The resulting histogram represents the fact that different pixels have different levels of temporal noise when measured over many frames.

这启发我们使用 Fisher–Tippet–Gumbel（FTG）分布（也称为对数 Weibull 或类型1极值）来模拟 flicker-noise 直方图的长尾。我们发现，FTG 分布无法很好地拟合直方图的主体区域和极端尾部值。因此，与模拟暗电流时相同（参见第 VI 节），我们使用均匀分布进行叠加。这使我们能够获得适当的尾部值。结果直方图表示不同像素在多帧上测量时具有不同级别的时间噪声。

This technique provides a reasonably accurate method of modeling the characterization data, as shown in Fig. 8. Al-though the simulated data does not rolloff like the characterization data, the ppm count is the same—something which is in fact more important than the absolute value of the noise, since the differences are likely to be imperceptible in a typical image.

该技术提供了一种相当准确的建模方法，如图7所示。虽然模拟数据不像表征数据那样下降，但ppm计数相同——实际上比噪声的绝对值更重要，因为这些差异在典型图像中可能是不可察觉的。

Fig. 7. Outline of a temporal noise distribution measured on a prototype sensor, and a fit to the real data using the FTG and uniform distributions.

The distribution shown in Fig. 7 is the variation in random temporal noise across the array. It does not mean that the random temporal noise for a specific pixel varies in time according to such a distribution. However, since no definitive understanding could be found on the statistical properties of flicker and RTS per pixel, these components could not be separated and parameterized individually. Our model, therefore, assumes that the overall flicker/RTS in each pixel obeys a zero-mean normal distribution, with the standard deviation taken from the FTG/uniform distribution. This is a simplification of the real behavior. The normal distribution thus provides the necessary temporal variation, albeit through a simple assumption.

图7所示的分布是整个阵列中随机时间噪声的变化。这并不意味着特定像素的随机时间噪声按照这样的分布随时间变化。然而，由于在像素级别上没有找到关于闪烁和RTS的统计特性的确定性认识，因此这些组成部分不能单独参数化。因此，我们的模型假定每个像素中的整体闪烁/RTS遵循零均值正态分布，其标准差取自FTG/均匀分布。这是实际行为的简化。因此，正态分布提供了必要的时间变化，尽管是通过一个简单的假设。

C. Modeling Row Noise

When a given row is released from reset, all pixels in that row are exposed to noise entering through the reset line, transfer gate, or read transistor. This appears in images as horizontal lines and has fixed and temporal components. Normally, the fixed component can be neglected, as fixed offsets between color channels can be removed by subsequent color reconstruction. Hence, our model only considers the temporal row noise.

当给定的行从复位状态释放时，该行中的所有像素都会暴露于通过复位线、传输门或读取晶体管进入的噪声。在图像中，这会表现为水平线，并且具有固定和暂态分量。通常情况下，可以忽略固定分量，因为颜色重建后可以消除通道之间的固定偏移。因此，我们的模型仅考虑暂态行噪声。

The distribution of row noise was measured and was found to follow a Gaussian distribution; therefore, it can be modeled by generating a random value per row and applying it to all the pixels in that row. As this noise is time varying, a new value is generated for each row in each frame to be simulated.

行噪声的分布已经被测量，并发现其遵循高斯分布；因此，它可以通过为每一行生成一个随机值并将其应用于该行中的所有像素来进行建模。由于这种噪声是时变的，因此需要为要模拟的每一帧中的每一行生成一个新值。

D. Modeling Column Noise Noise

Noise appearing in vertical lines is also highly noticeable, although the mechanism is different to that of row noise. Our model is based on an in-column ADC architecture, and so, temporal column noise includes thermal and 1/f noise in the column amplifiers, although the biggest component is introduced by the sample and hold capacitors during reset. This is known as column-level “kTC” noise.

出现在垂直线上的噪声也非常明显，虽然其机制与行噪声不同。我们的模型基于列式ADC架构，因此，暂态列噪声包括列放大器中的热噪声和1/f噪声，但最大的成分是在复位期间由采样保持电容器引入的。这被称为列级“kTC”噪声。

Vertical FPN (VFPN) is caused by mismatch between column amplifiers, leading to offset (signal-independent) and gain (signal-dependent) components. Through careful design and fabrication, differences in column gain can be reduced, which means that the offset component is more significant. Thus, the signal dependence may be neglected to a first order.

竖直固定模式噪声（VFPN）由列放大器之间的不匹配引起，导致偏移（与信号无关）和增益（与信号有关）组成部分。通过精心的设计和制造，可以减小列增益的差异，这意味着偏移成分更为显著。因此，可以在一定程度上忽略信号依赖性。

1)Temporal Column Noise: Our measurements revealed that the histogram of temporal column noise is right skewed, indicating that a few columns are much noisier than others, similarly to the pixel source-follower noise. It is, therefore, intuitive to consider applying the FTG distribution to model temporal column noise. By careful choice of input parameters, we were able to match the mean and also the outliers of the measured distribution rather well (no uniform-distribution addition was required). The simulation data is then generated in an algorithm similar to that used for temporal row noise, only the random data is arranged in columns and is derived from the FTG distribution.

1）时间列噪声：我们的测量结果表明，时间列噪声的直方图呈现右偏态，表明少数列比其他列噪声更大，与像素源跟随器噪声类似。因此，考虑采用FTG分布来模拟时间列噪声是直观的。通过精心选择输入参数，我们能够很好地匹配所测量分布的平均值和极端值（不需要添加均匀分布）。然后，模拟数据按照类似于时间行噪声的算法生成，只是随机数据按列排列，并从FTG分布中派生。

2)Vertical FPN (VFPN): The fixed-pattern component of column noise is known to be more significant than the temporal component, and fixed vertical lines are easily perceptible in video sequences. Once more, examination of the histogram revealed that the distribution of VFPN was Gaussian. It was modeled by generating a single value per column of pixels in the array and adding each value to the pixels in that column. To create an FPN, the column values remain fixed over multiple frames.

垂直固定模式噪声（VFPN）：列噪声的固定模式分量比暂态分量更显著，固定的垂直线在视频序列中很容易被察觉。再次检查直方图后发现，VFPN的分布是高斯分布。通过为阵列中的每列像素生成单个值并将每个值添加到该列的像素来对其进行建模。为了创建FPN，列值在多个帧上保持不变

VIII. ANALOG-TO-DIGITAL CONVERSION

It is now appropriate to move further through the imaging chain to consider the quantization noise introduced by the ADC. Our model is based on a column parallel ADC architecture [8], [36].

现在适合进一步考虑由ADC引入的量化噪声。我们的模型基于列并行ADC结构。

The digital ramp counter in column ADCs typically has a constant step size, resulting in linear quantization of the input signal. Since we aim to simulate the impact of quantization on end-images—not to develop a circuit model of the ADC—it is valid to simply view the ADC as a transfer function of electrons to codes. Modeling, therefore, entails scaling pixel electron values into an appropriate code-range set by the resolution of the ADC by appropriately scaling and quantizing.

列式 ADC 中的数字斜坡计数器通常具有恒定的步长，从而导致输入信号的线性量化。由于我们的目标是模拟量化对最终图像的影响，而不是开发 ADC 的电路模型，因此可以将 ADC 简单地视为将电子转换为代码的传递函数。因此，建模涉及将像素电子值缩放到由 ADC 分辨率设置的适当代码范围内，通过适当的缩放和量化进行处理。

IX. NOISE MODEL:OVERVIEW AND APPLICATION

A. Model Structure Each

Each of the individual noise sources is modeled using a separate MATLAB function, and so, they can be linked using a modular approach in order to produce a comprehensive simulation tool. This allows the sensor chain to be modeled in a physically realistic manner and enables the user to choose which noise components to simulate. At the outset, the user must specify the input parameters for the perfect image and the submodules. The code can then be executed, and the simulator works through the imager pipeline, modeling each of the chosen noise components. This produces the desired number of noisy output frames (more than one frame is necessary for simulating video sequences) and a report summarizing the simulation. In terms of processing time, a two-megapixel output frame required several minutes using a desktop PC (depending on scene content and algorithm settings), meaning that video sequences could be simulated in a matter of hours.

每个单独的噪声源都使用一个单独的MATLAB函数进行建模，因此它们可以使用模块化方法链接起来，以产生一个全面的仿真工具。这样可以以物理上真实的方式对传感器链进行建模，并使用户可以选择要模拟的噪声组件。一开始，用户必须指定完美图像和子模块的输入参数。然后可以执行代码，仿真器将通过模拟每个选择的噪声组件来处理图像pipeline。这将产生所需数量的噪声输出帧（模拟视频序列需要多个帧），以及总结仿真的报告。在处理时间方面，一个两百万像素的输出帧需要使用台式电脑数分钟的时间（具体取决于场景内容和算法设置），这意味着可以在几小时内模拟视频序列。

B. Model Overview

Fig. 8 shows a block diagram for our complete noise model, along with a list of the user-controllable input parameters for each module. The diagram also indicates whether the stages represent fixed-pattern or temporal components. As discussed earlier, the modular nature of the tool makes it trivial for the user to select which sources of noise to simulate in any given experiment.

图8显示了我们完整噪声模型的块图，以及每个模块的用户可控输入参数列表。该图还指出了这些阶段是否代表固定模式或时间模式。如前所述，该工具的模块化特性使得用户可以轻松选择在任何给定实验中模拟哪些噪声来源。

C. Sample Application: Dark-Current Couplets

Digital-signal processing can reduce the effect of white pixels caused by dark current. However, if two adjacent pixels are white, then a couplet results, and the processing may have difficulty performing (the high value in each pixel seems more reasonable given that its neighbor is of similar value). Unfortunately, to the observer, these white pixels are highly disturbing. These dark-current couplets are expected to be seen more as image resolution increases.

数字信号处理可以减少暗电流引起的白像素的影响。然而，如果两个相邻的像素是白色的，则会产生“偶偶对”现象，处理可能难以进行（每个像素中的高值似乎更合理，因为其邻居具有相似的值）。不幸的是，对观察者来说，这些白色像素非常令人不安。这些暗电流偶偶对随着图像分辨率的提高而预计会更多地出现。

It is, therefore, useful to compare the noise performance between different resolutions when zoomed into a region of interest in the scene, as demonstrated in Fig. 9. This experiment reveals the expected tradeoff between couplets and resolution and the difficulty in correcting them. Our tool can be used in evaluating the effectiveness of defect-correction and noise-reduction algorithms, which, in the future, will aim to tackle dark-current couplets.

因此，当放大场景的感兴趣区域时，比较不同分辨率之间的噪声性能是有用的，如图9所示。该实验揭示了偶联像素和分辨率之间的预期权衡以及纠正它们的难度。我们的工具可用于评估缺陷校正和降噪算法的有效性，这些算法将来将旨在解决暗电流偶联问题。

Fig. 9. Effect of dark current on zoomed images at different resolutions. White circles indicate couplets. Note that, for illustration purposes, the noise values chosen represent a worst case example of dark-current performance.
图9。暗电流对不同分辨率放大图像的影响。白色圆圈表示对联。注意，出于说明目的，所选的噪声值代表暗电流性能的最坏情况示例。

D. Sample Application: VFPN Analysis Image

Image quality is considerably degraded by the presence of VFPN, even though the standard-deviation value may appear low. Hence, it is interesting to investigate at what level VFPN becomes noticeable in still images at low to medium light, and our noise model offers a straightforward means of carrying out such an investigation. Since photon shot noise influences the degree to which the signal-dependent VFPN is perceptible, it is necessary to include both sources of noise in the simulation.

图像质量会因为VFPN的存在而明显降低，即使其标准差值可能很低。因此，有趣的是要研究在低到中等光线条件下，VFPN在静态图像中变得明显的程度，而我们的噪声模型提供了进行这种调查的简单方法。由于光子噪声会影响到信号相关的VFPN的可感知程度，因此在模拟中需要同时考虑这两种噪声。

Fig. 10 shows results from an experiment designed to demonstrate the impact VFPN levels at different light levels. From the images, it is clear that even a very low σVFPN of six electrons is noticeable in low light, whereas at higher light levels, it is much more difficult to perceive.

图10显示了一个实验的结果，该实验旨在证明不同光照水平下的影响VFPN水平。从图像中可以清楚地看到，即使六个电子的${\sigma }_{VFPN}$非常低，在低光下也很明显，而在更高的光水平下，它更难以感知。

E. Realistic End-Image A

A sample synthetic image incorporating all of the modeled noises is provided in Fig. 11. Although the parameter values chosen for this example are arguably rather extreme, the tool has been shown to produce realistic-looking output images.

本文提供了一张包含所有建模噪声的合成图像，如图11所示。虽然为此示例选择的参数值可能有些极端，但是该工具已经被证明能够产生逼真的输出图像。

X. CONCLUSION

A tool capable of accurately simulating CMOS image-sensor noise and its effect on images has been presented. In particular, we have demonstrated a novel approach in modeling dark current and temporal noise using more representative distributions than some previously published attempts [2], [3]. This leads to a realistic and practical simulation tool. Our noise model has been applied in industry to visualize the likely effect that the proposed process changes will have on output images. This has helped imager designers to decide upon, and justify, targets for new process parameters.

本文介绍了一种能够准确模拟CMOS图像传感器噪声及其对图像影响的工具。特别是，我们采用比一些先前发布的尝试更具代表性的分布来建模暗电流和时间噪声的新方法，从而实现了一种逼真而实用的模拟工具。我们的噪声模型已经应用于工业中，以可视化提议的流程变化对输出图像可能产生的影响。这有助于图像传感器设计人员确定和证明新流程参数的目标。

Nonetheless, there is much scope for extension. In particular, a more sophisticated means of modeling RTS and flicker noise is desirable. This would likely require more fundamental research in the field. Furthermore, the complete imaging chain could be simulated by also incorporating optical effects. Work is currently underway in pursuit of this goal, which can be accomplished either by developing appropriate submodules in our existing model or by linking the sensor model with an external optics model.

然而，仍有很多拓展空间。特别地，更复杂的RTS和闪烁噪声建模方法是值得探索的。这可能需要在该领域进行更基础的研究。此外，可以通过模拟光学效应来模拟完整的成像链。正在进行的工作致力于实现这个目标，可以通过在现有模型中开发适当的子模块或将传感器模型与外部光学模型链接来实现。