Computer Vision 计算机视觉常见问题整理

这里是一些学习CV时整理的一些问题集，可能有助于复习等。由于该笔记是很早以前制作的，暂时不做修改。

CV Question

1. what’s machine vision?

Machine vision (MV) is the technology and methods used to provide imaging-based automatic inspection and analysis for such applications as automatic inspection, process control, and robot guidance in industry.

Input: image, video, Output: inspection and analysis

Goal: give computers super human-level perception

2. Typical perception channel

Representation -> ‘fancy math’ -> output, and representation and output are the parts we are most interested in.

3. Common Applications

Automated visual inspection, object recognition, face detection, face makeovers, vision in cars. image stitching, virtual fitting, vr, kinect fusion, 3D reconstruction.

4. Subject connection

Image processing: digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing.

Computer Graphics: Computer graphics is the discipline of generating images with the aid of computers.

Pattern Recognition: Pattern recognition is the automated recognition of patterns and regularities in data.

Computer Vision: Computer vision is an interdisciplinary scientific field that deals with how computers can be made to gain high-level understanding from digital images or videos.

Difference between Computer Vision adn Machine Vision: Computer vision refers to automation of the capture and processing of images, with an emphasis on image analysis. In other words, CV’s goal is not only to see, but also to process and provide useful results based on the observation. Machine vision refers to the use of computer vision in industrial environments, making it a subcategory of computer vision.

Artificial intelligence: Computer science defines AI research as the study of “intelligent agents”: any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals.[1] A more elaborate definition characterizes AI as “a system’s ability to correctly interpret external data, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation.”

5. Vision Process

• Feature extraction and region segmentation. (Low)
• Modeling and Schema Representation (Midem)
• Describe and understand (high)

6. Difficulties faced by Machine Vision

• Image ambiguity: When 3D sense is projected as a 2D image, the info of the depth and invisible parts is lost. Therefore, 3D objects of different shapes projected on the image plane may produce the same image.
• Environment Factors: Factors in the scene such as lighting, objects shapes, surface colors, cameras, and the changes in spatial relationships, etc.
• Knowledge guidance: Under different knowledge guidance, the same image will produce different recognition results.
• Large amounts of data: Gray image, color image, and depth image have a huge amount of information.The huge amount of data requires a large storage space, and it is not easy to process quickly.

7. Human Vision System

Physical structure: HVS is composed of optical system, retina, visual pathway.

TODO: I don’t want to learn HVS knowledge first, so I skip it. If I have extra time, the remain knowledge will be made up.

8. Key tech in Computer Vision System

   1. Image process( Smooth denoising, Standardization, Missing/Outlier Value Process )
   2. Image feature extraction( Shape, Texture, Color, Spatial Relations )
   3. Image Recognition( GoogLeNet, ResNet… )

9. Image formation

The randomness of the Imaging Process and the complexity of the imaging object determine the nature of the image with a random signal.

An image bascially consists of:

• Illumination component $i(x,y)$
• Reflection component $r(x,y)$

So, The 2D function representation of the Image:
$$f(x,y)=i(x,y)\ast r(x,y)$$

10. Human eye brightness perception range

Total Range: $10^{-2}-10^6$, so Contrast $c=B_{max}/B_{min}=10^8$, and the Relative Contrast $c_r=100\%\times (B-B_0)/B_0$ where $B_0$ means background brightness and $B$ means the object brightness.

Relationship between subjective brightness S and actual brightness B:
$$S=K\ln B+K_0$$

11. Brightness adaptability

Visually sensitive is contrast, not the brightness value itself.

Weber theorem:

If the brightness of an object differs from the surrounding background $I$ (their ratio is a function). It is approximately constant within a certain range of brightness, with a constant value of 0.02, which is called the Weber ratio.
$$\frac{\Delta I}{I}=0.02$$
Mach Effect: The visual system is less sensitive to spatial high and low frequencies, while it is more sensitive to spatial intermediate frequencies.Therefore, a brightness overshoot occurs at a sudden change in brightness. This overshoot can enhance the outline of the scene seen by the human eye.

12. Color imaging model

Light energy itself is colorless. Color is a physiological and psychological phenomenon that occurs when people’s eyes perceive light.

Lightwave: Light is an electromagnetic wave that radiates according to its wavelength.

Young–Helmholtz theory(trichromatic theory): the three types of cone photoreceptors could be classified as short-preferring (violet), middle-preferring (green), and long-preferring (red).

13. Color property

Hue: the degree to which a stimulus can be described as similar to or different from stimuli that are described as red, green, blue, and yellow.

Saturation: colorfulness of an area judged in proportion to its brightness.

Intensity: Refers to the degree of light and darkness that the human eye feels due to color stimuli.

Grassman Laws:

First law:	Two colored lights appear different if they differ in either dominant wavelength, luminance or purity. Corollary: For every colored light there exists a light with a complementary color such that a mixture of both lights either desaturates the more intense component or gives uncolored (grey/white) light.
Second law:	The appearance of a mixture of light made from two components changes if either component changes. Corollary: A mixture of two colored lights that are non-complementary result in a mixture that varies in hue with relative intensities of each light and in saturation according to the distance between the hues of each light.
Third law:	There are lights with different spectral power distributions but appear identical. First corollary: such identical appearing lights must have identical effects when added to a mixture of light. Second corollary: such identical appearing lights must have identical effects when subtracted (i.e., filtered) from a mixture of light.
Fourth law:	The intensity of a mixture of lights is the sum of the intensities of the components.

14. Color

The result of interaction between physical light in the environment and our visual system

15. Color Space

• Linear color space
• RGB color space
• HSV color space
• CIE XYZ

16. White Balance

White balance (WB) is the process of removing unrealistic color casts, so that objects which appear white in person are rendered white in your photo.

Color temperature describes the spectrum of light which is radiated from a “blackbody” with that surface temperature.

Von Kries adaptation:

• Multiply each channel by a gain factor
• A more general transformation would correspond to an arbitrary 3x3 matrix

Best way: gray card:

• Take a picture of a neutral object
• Deduce the weight of each channel

Brightest pixel assumption (non-staurated)

• Highlights usually have the color of the light source
• Use weights inversely proportional to the values of the brightest pixels

Gamutmapping

• Gamut: convex hull of all pixel colors in an image
• Find the transformation that matches the gamut of the image to the gamut of a “typical” image under white light

17. Mathematical representation of an image

Optical radiation power of wavelength 𝜆 is received on the imaging target surface of the camera:
$$I=f(x,y,\lambda ,t)$$
Common image types:

• Binary image
• Grayscale image
• Index image
• RGB image

18. Common concepts

pixel neighborhood: 4-neighborhood($N_4(p)$), 8-neighborhood($N_8(p)$);

pixel adjacency ===> pixel connectivity;

Template(filter, mask) + convolution ===> filtering, smoothing, sharpening;

Convolution operation properties:

• Smoothness: Make the fine structure of each function smooth
• Diffusivity: Interval expansion, Diffusion of energy distribution

Application of convolution:

• Deconvolution
• Remove noise
• Feature enhancement

19. Pixels distance

Distance measurement function characteristics:

• $D(p,q)>0$ non-negativity or separation axiom
• $D(p,q)=0\iff p=q$ identity of indiscernibles
• $D(p,q)=D(q,p)$ symmetry
• $D(p,r)\le D(p,q)+D(q,r)$ ubadditivity or triangle inequality

Common distance metric functions:

• Euclidean Distance: $D_E(p,q)=[(x-s{)}^2+(y-t{)}^2{]}^{\frac{1}{2}}$
• City-block Distance: $D_4(p,q)=|x-s|+|y-t|$
• Chessboard Distance: $D(p,q)=max(|x-s|+|y-t|)$

p-norm: $ \norm{x}_p = (\sum_i{|x_i|^p}){\frac{1}{p}} $

Frobenius-norm: KaTeX parse error: Undefined control sequence: \norm at position 1: \̲n̲o̲r̲m̲{A}_F = \sqrt{\…

In the image, the L-2 norm constraint does not distinguish between the edge tangent direction and the gradient direction of the image, and does not reflect the difference between the texture area and the flat area.

The edges of the image will be blurred during the image restoration process.

The L-1 norm constraint only diffuses in the tangential direction of the edges, but does not diffuse in the gradient direction. The goal is to keep the edges of the image as close as possible. Will lead to poor noise suppression effect, step patching results.

20. Statistical characteristics of images

• Information entropy: $H=-{\sum }_{i=1}^k{p}_i{\log }_2{p}_i$

• Gray average: $\bar{f}=\frac{{\sum }_{i=0}^{M-1}}{MN}$