交叉相关与卷积 Cross-correlation v.s. convolution

卷积是一种广泛应用于信号处理、图像处理和其他工程/科学领域的技术。在深度学习中，一种模型结构卷积神经网络（CNN）就是以这种技术命名的。然而，深度学习中的卷积（Convolution）本质上是信号/图像处理中的互相关(Cross-correlation)。这两种操作之间有细微的区别，但不必深入细节。在信号/图像处理中，卷积被定义为两个函数的乘积在一个函数被反转和移位后的积分。另一方面，互相关被称为两个函数的滑动点积或滑动内积。互相关中的滤波器不反转。它直接通过函数f滑动。f和g之间的相交区域是互相关。

https://en.wikipedia.org/wiki/Convolution#/media/File:Comparison_convolution_correlation.svg 展示了相关性和互相关之间的区别。

在深度学习中，卷积中的滤波器是不可逆的。严格地说，这是相互关联的。我们基本上执行元素级的乘法和加法。但在深度学习中称之为卷积是一种惯例。这很好，因为过滤器的权重是在训练期间学习的。如果上述示例中的反向函数g是正确的函数，则经过训练后，学习的滤波器将看起来像反向函数g。因此，在训练前不需要像真正的卷积那样首先反转滤波器。

综上，从数学运算的角度上来看，互相关与卷积的区别为是否逆时针翻转核（Kernel or Filter）。互相关不翻转，卷积则将核逆时针翻转180°。在深度学习中，使用的卷积操作实际上就是互相关，这是一个习惯问题，不必深究。

其他相关的问题推导见此网站

tutorialspoint

Convolution
Convolution is a mathematical operation used to express the relation between input and output of an LTI system. It relates input, output and impulse response of an LTI system as

$$y(t)=x(t)\ast h(t)$$

Where y (t) = output of LTI

x (t) = input of LTI

h (t) = impulse response of LTI

There are two types of convolutions:

Continuous convolution

Discrete convolution

Continuous Convolution
continuous convolution
$ y(t) ,,= x (t) * h (t) $

$={\int }_{-\infty }^{\infty }x(\tau )h(t-\tau )d\tau$

(or)

$= \int_{-\infty}^{\infty} x(t - \tau) h (\tau)d\tau $

Discrete Convolution
Discrete convolution
$y(n)=x(n)\ast h(n)$

$= \Sigma_{k = - \infty}^{\infty} x(k) h (n-k) $

(or)

$= \Sigma_{k = - \infty}^{\infty} x(n-k) h (k) $

By using convolution we can find zero state response of the system.

Deconvolution
Deconvolution is reverse process to convolution widely used in signal and image processing.

Properties of Convolution
Commutative Property
$ x_1 (t) * x_2 (t) = x_2 (t) * x_1 (t) $

Distributive Property
$ x_1 (t) * [x_2 (t) + x_3 (t) ] = [x_1 (t) * x_2 (t)] + [x_1 (t) * x_3 (t)]$

Associative Property
$x_1 (t) * [x_2 (t) * x_3 (t) ] = [x_1 (t) * x_2 (t)] * x_3 (t) $

Shifting Property
$ x_1 (t) * x_2 (t) = y (t) $

$ x_1 (t) * x_2 (t-t_0) = y (t-t_0) $

$ x_1 (t-t_0) * x_2 (t) = y (t-t_0) $

$ x_1 (t-t_0) * x_2 (t-t_1) = y (t-t_0-t_1) $

Convolution with Impulse
$ x_1 (t) * \delta (t) = x(t) $

$ x_1 (t) * \delta (t- t_0) = x(t-t_0) $

Convolution of Unit Steps
$ u (t) * u (t) = r(t) $

$ u (t-T_1) * u (t-T_2) = r(t-T_1-T_2) $

$ u (n) * u (n) = [n+1] u(n) $

Scaling Property
If $x (t) * h (t) = y (t) $

then $x(at)\ast h(at)=\frac{1}{|a|}y(at)$

Differentiation of Output
if $y(t)=x(t)\ast h(t)$

then $ {dy (t) \over dt} = {dx(t) \over dt} * h (t) $

or

$ {dy (t) \over dt} = x (t) * {dh(t) \over dt}$

Note:

Convolution of two causal sequences is causal.

Convolution of two anti causal sequences is anti causal.

Convolution of two unequal length rectangles results a trapezium.

Convolution of two equal length rectangles results a triangle.

A function convoluted itself is equal to integration of that function.

Example: You know that $u(t)\ast u(t)=r(t)$

According to above note, $u(t)\ast u(t)=\int u(t)dt=\int 1dt=t=r(t)$

Here, you get the result just by integrating $u(t)$.

Limits of Convoluted Signal
If two signals are convoluted then the resulting convoluted signal has following range:

Sum of lower limits < t < sum of upper limits

Ex: find the range of convolution of signals given below

Limits of convoluted signal
Here, we have two rectangles of unequal length to convolute, which results a trapezium.

The range of convoluted signal is:

Sum of lower limits < t < sum of upper limits

$-1±2 < t < 2+2 $

$-3 < t < 4 $

Hence the result is trapezium with period 7.

Area of Convoluted Signal
The area under convoluted signal is given by $A_y=A_x{A}_h$

Where Ax = area under input signal

Ah = area under impulse response

Ay = area under output signal

Proof: $y(t)={\int }_{-\infty }^{\infty }x(\tau )h(t-\tau )d$