Harris corner detector algorithm

最新推荐文章于 2023-09-07 09:44:58 发布

fanbird2008

最新推荐文章于 2023-09-07 09:44:58 发布

阅读量2.4k

点赞数

分类专栏： Graphics Image/OpenCV

Graphics Image/OpenCV 专栏收录该内容

14 篇文章 0 订阅

订阅专栏

Harris and Stephens^[3] improved upon Moravec's corner detector by considering the differential of the corner score with respect to direction directly, instead of using shifted patches. (This corner score is often referred to asautocorrelation, since the term is used in the paper in which this detector is described. However, the mathematics in the paper clearly indicate that the sum of squared differences is used.)

Without loss of generality, we will assume a grayscale 2-dimensional image is used. Let this image be given by $I$ . Consider taking an image patch over the area $(u, v)$ and shifting it by $(x, y)$ . The weightedsum of squared differences (SSD) between these two patches, denoted $S$ , is given by:

$S(x,y) = \sum_u \sum_v w(u,v) \, \left( I(u+x,v+y) - I(u,v)\right)^2$

$I(u+x,v+y)$ can be approximated by aTaylor expansion. Let $I_x$ and $I_y$ be thepartial derivatives of $I$ , such that

$I(u+x,v+y) \approx I(u,v) + I_x(u,v)x+I_y(u,v)y$

This produces the approximation

$S(x,y) \approx \sum_u \sum_v w(u,v) \, \left( I_x(u,v)x + I_y(u,v)y \right)^2,$

which can be written in matrix form:

$S(x,y) \approx \begin{pmatrix} x & y \end{pmatrix} A \begin{pmatrix} x \\ y \end{pmatrix},$

where A is the structure tensor,

$A = \sum_u \sum_v w(u,v) \begin{bmatrix}I_x^2 & I_x I_y \\I_x I_y & I_y^2 \end{bmatrix}=\begin{bmatrix}\langle I_x^2 \rangle & \langle I_x I_y \rangle\\\langle I_x I_y \rangle & \langle I_y^2 \rangle\end{bmatrix}$

This matrix is a Harris matrix, and angle brackets denote averaging (i.e. summation over $(u,v)$ ). If a circular window (or circularly weighted window, such as aGaussian) is used, then the response will be isotropic.

A corner (or in general an interest point) is characterized by a large variation of $S$ in all directions of the vector $\begin{pmatrix} x & y \end{pmatrix}$ . By analyzing the eigenvalues of $A$ , this characterization can be expressed in the following way: $A$ should have two "large" eigenvalues for an interest point. Based on the magnitudes of the eigenvalues, the following inferences can be made based on this argument:

If $\lambda_1 \approx 0$ and $\lambda_2 \approx 0$ then this pixel $(x,y)$ has no features of interest.
If $\lambda_1 \approx 0$ and $\lambda_2$ has some large positive value, then an edge is found.
If $\lambda_1$ and $\lambda_2$ have large positive values, then a corner is found.

Harris and Stephens note that exact computation of the eigenvalues is computationally expensive, since it requires the computation of asquare root, and instead suggest the following function $M_c$ , where $\kappa$ is a tunable sensitivity parameter:

$M_c = \lambda_1 \lambda_2 - \kappa \, (\lambda_1 + \lambda_2)^2= \operatorname{det}(A) - \kappa \, \operatorname{trace}^2(A)$

Therefore, the algorithm does not have to actually compute the eigenvalue decomposition of the matrix $A$ and instead it is sufficient to evaluate thedeterminant and trace of $A$ to find corners, or rather interest points in general.

followings are excerpt from opencv tutorial.

How does it work?¶

Let’s look for corners. Since corners represents a variation in the gradient in the image, we will look for this “variation”.
Consider a grayscale image $I$ . We are going to sweep a window $w(x,y)$ (with displacements $u$ in the x direction and $v$ in the right direction) $I$ and will calculate the variation of intensity.

$E(u,v) = \sum _{x,y} w(x,y)[ I(x+u,y+v) - I(x,y)]^{2}$

where:
- $w(x,y)$ is the window at position $(x,y)$
- $I(x,y)$ is the intensity at $(x,y)$
- $I(x+u,y+v)$ is the intensity at the moved window $(x+u,y+v)$
Since we are looking for windows with corners, we are looking for windows with a large variation in intensity. Hence, we have to maximize the equation above, specifically the term:

$\sum _{x,y}[ I(x+u,y+v) - I(x,y)]^{2}$
Using Taylor expansion:

$E(u,v) \approx \sum _{x,y}[ I(x,y) + u I_{x} + vI_{y} - I(x,y)]^{2}$
Expanding the equation and cancelling properly:

$E(u,v) \approx \sum _{x,y} u^{2}I_{x}^{2} + 2uvI_{x}I_{y} + v^{2}I_{y}^{2}$
Which can be expressed in a matrix form as:

$E(u,v) \approx \begin{bmatrix} u & v \end{bmatrix} \left ( \displaystyle \sum_{x,y} w(x,y) \begin{bmatrix} I_x^{2} & I_{x}I_{y} \\ I_xI_{y} & I_{y}^{2} \end{bmatrix} \right ) \begin{bmatrix} u \\ v \end{bmatrix}$
Let’s denote:

$M = \displaystyle \sum_{x,y} w(x,y) \begin{bmatrix} I_x^{2} & I_{x}I_{y} \\ I_xI_{y} & I_{y}^{2} \end{bmatrix}$
So, our equation now is:

$E(u,v) \approx \begin{bmatrix} u & v \end{bmatrix} M \begin{bmatrix} u \\ v \end{bmatrix}$
A score is calculated for each window, to determine if it can possibly contain a corner:

$R = det(M) - k(trace(M))^{2}$

where:
- det(M) = $\lambda_{1}\lambda_{2}$
- trace(M) = $\lambda_{1}+\lambda_{2}$
a window with a score $R$ greater than a certain value is considered a “corner”

cv::cornerEigenValsAndVecs¶

Comments from the Wiki

void cornerEigenValsAndVecs (const Mat& src, Mat& dst, int blockSize, int apertureSize, int borderType=BORDER_DEFAULT ) ¶

Calculates eigenvalues and eigenvectors of image blocks for corner detection.

Parameters:	src – Input single-channel 8-bit or floating-point image dst – Image to store the results. It will have the same size as `src` and the type `CV_32FC(6)` blockSize – Neighborhood size (see discussion) apertureSize – Aperture parameter for the `Sobel()` operator boderType – Pixel extrapolation method; see `borderInterpolate()`

For every pixel $p$ , the function cornerEigenValsAndVecs considers a blockSize $\times$ blockSize neigborhood $S(p)$ . It calculates the covariation matrix of derivatives over the neighborhood as:

$M = \begin{bmatrix} \sum _{S(p)}(dI/dx)^2 & \sum _{S(p)}(dI/dx dI/dy)^2 \\ \sum _{S(p)}(dI/dx dI/dy)^2 & \sum _{S(p)}(dI/dy)^2 \end{bmatrix}$

Where the derivatives are computed using Sobel() operator.

After that it finds eigenvectors and eigenvalues of $M$ and stores them into destination image in the form $(\lambda_1, \lambda_2, x_1, y_1, x_2, y_2)$ where

$\lambda_1, \lambda_2$

are the eigenvalues of $M$ ; not sorted
$x_1, y_1$

are the eigenvectors corresponding to $\lambda_1$
$x_2, y_2$

are the eigenvectors corresponding to $\lambda_2$

The output of the function can be used for robust edge or corner detection.

See also: cornerMinEigenVal() , cornerHarris() , preCornerDetect()

cv::cornerHarris¶

Comments from the Wiki

void cornerHarris (const Mat& src, Mat& dst, int blockSize, int apertureSize, double k, int borderType=BORDER_DEFAULT ) ¶

Harris edge detector.

Parameters:

Parameters:	src – Input single-channel 8-bit or floating-point image dst – Image to store the Harris detector responses; will have type `CV_32FC1` and the same size as `src` blockSize – Neighborhood size (see the discussion of `cornerEigenValsAndVecs()` ) apertureSize – Aperture parameter for the `Sobel()` operator k – Harris detector free parameter. See the formula below boderType – Pixel extrapolation method; see `borderInterpolate()`

src – Input single-channel 8-bit or floating-point image
dst – Image to store the Harris detector responses; will have type CV_32FC1 and the same size as src
blockSize – Neighborhood size (see the discussion of cornerEigenValsAndVecs() )
apertureSize – Aperture parameter for the Sobel() operator
k – Harris detector free parameter. See the formula below
boderType – Pixel extrapolation method; see borderInterpolate()

The function runs the Harris edge detector on the image. Similarly to cornerMinEigenVal() and cornerEigenValsAndVecs() , for each pixel $(x, y)$ it calculates a $2\times2$ gradient covariation matrix $M^{(x,y)}$ over a $\texttt{blockSize} \times \texttt{blockSize}$ neighborhood. Then, it computes the following characteristic:

$\texttt{dst} (x,y) = \mathrm{det} M^{(x,y)} - k \cdot \left ( \mathrm{tr} M^{(x,y)} \right )^2$

Corners in the image can be found as the local maxima of this response map.

fanbird2008

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
Harris corner detector algorithm

Harris and Stephens[3] improved upon Moravec's corner detector by considering the differential of the corner score with respect to direction directly, instead of using shifted patches. (This corner
复制链接

扫一扫

专栏目录