目录
Quadratic forms
- A quadratic form on
R
n
\R^n
Rn is a function
Q
Q
Q defined on
R
n
\R^n
Rn whose value at a vector
x
\boldsymbol x
x in
R
n
\R^n
Rn can be computed by an expression of the form
Q
(
x
)
=
x
T
A
x
Q(\boldsymbol x)=\boldsymbol x^TA\boldsymbol x
Q(x)=xTAx, where
A
A
A is an
n
×
n
n \times n
n×n symmetric matrix. The matrix
A
A
A is called the matrix of the quadratic form (关于二次型的矩阵).
- The simplest example of a nonzero quadratic form is Q ( x ) = x T I x = ∥ x ∥ 2 Q(\boldsymbol x)=\boldsymbol x^TI\boldsymbol x=\left\|\boldsymbol x\right\|^2 Q(x)=xTIx=∥x∥2.
当 A A A 不为对称矩阵时,可以把它写为 A = A S + A A A=A^S+A^A A=AS+AA 的形式,其中 A S A^S AS 为对称矩阵, A A A^A AA 为反对称矩阵。可以证明, x T A x = x T A S x x^TAx=x^TA^Sx xTAx=xTASx
.
证明:
x T A A x = ∑ i , j x i A i j A x j = 1 2 ( ∑ i , j x i A i j A x j − ∑ i , j x j A j i A x i ) x^TA^Ax=\sum_{i,j}x_iA^A_{ij}x_j=\frac{1}{2}\left(\sum_{i,j}x_iA^A_{ij}x_j-\sum_{i,j}x_jA^A_{ji}x_i\right) xTAAx=i,j∑xiAijAxj=21(i,j∑xiAijAxj−i,j∑xjAjiAxi)可以将第 2 项中的 i , j i,j i,j 互换,可得
x T A A x = 1 2 ( ∑ i , j x i A i j A x j − ∑ i , j x i A i j A x j ) = 0 x^TA^Ax=\frac{1}{2}\left(\sum_{i,j}x_iA^A_{ij}x_j-\sum_{i,j}x_iA^A_{ij}x_j\right)=0 xTAAx=21(i,j∑xiAijAxj−i,j∑xiAijAxj)=0
- Examples 1 and 2 show the connection between any symmetric matrix A A A and the quadratic form x T A x \boldsymbol x^TA\boldsymbol x xTAx.
EXAMPLE 1
Let x = [ x 1 x 2 ] \boldsymbol x =\begin{bmatrix}x_1\\x_2\end{bmatrix} x=[x1x2]. Compute x T A x \boldsymbol x^TA\boldsymbol x xTAx for the following matrices:
SOLUTION
-
a
a
a.
- Note that the quadratic form associated with the diagonal matrix A A A in has no x 1 x 2 x_1x_2 x1x2 c r o s s cross cross- p r o d u c t product product term (交叉乘积).
-
b
b
b. There are two
−
2
-2
−2 entries in
A
A
A. Watch how they enter the calculations.
- The presence of − 4 x 1 x 2 -4x_1x_2 −4x1x2 in the quadratic form is due to the − 2 -2 −2 entries off the diagonal in the matrix A A A.
EXAMPLE 2
For x \boldsymbol x x in R 3 \R^3 R3, let Q ( x ) = 5 x 1 2 + 3 x 2 2 + 2 x 3 2 − x 1 x 2 + 8 x 2 x 3 Q(\boldsymbol x)= 5x_1^2+ 3x_2^2+ 2x_3^2- x_1x_2 + 8x_2x_3 Q(x)=5x12+3x22+2x32−x1x2+8x2x3. Write this quadratic form as x T A x \boldsymbol x^TA\boldsymbol x xTAx.
SOLUTION
- The coefficients of
x
1
2
,
x
2
2
,
x
3
2
\boldsymbol x_1^2,\boldsymbol x_2^2 , \boldsymbol x_3^2
x12,x22,x32 go on the diagonal of
A
A
A. To make
A
A
A symmetric, the coefficient of
x
i
x
j
x_ix_j
xixj for
i
≠
j
i\neq j
i=j must be split evenly between the
(
i
,
j
)
(i, j)
(i,j)- and
(
j
,
i
)
(j, i)
(j,i)-entries in
A
A
A. It is readily checked that
Change of Variable in a Quadratic Form
二次型的变量代换
通过坐标变换消除交叉乘积项
- In some cases, quadratic forms are easier to use when they have no cross-product terms—that is, when the matrix of the quadratic form is a diagonal matrix.
- Fortunately, the cross-product term can be eliminated by making a suitable change of variable.
- If
x
\boldsymbol x
x represents a variable vector in
R
n
\R^n
Rn, then a change of variable is an equation of the form
where P P P is an invertible matrix and y \boldsymbol y y is a new variable vector in R n \R^n Rn. Here y \boldsymbol y y is the coordinate vector of x \boldsymbol x x relative to the basis of R n \R^n Rn determined by the columns of P P P.
- If the change of variable (1) is made in a quadratic form
x
T
A
x
\boldsymbol x^TA\boldsymbol x
xTAx, then
and the new matrix of the quadratic form is P T A P P^TAP PTAP. Since A A A is symmetric, there is an orthogonal matrix P P P such that P T A P P^TAP PTAP is a diagonal matrix D D D, and the quadratic form in (2) becomes y T D y \boldsymbol y^TD\boldsymbol y yTDy.
主轴定理
- The columns of P P P in the theorem are called the principal axes (主轴) of the quadratic form x T A x \boldsymbol x^TA\boldsymbol x xTAx.
- The vector y \boldsymbol y y is the coordinate vector of x \boldsymbol x x relative to the orthonormal basis of R n \R^n Rn given by these principal axes.
A Geometric View of Principal Axes
- Suppose
Q
(
x
)
=
x
T
A
x
Q(\boldsymbol x)=\boldsymbol x^TA\boldsymbol x
Q(x)=xTAx, where
A
A
A is an invertible
2
×
2
2 \times 2
2×2 symmetric matrix, and let
c
c
c be a constant. It can be shown that the set of all
x
\boldsymbol x
x in
R
2
\R^2
R2 that satisfy
either corresponds to an ellipse (or circle), a hyperbola (双曲线), two intersecting lines, or a single point, or contains no points at all.
- If
A
A
A is a diagonal matrix, the graph is in standard position, such as in Figure 2.
- If
A
A
A is not a diagonal matrix, the graph of equation (3) is rotated out of standard position, as in Figure 3. Finding the principal axes (determined by the eigenvectors of
A
A
A) amounts to finding a new coordinate system with respect to which the graph is in standard position. The positive
y
1
y_1
y1-axis in Figure 3(b) is in the direction of the first column of the matrix
P
P
P, and the positive
y
2
y_2
y2-axis is in the direction of the second column of
P
P
P.
- If
A
A
A is a diagonal matrix, the graph is in standard position, such as in Figure 2.
Classifying Quadratic Forms
二次型的分类
- When
A
A
A is an
n
×
n
n\times n
n×n matrix, the quadratic form
Q
(
x
)
=
x
T
A
x
Q(\boldsymbol x)=\boldsymbol x^TA\boldsymbol x
Q(x)=xTAx is a real-valued function with domain
R
n
\R^n
Rn. Figure 4 displays the graphs of four quadratic forms with domain
R
2
\R^2
R2. For each point
x
=
(
x
1
,
x
2
)
\boldsymbol x=(x_1, x_2)
x=(x1,x2) in the domain of a quadratic form
Q
Q
Q, the graph displays the point
(
x
1
,
x
2
,
z
)
(x_1, x_2,z)
(x1,x2,z) where
z
=
Q
(
x
)
z= Q(\boldsymbol x)
z=Q(x).
- Notice that except at
x
=
0
\boldsymbol x=\boldsymbol 0
x=0, the values of
Q
(
x
)
Q(\boldsymbol x)
Q(x) are all positive in Figure 4(a) and all negative in Figure 4(d). The horizontal cross-sections (水平截面) of the graphs are ellipses in Figures 4(a) and 4(d) and hyperbolas in Figure 4
(
c
)
(c)
(c).
- Notice that except at
x
=
0
\boldsymbol x=\boldsymbol 0
x=0, the values of
Q
(
x
)
Q(\boldsymbol x)
Q(x) are all positive in Figure 4(a) and all negative in Figure 4(d). The horizontal cross-sections (水平截面) of the graphs are ellipses in Figures 4(a) and 4(d) and hyperbolas in Figure 4
(
c
)
(c)
(c).
正定, 负定, 不定, 半正定, 半负定
- The simple
2
×
2
2 \times 2
2×2 examples in Figure 4 illustrate the following definitions
positive definite (正定的); negative definite (负定的); indefinite (不定的)
- Also, Q Q Q is said to be positive semidefinite (半正定的) if Q ( x ) ≥ 0 Q(\boldsymbol x)\geq0 Q(x)≥0 for all x \boldsymbol x x, and to be negative semidefinite if Q ( x ) ≤ 0 Q(\boldsymbol x)\leq 0 Q(x)≤0 for all x \boldsymbol x x.
二次型与特征值
- 类似地,半正定矩阵的特征值都大于等于 0;半负定矩阵的特征值都小于等于 0
PROOF
- By the Principal Axes Theorem, there exists an orthogonal change of variable
x
=
P
y
\boldsymbol x = P\boldsymbol y
x=Py such that
where λ 1 , . . . , λ n \lambda_1,...,\lambda_n λ1,...,λn are the eigenvalues of A A A. Since P P P is invertible, there is a one-to-one correspondence between all nonzero x \boldsymbol x x and all nonzero y \boldsymbol y y. Thus the values of Q ( x ) Q(\boldsymbol x) Q(x) for x ≠ 0 \boldsymbol x\neq \boldsymbol 0 x=0 coincide with the values of the expression on the right side of (4), which is obviously controlled by the signs of the eigenvalues λ 1 , . . . , λ n \lambda_1,...,\lambda_n λ1,...,λn, in the three ways described in the theorem.
二次型与行列式
Another useful way to characterize quadratic forms
- Let
A
=
[
a
b
c
d
]
A=\begin{bmatrix}a&b\\c&d\end{bmatrix}
A=[acbd]. If
λ
1
\lambda_1
λ1 and
λ
2
\lambda_2
λ2 are the eigenvalues of
A
A
A, then the characteristic polynomial is
d
e
t
(
A
−
λ
I
)
=
λ
2
−
(
a
+
d
)
λ
+
a
d
−
b
c
det(A-\lambda I)=\lambda^2-(a+d)\lambda+ad-bc
det(A−λI)=λ2−(a+d)λ+ad−bc. Thus
λ
1
+
λ
2
=
a
+
d
\lambda_1+\lambda_2=a +d
λ1+λ2=a+d and
λ
1
λ
2
=
d
e
t
A
\lambda_1\lambda_2= detA
λ1λ2=detA. The following statements can be easily verified:
- a a a. Q Q Q is positive definite if d e t A > 0 detA > 0 detA>0 and a > 0 a > 0 a>0.
- b b b. Q Q Q is negative definite if d e t A > 0 detA > 0 detA>0 and a < 0 a < 0 a<0.
- c c c. Q Q Q is indefinite if d e t A < 0 detA < 0 detA<0.
- The 2 × 2 2 \times 2 2×2 case can be generalized to n × n n\times n n×n matrices.
正定矩阵, 负定矩阵, 不定矩阵, 半正/负定矩阵
- The classification of a quadratic form is often carried over to the matrix of the form. Thus a positive definite matrix (正定矩阵)
A
A
A is a symmetric matrix for which the quadratic form
x
T
A
x
\boldsymbol x^TA\boldsymbol x
xTAx is positive definite. Other terms, such as positive semidefinite matrix, are defined analogously.
- 注意到,正定矩阵特征值均大于 0,因此无 0 特征值 ( A x = 0 x A\boldsymbol x=0\boldsymbol x Ax=0x 无非零解),因此必可逆;同理负定矩阵也可逆
- 正定矩阵的逆矩阵也可逆
- 假设
A
A
A 为正定矩阵,易证
A
−
1
A^{-1}
A−1 为对称矩阵,因此有
x T A − 1 x = x T A − 1 A A − 1 x = x T ( A − 1 ) T A A − 1 x = ( A − 1 x ) T A ( A − 1 x ) x^TA^{-1}x=x^TA^{-1}AA^{-1}x=x^T(A^{-1})^T AA^{-1}x=(A^{-1}x)^TA(A^{-1}x) xTA−1x=xTA−1AA−1x=xT(A−1)TAA−1x=(A−1x)TA(A−1x)
- 假设
A
A
A 为正定矩阵,易证
A
−
1
A^{-1}
A−1 为对称矩阵,因此有
EXERCISE 27
Let A A A and B B B be symmetric n × n n\times n n×n matrices whose eigenvalues are all positive. Show that the eigenvalues of A + B A+ B A+B are all positive.
SOLUTION
- [Hint: Consider quadratic forms.]
Gram Matrix A T A A^TA ATA (格拉姆矩阵 / 内积矩阵)
- 设 G = A T A G=A^TA G=ATA 为 A A A 的格拉姆矩阵,则 g i j = a i ⋅ a j g_{ij}=\boldsymbol a_i\cdot\boldsymbol a_j gij=ai⋅aj 显然为一个对称矩阵
EXERCISE 25
Show that if B B B is m × n m \times n m×n, then B T B B^TB BTB is positive semidefinite; and if B B B is n × n n\times n n×n and invertible, then B T B B^TB BTB is positive definite.
SOLUTION
- [Hint: x T B T B x \boldsymbol x^TB^TB\boldsymbol x xTBTBx]
EXERCISE 26
Show that if an n × n n \times n n×n matrix A A A is positive definite, then there exists a positive definite matrix B B B such that A = B T B A = B^TB A=BTB.
SOLUTION
- [Hint: Write A = P D P T A = PDP^T A=PDPT, with P T = P – 1 P^T = P^{–1} PT=P–1. Produce a diagonal matrix C C C such that D = C T C D = C^TC D=CTC, and let B = P C P T B = PCP^T B=PCPT. Show that B B B works.]
EXERCISE
If A A A is m × n m \times n m×n, show that the Gram matrix of any matrix A A A has the same rank as A A A.
SOLUTION
- [Hint: Section 6.5 Theorem 14] or Exercise 13 in Section 4.6
楚列斯基分解
EXERCISE
Prove that an n × n n\times n n×n matrix A A A is positive definite if and only if A A A admits a Cholesky factorization, namely, A = R T R A= R^TR A=RTR for some invertible upper triangular matrix R R R whose diagonal entries are all positive. [Hint: Use a QR factorization and Exercise 26.]
SOLUTION
- If A = R T R A = R^TR A=RTR, where R R R is invertible, then x T A x = ( R x ) T ( R x ) ≥ 0 \boldsymbol x^TA\boldsymbol x=(R\boldsymbol x)^T(R\boldsymbol x)\geq 0 xTAx=(Rx)T(Rx)≥0 when x ≠ 0 \boldsymbol x\neq\boldsymbol 0 x=0. Thus A A A is positive definite.
- Conversely, suppose that
A
A
A is positive definite. Then by Exercise 26,
A
=
B
T
B
A = B^TB
A=BTB for some positive definite matrix
B
B
B. Since the eigenvalues of
B
B
B are positive, 0 is not an eigenvalue and so
B
B
B is invertible. In particular, the columns of
B
B
B are linearly independent. By Theorem 12 in Section 6.4,
B
=
Q
R
B = QR
B=QR for some
n
×
n
n\times n
n×n matrix
Q
Q
Q with orthonormal columns and some upper triangular matrix
R
R
R with positive elements on its diagonal. Since
Q
Q
Q is square,
Q
T
Q
=
I
Q^TQ = I
QTQ=I. So
A = B T B = ( Q R ) T ( Q R ) = R T Q T Q R = R T R A=B^{T} B=(Q R)^{T}(Q R)=R^{T} Q^{T} Q R=R^{T} R A=BTB=(QR)T(QR)=RTQTQR=RTR
References
- L i n e a r Linear Linear a l g e b r a algebra algebra a n d and and i t s its its a p p l i c a t i o n s applications applications