算梯度
f ′ ( x ) = lim h → 0 f ( x + h ) − f ( x ) h = lim h → 0 f ( x ) − f ( x − h ) h f'(x)=\lim \limits _{h\rightarrow0} \frac{f(x+h)-f(x)}{h}=\lim \limits _{h\rightarrow0} \frac{f(x)-f(x-h)}{h} f′(x)=h→0limhf(x+h)−f(x)=h→0limhf(x)−f(x−h)为数学上的方法,那么我们对于向量 [ x 1 , x 2 , … , x n ] [x_1, x_2, \dots,x_n] [x1,x2,…,xn]计算梯度可以根据该方法实现如下代码,通过该代码实现对每一个数字进行梯度计算。
def gradient(f, x):
h = 1e-4 或其他更小数
f_l = f(x-h)
f_r = f(x+h)
return (f_r - f_l) / 2 / h
假设 f ( x ) = 0.1 x 2 + x f(x) = 0.1x^2+x f(x)=0.1x2+x, x ⃗ = [ 2 , 1 , 4 , 3 , 7 ] \vec x=[2, 1, 4, 3, 7] x=[2,1,4,3,7],假设 h = 0.0001 h=0.0001 h=0.0001则有:
g r a d i e n t ( f , x ⃗ ) = [ f ( 2 + 0.0001 ) − f ( 2 − 0.0001 ) 0.0002 , … , f ( 7 + 0.0001 ) − f ( 7 − 0.0001 ) 0.0002 ] = 1 0.0002 [ f ( 2.0001 ) − f ( 1.9999 ) , … , f ( 7.0001 ) − f ( 6.9999 ) ] = [ 1.3995 , 1.2004 , 1.8001 , 1.5974 , 2.4033 ] \begin{aligned} gradient(f,\vec x) &= [\frac{f(2+0.0001)-f(2-0.0001)}{0.0002}, \dots, \frac{f(7+0.0001)-f(7-0.0001)}{0.0002}]\\ &=\frac{1}{0.0002}[f(2.0001)-f(1.9999), \dots, f(7.0001)-f(6.9999)]\\ &=[1.3995, 1.2004, 1.8001, 1.5974, 2.4033] \end{aligned} gradi