Tensorflow2.0——自动求导API

前言

TensorFlow 为自动微分提供了 tf.GradientTape API ，根据某个函数的输入变量来计算它的导数。在深度神经网络训练过程中最常用的误差反向传播算法(Error Back Propagation Training)是更新网络权重的关键，求偏导常用到这种机制。

一元二次方程求导

只有tf.Variable对象不需要使用watch方法。

import tensorflow as tf
#最简单的实现y= 2*x*x + x的求导
x = tf.constant(3.0)
with tf.GradientTape() as tape:
    tape.watch(x)
    y = 2*x*x + x
dx = tape.gradient(y,x)  # 一阶导数值   

<tf.Tensor: id=40, shape=(), dtype=float32, numpy=13.0>

下面实现二阶求导：

x = tf.constant(3.0)
with tf.GradientTape() as tape:
    tape.watch(x)
    with tf.GradientTape() as tape_2:
        tape_2.watch(x)
        y = 2*x*x + x
    dx = tape_2.gradient(y,x)  # 一阶导数值
dx_2 = tape.gradient(dx,x)  # 二阶导数值

<tf.Tensor: id=60, shape=(), dtype=float32, numpy=4.0>

二元二次方程求偏导

#最简单的实现y= 2*x1^2 + 5*x1 + 3*x2^2 - 6*x2的求导
x1 = tf.Variable(2.)
x2 = tf.Variable(3.)
with tf.GradientTape() as tape:
    y = 2*x1*x1 + 5*x1 + 3*x2*x2 - 6*x2
dx1,dx2 = tape.gradient(y,[x1,x2])  # 一阶导数值

dx1 = <tf.Tensor: id=207, shape=(), dtype=float32, numpy=13.0>
dx2 = <tf.Tensor: id=159, shape=(), dtype=float32, numpy=12.0>

下面实现二阶求导：

x1 = tf.Variable(2.)
x2 = tf.Variable(3.)
with tf.GradientTape() as tape:
    tape.watch(x)
    with tf.GradientTape() as tape_2:
        tape_2.watch(x)
        y = 2*x1*x1 + 5*x1 + 3*x2*x2 - 6*x2
    dx1,dx2 = tape_2.gradient(y,[x1,x2])  # 一阶导数值
dx1_2,dx2_2 = tape.gradient([dx1,dx2],[x1,x2])  # 二阶导数值

dx1_2 = <tf.Tensor: id=274, shape=(), dtype=float32, numpy=4.0>
dx2_2 = <tf.Tensor: id=342, shape=(), dtype=float32, numpy=6.0>