tensorflow1学习笔记之一些概念和深度前馈神经网络

一些概念

MLP、DNN

MLP：Multilayer Perceptron
习惯上也称为狭义的DNN（Deep Neural Networks深度神经网络）

后续待补充…

深度前馈神经网络

全连接神经网络

举个简单

加入偏置项参数b
代码如下：

import tensorflow as tf
# generate [x1, x2]
x = tf.constant([0.9, 0.85], shape=[1, 2])
# generate w1 2*3 w2 3*1
w1 = tf.Variable(tf.constant([[0.2, 0.1, 0.3], [0.2, 0.4, 0.3]], shape=[2, 3]), name="w1")
w2 = tf.Variable(tf.constant([0.2, 0.5, 0.25], shape=[3, 1]), name="w2")
# b1 and b2 as biase
b1 = tf.constant([-0.3, 0.1, 0.2], shape=[1, 3], name="b1")
b2 = tf.constant([-0.3], shape=[1], name="b2")

# initialize
init_op = tf.global_variables_initializer()

a = tf.matmul(x, w1)+b1
y = tf.matmul(a, w2)+b2

with tf.Session() as sess:
    sess.run(init_op)
    print(sess.run(y))
    #最后输出【【0.15625】】

通常我们并不知道网络的参数，对于weight权重参数，通常是把它初始化为一个随即矩阵，对于biase偏置项参数，通常用zeros()函数或者ones()函数进行初始化

import tensorflow as tf
# generate [x1, x2]
x = tf.constant([0.9, 0.85], shape=[1, 2])
# generate w1 2*3 w2 3*1
w1 = tf.Variable(tf.random_normal([2, 3], stddev=1, seed=1), name="w1")
w2 = tf.Variable(tf.random_normal([3, 1], stddev=1, seed=1), name="w2")
# b1 and b2 as biase
b1 = tf.Variable(tf.zeros([1, 3]), name="b1")
b2 = tf.Variable(tf.ones([1]), name="b2")

# initialize
init_op = tf.global_variables_initializer()

a = tf.matmul(x, w1)+b1
y = tf.matmul(a, w2)+b2

with tf.Session() as sess:
    sess.run(init_op)
    print(sess.run(y))

用激活函数去线性化

常用激活函数ReLU φ ( z ) = m a x { 0 , z } \varphi (z)=max\left \{0,z\right \} φ(z)=max{0,z}
在输入和0之间求最大值的函数
加入ReLu激活函数的神经元又称为整流线性单元，其一半的定义域输出为0
易于优化，当输出不为0时（处于激活状态），一阶导数保持一个较大值，二阶导数几乎处处为0
初始化时，偏执项设置为一个小的正值如0.1，可使得输入大部分处于激活状态。

import tensorflow as tf
# generate [x1, x2]
x = tf.constant([0.9, 0.85], shape=[1, 2])
# generate w1 2*3 w2 3*1
w1 = tf.Variable(tf.random_normal([2, 3], stddev=1, seed=1), name="w1")
w2 = tf.Variable(tf.random_normal([3, 1], stddev=1, seed=1), name="w2")
# b1 and b2 as biase
b1 = tf.nn.relu(tf.Variable(tf.constant([0.1, 0.1, 0.1], shape=[1, 3]), name="b1"))
b2 = tf.nn.relu(tf.Variable(tf.ones([1]), name="b2"))

# initialize
init_op = tf.global_variables_initializer()

a = tf.matmul(x, w1)+b1
y = tf.matmul(a, w2)+b2

with tf.Session() as sess:
    sess.run(init_op)
    print(sess.run(y))