一些概念
MLP、DNN
MLP:Multilayer Perceptron
习惯上也称为狭义的DNN(Deep Neural Networks深度神经网络)
后续待补充…
深度前馈神经网络
全连接神经网络
举个简单
加入偏置项参数b
代码如下:
import tensorflow as tf
# generate [x1, x2]
x = tf.constant([0.9, 0.85], shape=[1, 2])
# generate w1 2*3 w2 3*1
w1 = tf.Variable(tf.constant([[0.2, 0.1, 0.3], [0.2, 0.4, 0.3]], shape=[2, 3]), name="w1")
w2 = tf.Variable(tf.constant([0.2, 0.5, 0.25], shape=[3, 1]), name="w2")
# b1 and b2 as biase
b1 = tf.constant([-0.3, 0.1, 0.2], shape=[1, 3], name="b1")
b2 = tf.constant([-0.3], shape=[1], name="b2")
# initialize
init_op = tf.global_variables_initializer()
a = tf.matmul(x, w1)+b1
y = tf.matmul(a, w2)+b2
with tf.Session() as sess:
sess.run(init_op)
print(sess.run(y))
#最后输出【【0.15625】】
通常我们并不知道网络的参数,对于weight权重参数,通常是把它初始化为一个随即矩阵,对于biase偏置项参数,通常用zeros()函数或者ones()函数进行初始化
import tensorflow as tf
# generate [x1, x2]
x = tf.constant([0.9, 0.85], shape=[1, 2])
# generate w1 2*3 w2 3*1
w1 = tf.Variable(tf.random_normal([2, 3], stddev=1, seed=1), name="w1")
w2 = tf.Variable(tf.random_normal([3, 1], stddev=1, seed=1), name="w2")
# b1 and b2 as biase
b1 = tf.Variable(tf.zeros([1, 3]), name="b1")
b2 = tf.Variable(tf.ones([1]), name="b2")
# initialize
init_op = tf.global_variables_initializer()
a = tf.matmul(x, w1)+b1
y = tf.matmul(a, w2)+b2
with tf.Session() as sess:
sess.run(init_op)
print(sess.run(y))
用激活函数去线性化
常用激活函数ReLU
φ
(
z
)
=
m
a
x
{
0
,
z
}
\varphi (z)=max\left \{0,z\right \}
φ(z)=max{0,z}
在输入和0之间求最大值的函数
加入ReLu激活函数的神经元又称为整流线性单元,其一半的定义域输出为0
易于优化,当输出不为0时(处于激活状态),一阶导数保持一个较大值,二阶导数几乎处处为0
初始化时,偏执项设置为一个小的正值如0.1,可使得输入大部分处于激活状态。
import tensorflow as tf
# generate [x1, x2]
x = tf.constant([0.9, 0.85], shape=[1, 2])
# generate w1 2*3 w2 3*1
w1 = tf.Variable(tf.random_normal([2, 3], stddev=1, seed=1), name="w1")
w2 = tf.Variable(tf.random_normal([3, 1], stddev=1, seed=1), name="w2")
# b1 and b2 as biase
b1 = tf.nn.relu(tf.Variable(tf.constant([0.1, 0.1, 0.1], shape=[1, 3]), name="b1"))
b2 = tf.nn.relu(tf.Variable(tf.ones([1]), name="b2"))
# initialize
init_op = tf.global_variables_initializer()
a = tf.matmul(x, w1)+b1
y = tf.matmul(a, w2)+b2
with tf.Session() as sess:
sess.run(init_op)
print(sess.run(y))