对话系统（二）-普通神经网络

原理

流程

1. 生成数据
2. 生成权重
3. layer1：
4. layer2：

$x$：输入数据$(20,5)$
$w_1$：第一层权重$(5,3)$
$w_2$：第二层权重$(3,2)$
$a_1$：乘积$(20,3)$
$h_1$：过激活函数$(20,3)$
$a_2$：乘积$(20,2)$
$h_2$：过激活函数$(20,2)$

正向传播

$x$
$a_1=x\ast w_1$
$h_1=sigmoid(a_1)$
$a_2=h_1\ast w_2$
$h_2=sigmoid(a_2)$

推导

损失函数logloss：$J=-\frac{1}{m}\sum (y\log \hat{y}+(1-y)\log (1-\hat{y}))$

$\frac{\partial J}{\partial w_2}=\frac{\partial J}{\partial h_2}\ast \frac{\partial h_2}{\partial a_2}\ast \frac{\partial a_2}{\partial w_2}$

$\frac{\partial J}{\partial w_1}=\frac{\partial J}{\partial h_2}\ast \frac{\partial h_2}{\partial a_2}\ast \frac{\partial a_2}{\partial h_1}\ast \frac{\partial h_1}{\partial a_1}\ast \frac{\partial a_1}{\partial w_1}$

其中公共部分（前两个偏导）为：$\frac{\partial J}{\partial h_2}\ast \frac{\partial h_2}{\partial a_2}$

$\frac{\partial J}{\partial h_2}=-\frac{1}{m}\ast \frac{y-h_2}{h_2(1-h_2)}$

$\frac{\partial h_2}{\partial a_2}=h_2(1-h_2)$

$\frac{\partial a_2}{\partial w_2}=h_1$

$\frac{\partial a_2}{\partial h_1}=w_2$

$\frac{\partial h_1}{\partial a_1}=h_1\ast (1-h_1)$

$\frac{\partial a_1}{\partial w_1}=x$

1. 求$x_1$

代码

用numpy实现

import numpy as np

train_x_dim = 5
sample_1_num = 10
sample_0_num = 10
weight1_dim = 3
weight2_dim = 2

train_x_1 = np.random.rand(sample_1_num, train_x_dim)
train_x_0 = np.random.rand(sample_0_num, train_x_dim)*10

train_y_1 = np.ones(sample_1_num)
train_y_0 = np.zeros(sample_0_num)


weight1 = np.random.rand(train_x_dim, weight1_dim)

def sigmoid(x):
    return 1/(1+np.exp(-x))

a1 = np.dot(train_x_1, weight1)
h1 = sigmoid(a1)

weight2 = np.random.rand(weight1_dim, weight2_dim)
a2 = np.dot(h1, weight2)
h2 = sigmoid(a2)

def sigmoid_derv(x):
    return sigmoid(x)*(1-sigmoid(x))

用tf实现

from tensorflow import keras
# load data
fashion_mnist = keras.datasets.fashion_mnist
(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
# build model
model = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation=tf.nn.relu),
    keras.layers.Dense(10, activation=tf.nn.softmax)
])
# compile model
model.compile(optimizer=tf.train.AdamOptimizer(),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
# train model
model.fit(train_images, train_labels, epochs=5)
# evaluate
test_loss, test_acc = model.evaluate(test_images, test_labels)