python学习日记18keras训练发生了奇怪的事情

使用

train_history = model.fit(x=x_train_normalize,
y=y_trainOnehot,
validation_split = 0.2,
epochs=10,
batch_size=200,
verbose=1,
validation_data=(x_test_normalize,y_testOnehot))

训练，因为在anaconda中，

Layer (type) Output Shape Param #
===========================================================
dense (Dense) (None, 256) 200960

---

dense_1 (Dense) (None, 10) 2570
===========================================================

不会自动清理变量内存，加入了tf.reset_default_graph()之后，连执行结果都变了
另外理论上应该train的精度高误差小，但是实际执行的结果却是相反的,test效果更好。如果epoch增加的20-25，在最后train才会有微弱优势。
keras官网有说明，https://keras.io/getting-started/faq/#why-is-the-training-loss-much-higher-than-the-testing-loss

Why is the training loss much higher than the testing loss?
A Keras model has two modes: training and testing. Regularization mechanisms, such as Dropout and L1/L2 weight regularization, are turned off at testing time.
Besides, the training loss is the average of the losses over each batch of training data. Because your model is changing over time, the loss over the first batches of an epoch is generally higher than over the last batches. On the other hand, the testing loss for an epoch is computed using the model as it is at the end of the epoch, resulting in a lower loss.

对keras内部略有困惑
需要用tensorflow做出同样的分析图，然后对比，才能确定是数据本身的原因，还是训练方法的原因。
尝试tensorflow和keras做同样的训练后，考虑到过拟合时，train比test效果好。同时有如下现象：
1keras不太稳定，不用tf.reset_default_graph()，acc和loss曲线比较平滑。用了之后有些明显的折线。有时修改了图的隐层tensor数量，报错。修改回来也仍然报错，清除图和变量也没用。重新启动spyder，就会有正常结果。
2在对比tensorflow和keras时，二者使用的层数和tensor数即使相同，结果趋势也有明显差异。选择时要综合考虑开发速度和可控性。
3有些训练结果是train优于test,和epoch也有关系。二者逐渐趋近。
结果如图举例


参考文档：https://keras.io/models/sequential/