简单神经网络入门-损失率为nan

现象

运行一个简单到极点的神经网络，输入特征只有1个，训练数据共200个，50个一组共4组，对应4个标签[0,1,2,6]。

用Tensorflow搭了一个两层的神经网络，第一层对应数据维度，第二层对应标签维度。就这么一个简单的网络还报错，出现了loss为nan的情况，如下。。。。

Output exceeds the size limit. Open the full output data in a text editor
Epoch 1/50
7/7 [==============================] - 0s 3ms/step - loss: nan - accuracy: 0.2500
Epoch 2/50
7/7 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.2500
Epoch 3/50
7/7 [==============================] - 0s 3ms/step - loss: nan - accuracy: 0.2500
Epoch 4/50
7/7 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.2500
Epoch 5/50
7/7 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.2500
Epoch 6/50
7/7 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.2500
Epoch 7/50
7/7 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.2500
Epoch 8/50
7/7 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.2500
Epoch 9/50
7/7 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.2500
Epoch 10/50
7/7 [==============================] - 0s 3ms/step - loss: nan - accuracy: 0.2500
Epoch 11/50
7/7 [==============================] - 0s 4ms/step - loss: nan - accuracy: 0.2500
Epoch 12/50
7/7 [==============================] - 0s 3ms/step - loss: nan - accuracy: 0.2500
Epoch 13/50
...

Test loss nan

Test accuracy 0.25

原因

检查发现是标签[0,1,2,6]中，6这个标签与其他三个标签的跨度太大了，两层简单的神经网络没法处理这种“非线性”的标签跨度。将标签6改为标签3，训练效果就正常了，结果如下：

Output exceeds the size limit. Open the full output data in a text editor
Epoch 1/50
7/7 [==============================] - 0s 4ms/step - loss: 1.1826 - accuracy: 0.3700
Epoch 2/50
7/7 [==============================] - 0s 4ms/step - loss: 0.9503 - accuracy: 0.5000
Epoch 3/50
7/7 [==============================] - 0s 4ms/step - loss: 0.8174 - accuracy: 0.5000
Epoch 4/50
7/7 [==============================] - 0s 4ms/step - loss: 0.7410 - accuracy: 0.5000
Epoch 5/50
7/7 [==============================] - 0s 4ms/step - loss: 0.6902 - accuracy: 0.7000
Epoch 6/50
7/7 [==============================] - 0s 4ms/step - loss: 0.6595 - accuracy: 0.7500
Epoch 7/50
7/7 [==============================] - 0s 3ms/step - loss: 0.6346 - accuracy: 0.7500
Epoch 8/50
7/7 [==============================] - 0s 4ms/step - loss: 0.6140 - accuracy: 0.7500
Epoch 9/50
7/7 [==============================] - 0s 3ms/step - loss: 0.5949 - accuracy: 0.7500
Epoch 10/50
7/7 [==============================] - 0s 4ms/step - loss: 0.5765 - accuracy: 0.7650
Epoch 11/50
7/7 [==============================] - 0s 3ms/step - loss: 0.5564 - accuracy: 0.8050
Epoch 12/50
7/7 [==============================] - 0s 4ms/step - loss: 0.5372 - accuracy: 0.7700
Epoch 13/50
...

Test loss 0.040257006883621216

Test accuracy 1.0

其他

python渣一个在这里也记一下，标签是这么修改的：

label1[np.where(label1==6)] = 3

关于损失函数：
在简单神经网络中，损失函数用来表示神经网络输出与实际标签值之间的差距。
常见的损失函数有softmax等等。