python神经网络训练损失率nan_Python神经网络中不需要的[Nan]输出

newbie here. Just switched over from JS to Python to build Neural nets but getting [Nan] outputs from it.

The weird thing is that my sigmoid func. doesn't seem to encounter any overflow but the derivative causes chaos.

import numpy as np

def sigmoid(x):

return x*(1-x)

return 1/(1 + np.exp(-x))

#The function- 2

def Sigmoid_Derivative(x):

return x * (1-x)

Training_inputs = np.array([[0,0,1],

[1,1,1],

[1,0,1],

[0,1,1]])

Training_outputs = np.array([[0, 1, 1, 0]]).T

np.random.seed(1)

synaptic_weights = np.random.random((3, 1)) - 1

print ("Random starting synaptic weight:")

print (synaptic_weights)

for iteration in range(20000):

Input_Layer = Training_inputs

Outputs = sigmoid(np.dot(Input_Layer, synaptic_weights))

erorr = Training_outputs - Outputs

adjustments = erorr * Sigmoid_Derivative(Outputs)

synaptic_weights += np.dot(Input_Layer.T, adjustments)

# The print declaration----------

print ("Synaptic weights after trainig:")

print (synaptic_weights)

print ("Outputs after training: ")

print (Outputs)

This is the erorr message. I dunno why it Overflowing because the weights seem to be small enough.BTWPls give solutions in simple python as I am a newbie :--

Random starting synaptic weight:

[[-0.582978 ]

[-0.27967551]

[-0.99988563]]

/home/neel/Documents/VS-Code_Projects/Machine_Lrn(PY)/tempCodeRunnerFile.py:10: RuntimeWarning: overflow encountered in multiply

return x * (1-x)

Synaptic weights after trainig:

[[nan]

[nan]

[nan]]

Outputs after training:

[[nan]

[nan]

[nan]

[nan]]

解决方案

There are at least two issues with your code.

The first is the inexplicable use of 2 return statements in your sigmoid function, which should simply be:

def sigmoid(x):

return 1/(1 + np.exp(-x))

which gives the correct result for x=0 (0.5), and goes to 1 for large x:

sigmoid(0)

# 0.5

sigmoid(20)

# 0.99999999793884631

Your (wrong) sigmoid:

def your_sigmoid(x):

return x*(1-x)

return 1/(1 + np.exp(-x))

can easily lead to overflow:

your_sigmoid(20)

# -380

The other issue is that your derivative is wrong; it should be:

def Sigmoid_Derivative(x):

return sigmoid(x) * (1-sigmoid(x))

See the Derivative of sigmoid function thread at Math.SE, as well as the discussion here.