神经网络激活函数对数函数_神经网络中的激活函数

神经网络激活函数对数函数

Activation function, as the name suggests, decides whether a neuron should be activated or not based on the addition of a bias with the weighted sum of inputs. Hence, it is a very significant component of Deep Learning, as they in a way determine the output of models. The activation function has to be efficient so the model can scale along the increase in the number of neurons.

顾名思义，激活函数根据输入的加权和加上偏倚来决定是否激活神经元。 因此，它是深度学习的重要组成部分，因为它们以某种方式确定了模型的输出。 激活功能必须高效，因此模型可以随着神经元数量的增加而缩放。

To be precise, the activation function decides how much information of the input relevant for the next stage.

确切地说，激活功能决定了与下一阶段相关的输入信息量。

For example, suppose x1 and x2 are two inputs with w1 and w2 their respective weights to the neuron. The output Y = activation_function(y).

例如，假设x1和x2是两个输入，其中w1和w2分别占神经元的权重。 输出Y = activation_function(y)。

Here, y = x1.w1 + x2.w2 + b i.e. weighted sum of inputs and bias.

y = x1.w1 + x2.w2 + b，即输入和偏置的加权和。

Activation functions are mainly of 3 types. We will analyze the curves, pros and cons of each here. The input we work on will be an arithmetic progression in [-10, 10] with a constant difference of 0.1

激活功能主要有3种。 我们将在这里分析每个曲线的优缺点。 我们正在处理的输入将是[-10，10]的算术级数，常数差为0.1

x = tf.Variable(tf.range(-10, 10, 0.1), dtype=tf.float32)

二元步 (Binary step)

A binary step function is a threshold-based activation function. If the input value is above or below a certain threshold, the neuron is activated and sends exactly the same signal to the next layer.

二进制步进函数是基于阈值的激活函数。 如果输入值高于或低于某个阈值，则神经元被激活并将完全相同的信号发送到下一层。

#Binary Step Activation
def binary_step(x):
    return np.array([1 if each > 0 else 0 for each in list(x.numpy())])
do_plot(x.numpy(), binary_step(x), 'Binary Step')

The binary step is not used mostl