神经网络激活函数对数函数_神经网络中的激活函数

最新推荐文章于 2022-10-09 11:52:37 发布

weixin_26748959

最新推荐文章于 2022-10-09 11:52:37 发布

阅读量430

点赞数

文章标签：神经网络 python 深度学习 tensorflow 人工智能

原文链接：https://medium.com/analytics-vidhya/activation-functions-in-neural-networks-bcafebfddbf2

版权

神经网络激活函数对数函数

什么是激活功能？ (What are Activation Functions?)

Activation functions in neural networks are used to define the output of the neuron given the set of inputs. These are applied to the weighted sum of the inputs and transform them into output depending on the type of activation used.

在给定输入集的情况下，神经网络中的激活函数用于定义神经元的输出。将这些应用于输入的加权总和，并根据使用的激活类型将其转换为输出。

Output of neuron = Activation(weighted sum of inputs + bias)

神经元的输出=激活(输入的加权总和+偏差)

为什么我们需要激活功能？ (Why we need Activation Functions?)

The main idea behind using activation functions is to add non-linearity.

使用激活函数的主要思想是添加非线性 。

Now, the question arises why we need non-linearity? We need neural network models to learn and represent complex functions. Thus, using activation functions in neural networks, aids in process of learning complex patterns from data and adds the capability to generate non-linear mappings from inputs to outputs.

现在，出现了一个问题，为什么我们需要非线性？我们需要神经网络模型来学习和表示复杂的功能。 因此，在神经网络中使用激活函数，有助于从数据中学习复杂模式，并增加了从输入到输出生成非线性映射的功能。

激活功能的类型 (Types of Activation Functions)

Sigmoid- It limits the input value between 0 and 1.
Sigmoid-将输入值限制在0到1之间。

Sigmoid maps the input to a small range of [0, 1]. As a result, there are large regions of the input space which are mapped to a very small range. This leads to a problem called vanishing gradient. It means that the most upstream layers will learn very slowly because essentially the computed gradient is very small due to the way the gradients are chained together.

Sigmoid将输入映射到较小的[0，1]范围。结果，输入空间中有很大的区域映射到很小的范围。这导致了一个称为消失梯度的问题。这意味着最上游的层将学习得非常缓慢，因为从本质上来说，由于将梯度链接在一起的方式，计算出的梯度非常小。

2. Tanh- It limits the value between -1 and 1.

2. Tanh-将值限制在-1和1之间。

Difference between tanh and sigmoid — Apart from the difference in the range of these activation functions, tanh function is symmetric around the origin, whereas the sigmoid function is not.

tanh和sigmoid之间的差异—除了这些激活函数范围的差异之外，tanh函数在原点周围是对称的，而sigmoid函数不是。

Both sigmoid and tanh pose vanishing gradient problems when used as activation functions in neural networks.

当在神经网络中用作激活函数时，S形和tanh都会出现消失的梯度问题。

3. ReLU(Rectified Linear Unit)- It is the most popular activation function.

3. ReLU(整流线性单位)-这是最流行的激活功能。

Outputs the same value for a positive value and zeroes out negative values.
为正值输出相同的值，并将负值清零。
It is very fast to compute (given the simplicity of logic), thus improving training time.
计算非常快(考虑到逻辑的简单性)，从而缩短了训练时间。
ReLU does not pose vanishing gradient problem.
ReLU不会造成梯度消失的问题。
It does not have a maximum value.
它没有最大值。

There are different variations of ReLU that are available like LeakyReLU, SELU, ELU, SReLU. Still, ReLU is widely used as it is simple, fast, and efficient.

ReLU有不同的变体，如LeakyReLU，SELU，ELU，SReLU。尽管如此，ReLU简单，快速且高效，因此被广泛使用。

Implementation in PyTorch

在PyTorch中实施

torch.nn.ReLU() — applies the rectified linear unit function element-wise.

torch.nn.ReLU() —将整流的线性单位函数按元素应用。