医疗影像分割基础（一）激活函数复现

智能之心

已于 2022-05-13 09:31:18 修改

阅读量595

点赞数 1

分类专栏： # 图像分割专栏 # 网络基础专栏文章标签：深度学习人工智能

于 2022-05-12 13:23:08 首次发布

本文链接：https://blog.csdn.net/weixin_41275726/article/details/124728314

版权

图像分割专栏同时被 2 个专栏收录

14 篇文章 10 订阅

订阅专栏

网络基础专栏

1 篇文章 0 订阅

订阅专栏

深度学习网络搭建常用激活函数的详细介绍，新学一个记录一个。

激活函数 Relu

$\LARGE {\color{Red} \text{Relu}(x) = \max(0,x)}$

# ReLU(x) 函数实现
import math
import numpy as np
import matplotlib.pyplot as plt

x = np.arange(-10, 10, 0.1)
alpha = 1  # 控制着ELU负值部分在何时饱和
ReLU = np.maximum(0, x)
plt.title("function ReLU(x)")
plt.grid()
plt.xlabel('x')
plt.ylabel('y')
plt.xlim([-4, 4])
plt.ylim([-2, 3])
plt.xticks(np.arange(-4, 4, 1))
plt.yticks(np.arange(-2, 3, 1))
plt.plot(x, ReLU, "r-", linewidth=2)
plt.legend(['ReLU(x)'])
plt.plot(x, np.zeros(len(x)), "black", linewidth=1)
plt.plot(np.zeros(len(x)), x, "black", linewidth=1)
plt.savefig('ReLU.png', dpi=180)

激活函数 PRelu

$\LARGE {\color{Red} \text{PReLU}(x) = \max(0,x) + \text{weight} * \min(0,x)}$

# PRelu(x) 函数实现
import math
import numpy as np
import matplotlib.pyplot as plt

x = np.arange(-10, 10, 0.1)
weight = 0.1  # 控制着weight负值部分不为 0
PRelu = np.maximum(0, x) + weight * np.minimum(x, 0)
plt.title("function PRelu(x)")
plt.grid()
plt.xlabel('x')
plt.ylabel('y')
plt.xlim([-4, 4])
plt.ylim([-2, 3])
plt.xticks(np.arange(-4, 4, 1))
plt.yticks(np.arange(-2, 3, 1))
plt.plot(x, PRelu, "r-", linewidth=2)
plt.legend(['PRelu(x)'])
plt.plot(x, np.zeros(len(x)), "black", linewidth=1)
plt.plot(np.zeros(len(x)), x, "black", linewidth=1)
plt.savefig('PRelu.png', dpi=180)

激活函数 ELU

$\LARGE {\color{Red} \text{ELU}(x) = \max(0,x) + \min(0, \alpha * (\exp(x) - 1))}$

# ELU(x) 函数实现
import math
import numpy as np
import matplotlib.pyplot as plt

x = np.arange(-10, 10, 0.1)
alpha = 1  # 控制着ELU负值部分在何时饱和
elu = np.maximum(0, x) + np.minimum(0, alpha * (np.exp(x) - 1))
plt.title("function ELU(x)")
plt.grid()
plt.xlabel('x')
plt.ylabel('y')
plt.xlim([-4, 4])
plt.ylim([-2, 3])
plt.xticks(np.arange(-4, 4, 1))
plt.yticks(np.arange(-2, 3, 1))
plt.plot(x, elu, "r-", linewidth=2)
plt.legend(['ELU(x)'])
plt.plot(x, np.zeros(len(x)), "black", linewidth=1)
plt.plot(np.zeros(len(x)), x, "black", linewidth=1)
plt.savefig('ELU.png', dpi=180)

激活函数 SELU

$\LARGE {\color{Red} \text{SELU}(x) = scale * (\max(0,x) + \min(0, \alpha * (\exp(x) - 1)))}$

# SELU(x) 函数实现 - 可以不用加入BN层
import math
import numpy as np
import matplotlib.pyplot as plt

x = np.arange(-10, 10, 0.1)
alpha = 1.6732632423543772848170429916717
scale = 1.0507009873554804934193349852946
SELU = scale * (np.maximum(0, x) + np.minimum(0, alpha*(np.exp(x)-1)))
plt.title("function SELU(x)")
plt.grid()
plt.xlabel('x')
plt.ylabel('y')
plt.xlim([-4, 4])
plt.ylim([-2, 3])
plt.xticks(np.arange(-4, 4, 1))
plt.yticks(np.arange(-2, 3, 1))
plt.plot(x, SELU, "r-", linewidth=2)
plt.legend(['SELU(x)'])
plt.plot(x, np.zeros(len(x)), "black", linewidth=1)
plt.plot(np.zeros(len(x)), x, "black", linewidth=1)
plt.savefig('SELU.png', dpi=180)