代码地址:https://github.com/lessw2020/mish
1. mish的公式表达和曲线
公式表达:Mish=x * tanh(ln(1+e^x))
曲线表达,如下图所示:
![](https://img-blog.csdnimg.cn/20200622170343939.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0d1b19QeXRob24=,size_16,color_FFFFFF,t_70)
2. mish的pytorch实现
class Mish(torch.nn.Module):
def __init__(self):
super().__init__()
def forward(self, x):
x = x * (torch.tanh(torch.nn.functional.softplus(x)))
return x
效果还不错,你也试试吧!