场景一: Hard-Mish
…
场景一: Hard-Mish
class Mish(nn.Module):
@staticmethod
def forward(x):
# x*tanh( ln(1+e^x) )
return x * F.softplus(x).tanh()
class Hard_Mish(nn.Module):
@staticmethod
def forward(x):
return x *F.hardtanh( F.softplus(x))
#return x * F.softplus(x) if 0<F.softplus(x)<1 else x
实验结果:
you did it