Hard-Mish

场景一: Hard-Mish

…

场景一: Hard-Mish

class Mish(nn.Module):
    @staticmethod
    def forward(x):
        # x*tanh( ln(1+e^x) )
        return x * F.softplus(x).tanh()

class Hard_Mish(nn.Module):
    @staticmethod
    def forward(x):
        return x *F.hardtanh( F.softplus(x))
        #return  x * F.softplus(x)   if  0<F.softplus(x)<1 else x

实验结果:

you did it