L1正则化 L2正则化的Python 实现

上一篇文档 https://blog.csdn.net/xingzhe2001/article/details/86316712 介绍了L1 L2正则化

本文介绍L1, L2 正则化的实现

L1正则化

代码

def L1Norm(l, theta):
    return  np.dot(np.abs(theta), np.ones(theta.size)) * l

def L1NormPartial(l, theta):
    return np.sign(theta) * l

以线性回归为例，J函数现在变成了

L2正则化

def L2Norm(l, theta):
    return  np.dot(theta, theta) * l 

def L2NormPartial(l, theta):
    return theta * l

以线性回归为例 J函数变为

    def __Jfunction(self):        
        sum = 0
        
        for i in range(0, self.m):
            err = self.__error_dist(self.x[i], self.y[i])
            sum += np.dot(err, err)
        
        sum += Regularization.L2Norm(0.8, self.theta)
        
        return 1/(2 * self.m) * sum

J函数导数变为

    def __partialderiv_J_func(self):
        sum = 0

        for i in range(0, self.m):
            err = self.__error_dist(self.x[i], self.y[i])
            sum += np.dot(self.x[i], err)

        sum += Regularization.L2NormPartial(0.8, self.theta)
   
        return 1/self.m * sum