机器学习8/100天-Logistic回归原理与实现

最新推荐文章于 2024-05-02 11:42:15 发布

zombee0

最新推荐文章于 2024-05-02 11:42:15 发布

阅读量208

点赞数

分类专栏：编程

本文链接：https://blog.csdn.net/zombee0/article/details/85540744

版权

编程专栏收录该内容

72 篇文章 0 订阅

订阅专栏

Day 8 Logistic回归原理与实现

github: 100DaysOfMLCode

最大似然函数

$\prod_{i=0}^np^{y_i}(1-p)^{1-y_i}$
取对数得
$\sum_{i=0}^ny_ilog(p)+(1-y_i)log(1-p)$
$\frac{1}{1+e^{z}}$
$z = w x + b$
因此梯度下降法求偏导数可得：
$\frac{\partial{L}}{\partial{w}}=\frac{\partial{L}}{\partial{p}}\frac{\partial{p}}{\partial{z}}\frac{\partial{z}}{\partial{w}}=(\frac{-y}{p}+\frac{1-y}{1-p})(p(1-p))x=(a-y)x$
$\frac{\partial{L}}{\partial{b}}=(a-y)$

def weightInitialization(n_features):
    w = np.zeros((1,n_features))
    b = 0
    return w,b
def sigmoid_activation(result):
    final_result = 1/(1+np.exp(-result))
    return final_result

def model_optimize(w, b, X, Y):
    m = X.shape[0]
    
    #Prediction
    final_result = sigmoid_activation(np.dot(w,X.T)+b)
    Y_T = Y.T
    cost = (-1/m)*(np.sum((Y_T*np.log(final_result)) + ((1-Y_T)*(np.log(1-final_result)))))
    #
    
    #Gradient calculation
    dw = (1/m)*(np.dot(X.T, (final_result-Y.T).T))
    db = (1/m)*(np.sum(final_result-Y.T))
    
    grads = {"dw": dw, "db": db}
    
    return grads, cost
def model_predict(w, b, X, Y, learning_rate, no_iterations):
    costs = []
    for i in range(no_iterations):
        #
        grads, cost = model_optimize(w,b,X,Y)
        #
        dw = grads["dw"]
        db = grads["db"]
        #weight update
        w = w - (learning_rate * (dw.T))
        b = b - (learning_rate * db)
        #
        
        if (i % 100 == 0):
            costs.append(cost)
            #print("Cost after %i iteration is %f" %(i, cost))
    
    #final parameters
    coeff = {"w": w, "b": b}
    gradient = {"dw": dw, "db": db}
    
    return coeff, gradient, costs
def predict(final_pred, m):
    y_pred = np.zeros((1,m))
    for i in range(final_pred.shape[1]):
        if final_pred[0][i] > 0.5:
            y_pred[0][i] = 1
    return y_pred