Week3_逻辑回归---二分类问题

二. 逻辑回归

逻辑回归，也就是预测的结果进行分类的问题

1 . 简单的二分类问题 分为 0 和 1

对于线性回归的问题，我们得到的是一个具体大小的输出值，我们需要确定一个阈值，高于这个阈值我们认为是 1 ，低于这个阈值 我们认为是 0 。

我们引入了一个函数，来对原先的输出进行隶属度分类

* sigmoid函数*

sigmoid函数的代码实现

def sigmoid(z):
    """
    Compute the sigmoid of z

    Args:
        z (ndarray): A scalar, numpy array of any size.

    Returns:
        g (ndarray): sigmoid(z), with the same shape as z

    """

    g = 1/(1+np.exp(-z))

    return g

利用sigmoid函数，此时的输出值就变为（0,1）之间的数字

线性回归模型就变为下图所示，

根据表达是和sigmoid函数图像我们可以看出 当自变量z大于等于0是 函数值近似为1 ，z小于0是函数值近似为0

但是我们要确定的是 w 和 b 的值
$$w\ast x+b>=0，y=1$$

$$w\ast x+b<0，y=0$$

逻辑回归的成本函数

此时的成本函数并不是凸函数，无法正确的找到为一极值

$$J(w,b)=\frac{1}{2m}\sum _{i=0}^{m-1}(f_{w,b}(x^{(i)})-y^{(i)}{)}^2$$

$$f_{w,b}(x^{(i)})=sigmoid(w{x}^{(i)}+b)$$

逻辑回归的损失函数

为了解决成本函数不是凸函数的问题我们重新定义一个损失函数，让损失函数代替成本函数进行梯度下降

$$loss(f_{\mathbf{w},b}({\mathbf{x}}^{(i)}),y^{(i)})=(-y^{(i)}\log \left(f_{\mathbf{w},b}\left({\mathbf{x}}^{(i)}\right)\right)-\left(1-y^{(i)}\right)\log \left(1-f_{\mathbf{w},b}\left({\mathbf{x}}^{(i)}\right)\right)$$

逻辑回归的成本函数代码实现

def compute_cost_logistic(X, y, w, b):
    """
    Computes cost

    Args:
      X (ndarray (m,n)): Data, m examples with n features
      y (ndarray (m,)) : target values
      w (ndarray (n,)) : model parameters  
      b (scalar)       : model parameter

    Returns:
      cost (scalar): cost
    """

    m = X.shape[0]
    cost = 0.0
    for i in range(m):
        z_i = np.dot(X[i],w) + b
        f_wb_i = sigmoid(z_i)
        cost +=  -y[i]*np.log(f_wb_i) - (1-y[i])*np.log(1-f_wb_i)

    cost = cost / m
    return cost

逻辑回归的梯度下降

对于导数的求解与线性回归的求解步骤一样，只不过将函数换成了sigmoid之后的函数

求导数的代码实现

def compute_gradient_logistic(X, y, w, b): 
    """
    Computes the gradient for linear regression 

    Args:
      X (ndarray (m,n): Data, m examples with n features
      y (ndarray (m,)): target values
      w (ndarray (n,)): model parameters  
      b (scalar)      : model parameter
    Returns
      dj_dw (ndarray (n,)): The gradient of the cost w.r.t. the parameters w. 
      dj_db (scalar)      : The gradient of the cost w.r.t. the parameter b. 
    """
    m,n = X.shape
    dj_dw = np.zeros((n,))                           #(n,)
    dj_db = 0.

    for i in range(m):
        f_wb_i = sigmoid(np.dot(X[i],w) + b)          #(n,)(n,)=scalar
        err_i  = f_wb_i  - y[i]                       #scalar
        for j in range(n):
            dj_dw[j] = dj_dw[j] + err_i * X[i,j]      #scalar
        dj_db = dj_db + err_i
    dj_dw = dj_dw/m                                   #(n,)
    dj_db = dj_db/m                                   #scalar

    return dj_db, dj_dw

梯度下降的代码实现

def gradient_descent(X, y, w_in, b_in, alpha, num_iters): 
    """
    Performs batch gradient descent
    
    Args:
      X (ndarray (m,n)   : Data, m examples with n features
      y (ndarray (m,))   : target values
      w_in (ndarray (n,)): Initial values of model parameters  
      b_in (scalar)      : Initial values of model parameter
      alpha (float)      : Learning rate
      num_iters (scalar) : number of iterations to run gradient descent
      
    Returns:
      w (ndarray (n,))   : Updated values of parameters
      b (scalar)         : Updated value of parameter 
    """
    # An array to store cost J and w's at each iteration primarily for graphing later
    J_history = []
    w = copy.deepcopy(w_in)  #avoid modifying global w within function
    b = b_in
    
    for i in range(num_iters):
        # Calculate the gradient and update the parameters
        dj_db, dj_dw = compute_gradient_logistic(X, y, w, b)   

        # Update Parameters using w, b, alpha and gradient
        w = w - alpha * dj_dw               
        b = b - alpha * dj_db               
      
        # Save cost J at each iteration
        if i<100000:      # prevent resource exhaustion 
            J_history.append( compute_cost_logistic(X, y, w, b) )

        # Print cost every at intervals 10 times or as many iterations if < 10
        if i% math.ceil(num_iters / 10) == 0:
            print(f"Iteration {i:4d}: Cost {J_history[-1]}   ")
        
    return w, b, J_history         #return final w,b and J history for graphing