【吴恩达·2022机器学习专项课程】Logistic Regression - Cost Function

Cost function

Recall that for logistic regression, the cost function is of the form

$$J(\mathbf{w},b)=\frac{1}{m}\sum _{i=0}^{m-1}\left[loss(f_{\mathbf{w},b}({\mathbf{x}}^{(i)}),y^{(i)})\right]\text{\hspace{1em}(1)}$$

where

• $loss(f_{\mathbf{w},b}({\mathbf{x}}^{(i)}),y^{(i)})$ is the cost for a single data point, which is:

  $$loss(f_{\mathbf{w},b}({\mathbf{x}}^{(i)}),y^{(i)})=-y^{(i)}\log \left(f_{\mathbf{w},b}\left({\mathbf{x}}^{(i)}\right)\right)-\left(1-y^{(i)}\right)\log \left(1-f_{\mathbf{w},b}\left({\mathbf{x}}^{(i)}\right)\right)\text{\hspace{1em}(2)}$$

• where m is the number of training examples in the data set and:
  $$\begin{array}{rl}{f}_{\mathbf{w},b}({\mathbf{x}}^{(\mathbf{i})})& =g(z^{(i)})\\ z^{(i)}& =\mathbf{w}\cdot {\mathbf{x}}^{(i)}+b\\ g(z^{(i)})& =\frac{1}{1+e^{-z^{(i)}}}\end{array}\text{\hspace{1em}(3)(4)(5)}$$

Code Description

The algorithm for compute_cost_logistic loops over all the examples calculating the loss for each example summing.

Note that the variables X and y are not scalar values but matrices of shape ($m,n$) and ($m$,) respectively, where $n$ is the number of features and $m$ is the number of training examples.

def compute_cost_logistic(X, y, w, b):
    """
    Computes cost

    Args:
      X (ndarray (m,n)): Data, m examples with n features
      y (ndarray (m,)) : target values
      w (ndarray (n,)) : model parameters  
      b (scalar)       : model parameter
      
    Returns:
      cost (scalar): cost
    """

    m = X.shape[0]   # m - the number of training examples
    cost = 0.0       # initialize cost with 0
    
    # loops over all the examples calculating the loss for each example and sum up
    for i in range(m):
        z_i = np.dot(X[i],w) + b # calculate z - (4)
        f_wb_i = sigmoid(z_i)    # calculate f - (3)&(5)
        cost +=  -y[i]*np.log(f_wb_i) - (1-y[i])*np.log(1-f_wb_i) # calculate loss - (2) and add to cost
             
    cost = cost / m  # divide m to get the averaged cost
    return cost