Loss Function

• Def. - what it is

•  It’s a method of evaluating how well your algorithm models your dataset.
• Output a higher number -> bad prediction
• Output a smaller number -> good prediction 

• Types of loss functions :

• Mean Squared Error :

the avearage of the distance(the square of the difference) between the prediction and the ground truth across the whole dataset

def MSE(y_predicted, y):
squared_error = (y_predicted - y) ** 2
sum_squared_error = np.sum(squared_error)
mse = sum_squared_error / y.size
return(mse)

• Likelihood loss (commonly used in classification problemes)

 The function takes the predicted probability for each input example and multiplies them. 

     label        [0,   1,   1,   0]
predicted outputs [0.4, 0.6, 0.9, 0.1]
                   1-p    p   p   1-p   
likelihood loss:
(1-0.4) * (0.6) * (0.9) * (1-0.1) = 0.2916

"We multiply the model's outputted probabilities together for the actual ouotcomes."

• Log Loss(Cross Entropy Loss, also used frequently in classification problemes
  • Log Loss is the most important classification metric based on probabilities.
  • It's hard to interpret raw log-loss values, but log-loss is still a good metric for comparing models. For any given problem, a lower log-loss value means better predictions.
  • Log Loss is a slight twist on something called the Likelihood Function. 

                        Log Loss = (-1) * log (Likelyhood loss)

" It penalizes heavily for being  very confident and  very wrong."

• Loss Function & Optimizers (Gradient Descent for ex.) work toghter to fit the algorithm to the dataset in the best way possible.