机器学习算法（一）: 基于逻辑回归的分类预测①

理论探究

The goal of binary logistic regression is to train a classifier that can make a binary decision about the class of a new input observation.

Consider a single input observation x, which we will represent by a vector of features
$$[x1,x2,...,xn]$$
The classifier output y can be 1 (meaning the observation is a member of the class) or 0(the observation is not a member of the class).

We want to know the probability
$$P(y=1|x)$$
And logistic regression (LR) solves this task by learning, from a training set, a vector of weights and a bias term.

After we’ve learned the weights in training, the resulting single number z expresses the weighted sum of the evidence for the class
$$z=(\sum _{i=1}^n{w}_i{x}_i)+b$$
In the rest of the book we’ll represent such sums using the dot product notation from linear algebra. The dot product of two vectors a and b, written as a·b is the sum of the products of the corresponding elements of each vector. Thus we have the following formation
$$z=w\cdot x+b$$
It’s obvious that z ranges from −∞ to ∞. But we hope that z lies between 0 and 1. So, we use the sigmoid function.
$$\begin{gathered} y=\sigma (z)=\frac{1}{1+e^{-z}} \\ \underset{z\to \infty }{\lim }\frac{1}{1+e^{-z}}=1 \\ \underset{z\to -\infty }{\lim }\frac{1}{1+e^{-z}}=0 \end{gathered}$$
We’re almost there. If we apply the sigmoid to the sum of the weighted features,we get a number between 0 and 1. To make it a probability, we just need to make sure that the two cases,p(y=1) and p(y=0), sum to 1. We can do this as follows:
$$\begin{gathered} P(y=1)=\sigma (w\cdot x+b)=\frac{1}{1+e^{-w\cdot x+b}} \\ P(y=0)=1-\sigma (w\cdot x+b)=\frac{e^{-w\cdot x+b}}{1+e^{-w\cdot x+b}} \end{gathered}$$
Now we have an algorithm that given an instance x computes the probability P(y=1|x). For a test instance x, we say yes if the probability P(y=1|x) is more than 0.5, and no otherwise. We call 0.5 the decision boundary:
$$\begin{gathered} \hat{y}=1ifP(y=1|x)>0.5 \\ \hat{y}=0otherwise \end{gathered}$$