logistics Regression using Gradient Descent

回归来源于"线性回归"的z=XW,使用线性回归去拟合逼近一个’界’,使得按照这个界进行数据分类后得到的cost最小(决策边界).

Function:
for two way classification and give it realtive probablities.
such as

• differentiate whether it is junk mail
• distinguish whether this customer will buy the product

it can be extended to muti-classfication problems

Function Sigmoid

this function can accept all the input and map it to 0 or 1, suitable for classification

• 0≤hθ(x)≤1

it mainly because -z to the power of e is always smaller than 1 and the denominator is always bigger than 1 , as a result , it always smaller than 1, In other words, it can be seen as probability.

w0 is intercept(截距), w1 w2 is coefficient(系数)
the main formula to caculate weights

Cost for classfication

when the label == 1

cost = -log(h)

h in this formula is probabilty been classified as label 1

when the label == 0

cost = -log(1-h)

h in this formula is probabilty been classified as label 0

P(y=1|x;θ) 表示 x在样本θ下为1的概率

so the total cost of classification is

接下来就是找出最小的θ，使得cost function达到最小