# 回归分析之逻辑回归-Logistic Regression

Github：https://github.com/thinkgamer

# Sigmoid函数

## 数学表达式和图形

f ( x ) = 1 1 + e − x f(x)=\frac{1}{1+e^{-x}}

import math
import matplotlib.pyplot as plt

def sigmoid(x):
return 1 / (1 + math.exp(-x))

# python2 中range生成的是一个数组，py3中生成的是一个迭代器，可以使用list进行转换
X = list(range(-9,10))
Y = list(map(sigmoid,X))

#画图
plt.plot(X,Y)
plt.show()


![sigmoid](https://imgconvert.csdnimg.cn/aHR0cDovL2ltZy5ibG9nLmNzZG4ubmV0LzIwMTcxMjE5MTkyMzI5MzI4?x-oss-process=image/format,png)

## LR为什么要使用Sigmoid

p ( y ∣ x ) = P ( x ∣ y ) P ( y ) ∑ P ( x ∣ y ) P ( y ) p(y|x) = \frac{P(x|y)P(y)}{\sum{P(x|y)P(y)}}

log ⁡ P ( y = 1 ∣ x ) P ( y = 0 ∣ x ) = log ⁡ P ( x ∣ y = 1 ) P ( x ∣ y = 0 ) + log ⁡ P ( y = 1 ) P ( y = 0 )   = − ( x − μ 1 ) 2 2 σ 1 2 + ( x − μ 0 ) 2 2 σ 0 2   + θ 0 \log\frac{P(y=1|x)}{P(y=0|x)} = \log\frac{P(x|y=1)}{P(x|y=0)} + \log\frac{P(y=1)}{P(y=0)} \ = -\frac{(x-\mu_1)^2}{2\sigma_1^2} + \frac{(x-\mu_0)^2}{2\sigma_0^2}\ + \theta_0

log ⁡ P ( y = 1 ∣ x ) P ( y = 0 ∣ x ) = θ T x \log\frac{P(y=1|x)}{P(y=0|x)} = \theta^T x

P ( y = 1 ∣ x ) = e θ T x 1 + e θ T x = 1 1 + e − θ T x P(y=1|x) = \frac{e^{\theta^T x}}{1+e^{\theta^T x}} = \frac{1}{1+e^{-\theta^T x}}

# Logistic Regression 分类器

P ( y = 1 ∣ x ; θ ) = g ( θ T x ) = 1 1 + e − θ T ∗ x P(y=1|x;\theta) = g(\theta^T x) = \frac{1}{1 + e ^ {-\theta^T * x}}

y ∗ = 1 ,   if   P ( y = 1 ∣ x ) > 0.5 y^* = 1, \, \textrm{if} \, P(y=1|x) > 0.5

# 参数求解

L ( θ ) = P ( D ∣ θ ) = ∏ P ( y ∣ x ; θ ) = ∏ g ( θ T x ) y ( 1 − g ( θ T x ) ) 1 − y L(\theta) = P(D|\theta) = \prod P(y|x;\theta) = \prod g(\theta^T x) ^ y (1-g(\theta^T x))^{1-y}

l ( θ ) = ∑ y log ⁡ g ( θ T x ) + ( 1 − y ) log ⁡ ( 1 − g ( θ T x ) ) l(\theta) = \sum {y\log{g(\theta^T x)} + (1-y)\log{(1-g(\theta^T x))}}

− y log ⁡ p ( y ∣ x ) − ( 1 − y ) log ⁡ 1 − p ( y ∣ x ) -y\log{p(y|x)}-(1-y)\log{1-p(y|x)}

J ( θ ) = − 1 N l ( θ ) J(\theta) = -\frac{1}{N} l(\theta)

• 选择下降方向（梯度方向， ∇ J ( θ ) \nabla {J(\theta)}
• 选择步长，更新参数 θ i = θ i − 1 − α i ∇ J ( θ i − 1 ) \theta^i = \theta^{i-1} - \alpha^i \nabla {J(\theta^{i-1})}
• 重复以上两步直到满足终止条件

∂ J ∂ θ = − 1 n ∑ i ( y i − y i ∗ ) x i + λ θ \frac{\partial{J}}{\partial{\theta}} = -\frac{1}{n}\sum_i (y_i - y_i^*)x_i + \lambda \theta

# 资料参考

http://blog.csdn.net/han_xiaoyang/article/details/49123419

https://www.cnblogs.com/sxron/p/5489214.html

https://tech.meituan.com/intro_to_logistic_regression.html

【技术服务】，详情点击查看： https://mp.weixin.qq.com/s/PtX9ukKRBmazAWARprGIAg

Thinkgamer_ CSDN认证博客专家 图书作者 推荐系统研究者
Thinkgamer，著有《推荐系统开发实战》，「搜索与推荐Wiki」公众号独立作者，愿景是持续输出精品系列内容，让知识能够平等的到达每一个人。从事推荐系统相关工作多年！

11-26

12-30 1万+
01-04
12-04
11-12 2万+
11-10 21万+
04-27
07-16 28万+
10-14 3432