线性回归及逻辑回归算法的推导、手动实现逻辑回归

逻辑回归相当于将多元线性回归的结果利用激活函数映射到0-1之间，表达一种成功的概率，推倒逻辑回归需要先推导出线性回归的结果

线性回归算法推倒

根据假设，线性回归的误差服从均值为0的正态分布，即$p\left({ϵ}^{(i)}\right)=\frac{1}{\sqrt{2\pi }\sigma }\exp \left(-\frac{{\left({ϵ}^{(i)}\right)}^2}{2{\sigma }^2}\right)$
再根据$y^{(i)}={\theta }^T{x}^{(i)}+{\epsilon }^{(i)}$

得出$$p\left(y^{(i)}|x^{(i)};\theta \right)=\frac{1}{\sqrt{2\pi }\sigma }\exp \left(-\frac{{\left(y^{(i)}-{\theta }^T{x}^{(i)}\right)}^2}{2{\sigma }^2}\right)$$
得出似然函数$$L(\theta )=\prod _{i=1}^{m}p\left(y^{(i)}|x^{(i)};\theta \right)=\prod _{i=1}^m\frac{1}{\sqrt{2\pi }\sigma }\exp \left(-\frac{{\left(y^{(i)}-{\theta }^T{x}^{(i)}\right)}^2}{2{\sigma }^2}\right)$$
取对数后$$\log L(\theta )=\log \prod _{i=1}^m\frac{1}{\sqrt{2\pi }\sigma }\exp \left(-\frac{{\left(y^{(i)}-{\theta }^T{x}^{(i)}\right)}^2}{2{\sigma }^2}\right)$$
展开化简 ∑ i = 1 m log ⁡ 1 2 π γ σ exp ⁡ ( − ( y ( i ) − θ T x ( i ) ) 2 2 σ 2 ) = m log ⁡ 1 2 π σ − 1 σ 2 ⋅ 1 2 ∑ i = 1 m ( y ( i ) − θ T x ( i ) ) 2 \begin{array}{l}{\sum_{i=1}^{m} \log \frac{1}{\sqrt[\gamma]{2 \pi} \sigma} \exp \left(-\frac{\left(y^{(i)}-\theta^{T} x^{(i)}\right)^{2}}{2 \sigma^{2}}\right)} \\ {=m \log \frac{1}{\sqrt{2 \pi} \sigma}-\frac{1}{\sigma^{2}} \cdot \frac{1}{2} \sum_{i=1}^{m}\left(y^{(i)}-\theta^{T} x^{(i)}\right)^{2}}\end{array} ∑i=1m​logγ2π ​σ1​exp(−2σ2(y(i)−θTx(i))2​)=mlog2π ​σ1​−σ21​⋅21​∑