linear_regression

最新推荐文章于 2024-04-28 22:14:08 发布

wo1185246535

最新推荐文章于 2024-04-28 22:14:08 发布

阅读量537

点赞数

分类专栏：机器学习与模式识别文章标签：机器学习

本文链接：https://blog.csdn.net/yangnanhai93/article/details/51570432

版权

机器学习与模式识别专栏收录该内容

6 篇文章 0 订阅

订阅专栏

Linear Regression

The objective of linear regression is to minimise the cost fucntion:

J (θ) = 1 2 \sum i = 1 m (h θ (x (i)) - y (i)) 2

$J(\theta)=\frac{1}{2}\sum_{i=1}^{m}(h_{\theta}(x^{(i)})-y^{(i)})^{2}$

h θ = θ T x = θ 0 + θ 1 x 1 + . . . θ n x n

$h_{\theta}=\theta^Tx=\theta_0+\theta_1x_1+...\theta_nx_n$

The parameters of your model are the $\theta$ values.
There are two kinds of functions to solve this problem:

Least Square method

θ = (X T X) - 1 X T y

$\theta=(X^TX)^{-1}X^Ty$

Batch Gradient Descent

Derivative of $J(\theta)$ on $\theta_j$ :

J ( θ ) θ j' = 1 m \sum i = 1 m (h θ (x (i)) - y (i)) x j = 1 m (h θ (x) - y) x j

$\frac{J(\theta)}{\theta_j}^{'}=\frac{1}{m}\sum_{i=1}^{m}(h_{\theta}(x^{(i)})-y^{(i)})x_j=\frac{1}{m}(h_\theta(x)-y)x_j$

#!/usr/bin/env python
# -*- coding: utf-8 -*-
################################
##Author: Vincent.Y
################################

import numpy as np
class LinearRegression(object):
    """
    solver: 
        ls for least square method, sgd for gridient descent
    """
    def __init__(self,solver="ls",lr=0.2,max_iter=200,bias=False):
        self.solver=solver
        self.coef_=None
        self.bias=bias
        if self.solver=='sgd':
            self.lr=lr
            self.max_iter=max_iter

    def gradient_descent(self,X,y):
        m=len(y)
        for i in xrange(0,self.max_iter):
            pred=X.dot(self.coef_)
            for j in xrange(0,X.shape[1]):
                tmp=X[:,j]
                errors = (pred - y) * tmp#element-wise multi
                self.coef_[j]=self.coef_[j] - self.lr * np.mean(errors)
        return self.coef_

    def fit(self,X,y):
        if self.bias:
            X = np.hstack([X,np.ones((X.shape[0],1))])

        if self.solver=="ls":
            self.coef_=np.linalg.lstsq(X,y)[0]
        else:
            self.coef_=np.zeros(X.shape[1])
            self.coef_=self.gradient_descent(X,y)

    def predict(self,X):
        if self.bias:
            X = np.hstack([X,np.ones((X.shape[0],1))])

        return X.dot(self.coef_)

if __name__=="__main__":
    x=np.array([1,2,3])
    x=x.reshape(-1,1)
    y=np.array([3,5,7])

    model=LinearRegression(bias=True)
    model.fit(x,y)
    print model.coef_
    print model.predict(x)

    model=LinearRegression(bias=True,solver='sgd',max_iter=1000)
    model.fit(x,y)
    print model.coef_
    print model.predict(x)