单特征线性回归代码实现

1.数据预处理

将自己准备好的数据进行预处理，对数据的分布进行标准化，然后为了方便矩阵计算，为偏置项添加一列1

def prepare_for_training(data, polynomial_degree=0, sinusoid_degree=0, normalize_data=True):

    # 计算样本总数
    num_examples = data.shape[0]

    data_processed = np.copy(data)

    # 预处理
    features_mean = 0
    features_deviation = 0
    data_normalized = data_processed
    if normalize_data:
        (
            data_normalized,
            features_mean,
            features_deviation
        ) = normalize(data_processed)

        data_processed = data_normalized

    # 特征变换sinusoidal
    if sinusoid_degree > 0:
        sinusoids = generate_sinusoids(data_normalized, sinusoid_degree)
        data_processed = np.concatenate((data_processed, sinusoids), axis=1)

    # 特征变换polynomial
    if polynomial_degree > 0:
        polynomials = generate_polynomials(data_normalized, polynomial_degree, normalize_data)
        data_processed = np.concatenate((data_processed, polynomials), axis=1)

    # 加一列1
    data_processed = np.hstack((np.ones((num_examples, 1)), data_processed))

    return data_processed, features_mean, features_deviation

标准化方法,减去均值，在除标准差

def normalize(features):

    features_normalized = np.copy(features).astype(float)

    # 计算均值
    features_mean = np.mean(features, 0)

    # 计算标准差
    features_deviation = np.std(features, 0)

    # 标准化操作
    if features.shape[0] > 1:
        features_normalized -= features_mean

    # 防止除以0
    features_deviation[features_deviation == 0] = 1
    features_normalized /= features_deviation

    return features_normalized, features_mean, features_deviation

然后在LnearRegression类中调用数据预处理方法，得到处理后的数据

class LinearRegression:
    """
    1,对数据进行预处理操作
    2，先得到特征个数
    3，初始化参数矩阵
    """
    def __init__(self,data,labels,polynomial_degree=0,sinusoid_degree=0, normalize_data=True):
        (data_processed,
         features_mean,
         features_deviation)=prepare_for_training(data,polynomial_degree=0,sinusoid_degree=0, normalize_data=True) #数据预处理
        self.data = data_processed
        self.labels = labels
        self.features_mean = features_mean
        self.features_deviation = features_deviation
        self.polynomial_degree = polynomial_degree
        self.sinusoid_degree = sinusoid_degree
        self.normalize_data = normalize_data

        num_features = self.data.shape[1]
        self.theta = np.zeros((num_features,1))#初始化参数

2.线性回归实现

2.1 参数更新

得到样本量，预测值与真实值的差delta，就可以将线性回归的公式用代码写出来，完成参数的更新。预测值使用当前的theta求出

 def gradient_step(self,alpha):
        '''

        梯度下降参数更新的计算方法，是矩阵的计算
        '''
        num_examples = self.data.shape[0]
        prediction = LinearRegression.hypothesis(self.data,self.theta)
        delta = prediction - self.labels
        theta = self.theta
        theta = theta - alpha*(1/num_examples)*(np.dot(delta.T,self.data)).T
        self.theta = theta
 
  @staticmethod
  def hypothesis(data,theta):
        '''
        求出预测值
        '''
        predictions = np.dot(data,theta)
        return predictions

2.2 梯度下降

根据预设的迭代次数和步长，对上一步参数更新进行迭代，用cost_history来记录每一次迭代的损失值

    def gradient_descent(self,alpha,num_iterations):
        '''

        梯度下降的迭代，共迭代num_iterations次
        '''
        cost_history = []
        for _ in range(num_iterations):
            self.gradient_step(alpha)
            cost_history.append(self.cost_function(self.data,self.labels))
        return cost_history

损失函数

       def cost_function(self,data,labels):
        '''
        损失值
        '''
        num_examples = self.data.shape[0]
        delta = LinearRegression.hypothesis(self.data,self.theta) - labels
        cost = (1/2)*np.dot(delta.T,delta)/num_examples
        return cost[0][0]

到这里训练部分基本上就完成了

2.3 预测模块

同样要对需要预测的数据进行预处理，再用训练好的参数进行预测，使用get_cost方法得到预测结果的损失值

    def get_cost(self,data,labels):
        data_processed = prepare_for_training(data,self.polynomial_degree,self.sinusoid_degree, self.normalize_data)[0]
        return self.cost_function(data_processed,labels)
    
    def predict(self,data):
        '''
        利用训练好的参数进行预测
        :return:
        '''
        data_processed = prepare_for_training(data, self.polynomial_degree, self.sinusoid_degree, self.normalize_data)[0]
        predictions = LinearRegression.hypothesis(data_processed,self.theta)
        return predictions

方法已经写好了，下面开始调用

3. 开始训练

3.1 读取数据

将数据读取进来，并且设置好训练集和测试集。特征为GDP，要预测幸福指数。画一个散点图将数据显示出来。

data = pd.read_csv('./data/world-happiness-report-2017.csv')
train_data = data.sample(frac=0.8)
test_data = data.drop(train_data.index)

input_param_name = 'Economy..GDP.per.Capita.'
output_param_name = 'Happiness.Score'

x_train = train_data[[input_param_name]].values
y_train = train_data[[output_param_name]].values

x_test = test_data[input_param_name].values
y_test = test_data[output_param_name].values

plt.scatter(x_train,y_train,label='train data')
plt.scatter(x_test,y_test,label='test data')
plt.xlabel(input_param_name)
plt.ylabel(output_param_name)
plt.legend()
plt.show()

3.2 设置参数，开始训练

设置迭代次数为500，学习率（步长）为0.01。开始训练，并把开始的损失值和最后训练完成的损失值打印出来。画一个折线图展示梯度下降

num_iterations = 500
learning_rate = 0.01

linear_regression = LinearRegression(x_train,y_train)
(theta,cos_thistory) = linear_regression.train(learning_rate,num_iterations)

print('开始的损失值：',cos_thistory[0])
print('最后的损失值：',cos_thistory[-1])

plt.plot(range(num_iterations),cos_thistory)
plt.xlabel('iter')
plt.ylabel('loss')
plt.show()

3.3 预测并画出回归方程

创建100个线性数据，最小值和最大值为训练集中的最小值和最大值，然后输入到模型中，画出结果

num_predictions=100
x_prediction=np.linspace(x_train.min(),x_train.max(),num_predictions).reshape(num_predictions,1)
y_prediction=linear_regression.predict(x_prediction)

plt.scatter(x_train,y_train,label='train data')
plt.scatter(x_test,y_test,label='test data')
plt.plot(x_prediction,y_prediction)
plt.xlabel(input_param_name)
plt.ylabel(output_param_name)
plt.legend()
plt.show()