every blog every motto: There’s only one corner of the universe you can be sure of improving, and that’s your own self.
0. 前言
记录两种方式拟合非线性函数及其预测
神经网络对于非训练区间不能很好的进行预测,暂时未解决。
1. 正文
问题: 拟合函数 y = x 2 x^2 x2 - 2 ,训练数据输入在[-1,1]范围,测试区间在[1,3]
2.1 机器学习(多项式拟合)
简单来说就是求Y = Ax+b的参数,具体参考后面文献
假如我们有特征(a,b),PolynomialFeatures用于构造多项式特征,若其中参数degree=2,则,构造的特征为[1,a,b, a 2 a^2 a2,ab, b 2 b^2 b2]
eg1: 假如我们的特征为[1,2],则:
poly2 = PolynomialFeatures(degree=2)
a = np.array([[1, 2]])
print('shape: ', a.shape)
aa = poly2.fit_transform(a)
print('构造的特征为:', aa)
结果如下:
eg2: 假如我们的特征为[[1],[2]],则:
poly2 = PolynomialFeatures(degree=2)
a = np.array([[1],[2]])
print('shape: ', a.shape)
aa = poly2.fit_transform(a)
print('构造的特征为:', aa)
我们的多项式特征不在是[1,a,b, a 2 a^2 a2,ab, b 2 b^2 b2],而是[1, a 2 a^2 a2, b 2 b^2 b2],这是因为,我们只有一个特征,注意看shape,其为(2,1),最后一维的1即特征维度,2为样本个数,我们这里是一个特征,所以没有交叉特征。而eg1中有一个样本,两个特征。
结果如下:
import matplotlib.pyplot as plt
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
# 生成原始数据
x = np.linspace(-1, 1, 50)
y = x ** 2 - 2
test_x = np.linspace(1, 3, 50)
test_y = test_x ** 2 - 2
x = x.reshape(-1, 1)
y = y.reshape(-1, 1)
test_x = test_x.reshape(-1, 1)
test_y = test_y.reshape(-1, 1)
print(y.shape)
# print(x.shape)
# 构造特征 eg: (a,b) -> (1,a,b,a^2,ab,b^2)
poly2 = PolynomialFeatures(degree=2)
x_feature = poly2.fit_transform(x)
# 创建模型
line_model = LinearRegression()
# 训练,求得参数
line_model.fit(x_feature, y)
# 预测
y_pred = line_model.predict(poly2.transform(test_x))
# 可视化
plt.scatter(x, y) # 训练数据
plt.plot(test_x, test_y, label='y_test_true', c='r') # 测试数据
plt.plot(test_x, y_pred, label='y_test_predict', c='g') # 预测数据
plt.legend()
plt.show()
2.2 神经网络(部分问题未解决)
拟合比较好实现,但是在非训练集范围内进行预测比较麻烦,一直没有得到好的解决
我们的任务是在[-1,1]上训练,在[1,3]上进行预测,但是最终效果并不好。具体可以参考 文献20
说明:
- 最终代码没使用验证集,所以其中部分代码没用,但保证代码的完整性就不做处理了
- 两层网络(单元=10)就可以,也可以多点,但效果差不多
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
import tensorflow as tf
tf.keras.backend.set_floatx('float64')
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from tensorflow.keras.optimizers import SGD
def fun(x):
"""生成y值"""
return np.square(x) + np.random.normal(0, 0.02, x.shape)
def get_data():
# 生成原始数据
x = np.linspace(-1, 1, 400)
noise = np.random.normal(0, 0.02, x.shape)
# y = x ** 2 + noise
# y = np.square(x) + noise
y = fun(x)
xy = zip(x, y)
xy = list(xy)
# print(list(xy))
rat = (int(0.8 * len(x)))
# 训练集,采样80%
train_xy = xy[-rat:]
# 验证集,剩下的20%
valid_xy = xy[:-rat]
return xy
def get_x_y(xy):
"""拆分元组中的x,y。(x,y) -> x,y"""
x = []
y = []
for ele in xy:
x.append(ele[0])
y.append(ele[1])
x = np.array(x).reshape(-1, 1)
y = np.array(y).reshape(-1, 1)
return x, y
xy = get_data() # xy
x, y = get_x_y(xy) # 训练集
# # 训练、验证集
# train_x, train_y = get_x_y(train_xy)
# valid_x, valid_y = get_x_y(valid_xy)
# 测试集
test_x = np.linspace(1, 3, 30)
test_x = test_x.reshape(-1, 1)
test_y = fun(test_x)
test_y = test_y.reshape(-1, 1)
# 归一化
# scaler = StandardScaler()
# x = scaler.fit_transform(x)
# test_x = scaler.transform(test_x)
# print(train_x.shape)
# train_x = scaler.fit_transform(train_x)
# valid_x = scaler.transform(valid_x)
# ----------------------------------------------------------------
model = tf.keras.models.Sequential([
# tf.keras.layers.Dense(1, activation='sigmoid'),
tf.keras.layers.Dense(10, activation='relu', input_dim=1),
# tf.keras.layers.Dense(100, activation='relu'),
# tf.keras.layers.Dense(100, activation='relu'),
tf.keras.layers.Dense(1)
])
# 编译
model.compile(loss='mse', optimizer=SGD(0.3))
history = model.fit(x, y, epochs=300, verbose=2)
# 画图
def plot_learning_curves(history):
pd.DataFrame(history.history).plot(figsize=(8, 5))
plt.grid(True)
plt.gca().set_ylim(0, 0.3)
plt.show()
plot_learning_curves(history)
y_all = model.predict(x)
# y_train_pred = model.predict(train_x)
y_pred = model.predict(test_x)
# 可视化
plt.scatter(x, y) # 训练数据
plt.plot(x, y_all, label='y_train_all', c='pink') # 训练和验证数据
# plt.plot(train_x, y_train_pred, label='y_train_true', c='m', linestyle='--') # 训练数据
plt.plot(test_x, test_y, label='y_test_true', c='r') # 测试数据 真实值
plt.plot(test_x, y_pred, label='y_test_pred', c='g') # 预测数据
plt.legend()
plt.show()
损失:
训练和预测:
参考文献
[1] https://www.jianshu.com/p/5a5ac2ee8faa
[2] https://www.cnblogs.com/ai-ldj/p/14248228.html
[3] https://blog.csdn.net/OldDriver1995/article/details/105240965
[4] https://blog.csdn.net/tiange_xiao/article/details/79755793
[5] https://www.cnblogs.com/liweiwei1419/p/9715702.html
[7] https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html
[8] https://www.physicsforums.com/threads/fit_transform-vs-transform.933224/
[9] https://blog.csdn.net/funnyrand/article/details/110199821
[10] https://blog.csdn.net/qq_29537269/article/details/120997509
[11] https://blog.csdn.net/qq_44614026/article/details/94589671
[12] https://blog.csdn.net/great_yzl/article/details/120678800
[13] https://blog.csdn.net/qq_42496984/article/details/120609205
[14] https://blog.csdn.net/panguangyuu/article/details/87480051
[15] https://www.cnblogs.com/mengxiaoleng/p/11792802.html
[16] https://blog.csdn.net/qq_32023541/article/details/81015897
[17] https://blog.csdn.net/lny161224/article/details/120480402https://blog.csdn.net/qq_41249412/article/details/115709997
[18] https://blog.csdn.net/qq_41249412/article/details/115709997
[19] https://blog.csdn.net/qq_43631827/article/details/121563621