tensorflow 时间序列分析以及多个模型效果对比

tensorflow 时间序列分析以及多个模型效果对比

序言

在我们的生活中，有些东西是可以很好的预测的，比如气温， 降雨， 风速等，只要模具足够好，预测的效果也不错， 但有一些东西却不好预测， 如股票价格，因为股票受到太多因素的影响，变化很快， 就是鲁迅说的，地上本无路，走的人多了便成了路， 在股市中正好相反，本来有一条通往财富的路，但由于走的人多了，就把路压塌了。。。， 说白了，就是股市一赚二平七亏长期有效，拿一个模型去预测股市正好被别人收割。
言归正转， 下面还是说说靠普点的风速预测（从tensorflow官网抄的作业，增加了自已的理解）

一，导入相关包， 并下载数据

导入模块

import os
import datetime

import IPython
import IPython.display
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
import tensorflow as tf

mpl.rcParams['figure.figsize'] = (8, 6)
mpl.rcParams['axes.grid'] = False

下载数据

zip_path = tf.keras.utils.get_file(
    origin='https://storage.googleapis.com/tensorflow/tf-keras-datasets/jena_climate_2009_2016.csv.zip',
    fname='jena_climate_2009_2016.csv.zip',
    extract=True)
csv_path, _ = os.path.splitext(zip_path)

查看一下数据情况

df = pd.read_csv(csv_path)
df.head()

看一下数据的形状：

df.shape

结果输出：
(420551, 15)

二， 数据预处理

用切片的方式对数据每隔6个采一个样本

df = df[5::6]
df.head()

对时间进行格式化处理

date_time = pd.to_datetime(df.pop('Date Time'), format='%d.%m.%Y %H:%M:%S')
date_time

结果输出：
5 2009-01-01 01:00:00
11 2009-01-01 02:00:00
17 2009-01-01 03:00:00
23 2009-01-01 04:00:00
29 2009-01-01 05:00:00
…
420521 2016-12-31 19:10:00
420527 2016-12-31 20:10:00
420533 2016-12-31 21:10:00
420539 2016-12-31 22:10:00
420545 2016-12-31 23:10:00
Name: Date Time, Length: 70091, dtype: datetime64[ns]

再查看一下数据：

df.head()

取出其中的三列看一下：

plot_cols = ['T (degC)', 'p (mbar)','rho (g/m**3)']
plot_features = df[plot_cols]
plot_features.head()

将前面取出来的时间列加入到上面的表中：

plot_features.index = date_time
plot_features.index

DatetimeIndex([‘2009-01-01 01:00:00’, ‘2009-01-01 02:00:00’,
‘2009-01-01 03:00:00’, ‘2009-01-01 04:00:00’,
‘2009-01-01 05:00:00’, ‘2009-01-01 06:00:00’,
‘2009-01-01 07:00:00’, ‘2009-01-01 08:00:00’,
‘2009-01-01 09:00:00’, ‘2009-01-01 10:00:00’,
…
‘2016-12-31 14:10:00’, ‘2016-12-31 15:10:00’,
‘2016-12-31 16:10:00’, ‘2016-12-31 17:10:00’,
‘2016-12-31 18:10:00’, ‘2016-12-31 19:10:00’,
‘2016-12-31 20:10:00’, ‘2016-12-31 21:10:00’,
‘2016-12-31 22:10:00’, ‘2016-12-31 23:10:00’],
dtype=‘datetime64[ns]’, name=‘Date Time’, length=70091, freq=None)
1
plot_features.head()

plot_features.head()

打印一下图片：

_ = plot_features.plot(subplots=True)

取出前30行看，放大看一下：

plot_features = df[plot_cols][:480]
plot_features.index = date_time[:480]
_ = plot_features.plot(subplots=True)

用describe查看一下统计情况：

df.describe().transpose()

将表转置一下

df.describe().transpose()

对表中异常数据清洗一下（把-9999.0 改为0， 等同于df.where方法）：

wv = df['wv (m/s)']
bad_wv = (wv == -9999.0)
wv[bad_wv] = 0.0
max_wv = df['max. wv (m/s)']
bad_max_wv = max_wv == -9999.0
max_wv[bad_max_wv] = 0.0

将前面拿出来的这三列用plt.hist方法打印看一下：

plt.hist2d(df['wd (deg)'], df['wv (m/s)'], bins=(50,50), vmax=480)
plt.colorbar()
plt.xlabel('wind Directon [deg]')
plt.ylabel('Wind Velocity [m/s]')

用三角函数将风速分解成X轴， y轴对的分量：

wv = df.pop('wv (m/s)')
max_wv = df.pop('max. wv (m/s)')

# 将风的角度转为弧度
wd_rad = df.pop('wd (deg)')*np.pi/180

# 计算出对风速在对应弧度上，x与y轴的数值
df['Wx'] = wv*np.cos(wd_rad)
df['Wy'] = wv*np.sin(wd_rad)

# 计算出最大风力对应的x与y轴的数值
df['max Wx'] = max_wv*np.cos(wd_rad)
df['max Wy'] = max_wv*np.sin(wd_rad)
df.head()

再次查看一下图形：

plt.hist2d(df['Wx'], df['Wy'], bins=(50, 50), vmax=400)
plt.colorbar()
plt.xlabel('Wind X [m/s]')
plt.ylabel('Wind Y [m/s]')
ax = plt.gca()
ax.axis('tight')

将时间转化成时间戳：

timestamp_s = date_time.map(pd.Timestamp.timestamp)
timestamp_s

5 1.230772e+09
11 1.230775e+09
17 1.230779e+09
23 1.230782e+09
29 1.230786e+09
…
420521 1.483211e+09
420527 1.483215e+09
420533 1.483219e+09
420539 1.483222e+09
420545 1.483226e+09
Name: Date Time, Length: 70091, dtype: float64

下面增加几个变量（一天的秒数与一年的秒数）

day = 24*60*60 #一天的秒数
year = (365.2425)*day # 一年的秒数
day,  year

结果输出：
(86400, 31556952.0)

将时间通过三角函数的形式，加入到表中（类似于正规化）：

df['Day sin'] = np.sin(timestamp_s*(2*np.pi/day))
df['Day cos'] = np.cos(timestamp_s*(2*np.pi/day))
df['Year sin'] = np.sin(timestamp_s*(2*np.pi/year))
df['Year cos'] = np.cos(timestamp_s*(2*np.pi/year))

打印看一下刚才的几列数据：

plt.plot(np.array(df['Day sin'])[:25])
plt.plot(np.array(df['Day cos'])[:25])
plt.xlabel('Time [h]')
plt.title('Time of day signal')

将表中的温度这一列进行傅里叶变化(类似用多个正弦与余弦拟合原函数)

fft = tf.signal.rfft(df['T (degC)'])
f_per_dataset = np.arange(0, len(fft))
fft

结果输出：
<tf.Tensor: shape=(35046,), dtype=complex64, numpy=
array([ 6.6282938e+05-1.6778431e+00j, -4.6723877e+03+3.7432691e+04j,
-2.7167764e+03+1.2400171e+04j, …, -2.9608212e+01+2.2248232e+01j,
6.4756527e+00-4.9150139e+01j, 6.8415909e+01+2.2313152e+01j],
dtype=complex64)>

f_per_dataset

结果输出:
array([ 0, 1, 2, …, 35043, 35044, 35045])

建几个变量：
总样本数

n_samples_h = len(df['T (degC)'])
n_samples_h

结果输出：
70091
一年的小时数量：

hours_per_year = 24*365.2524
hours_per_year

结果输出：
8766.0576

每小时的采样的数量：

years_per_dataset = n_samples_h/(hours_per_year)
years_per_dataset

结果输出：
7.995726608047841

将上面的结果与序列数据相除（这个得到什么数据，不太懂了？）

f_per_year = f_per_dataset/years_per_dataset
f_per_year

array([0.00000000e+00, 1.25066807e-01, 2.50133615e-01, …,
4.38271613e+03, 4.38284120e+03, 4.38296627e+03])
看一下， 上面数据的阶梯图：

plt.step(f_per_year, np.abs(fft))
plt.xscale('log')
plt.ylim(0, 400000)
plt.xlim([0.1, max(plt.xlim())])
plt.xticks([1, 365.2425], labels=['1/year', '1/day'])
_ = plt.xlabel('Frequency (log scale)')

将列与下标做成字典的形式：

column_indices = {name: i  for i, name in enumerate(df.columns)}

n = len(df)
n, column_indices

(70091,
{‘p (mbar)’: 0,
‘T (degC)’: 1,
‘Tpot (K)’: 2,
‘Tdew (degC)’: 3,
‘rh (%)’: 4,
‘VPmax (mbar)’: 5,
‘VPact (mbar)’: 6,
‘VPdef (mbar)’: 7,
‘sh (g/kg)’: 8,
‘H2OC (mmol/mol)’: 9,
‘rho (g/m**3)’: 10,
‘Wx’: 11,
‘Wy’: 12,
‘max Wx’: 13,
‘max Wy’: 14,
‘Day sin’: 15,
‘Day cos’: 16,
‘Year sin’: 17,
‘Year cos’: 18})
将数据分成训练数据， 验证数据， 预测数据：

train_df = df[0:int(n*0.7)]
val_df = df[int(n*0.7):int(n*0.9)]
test_df = df[int(n*0.9):]
len(train_df), len(val_df), len(test_df), len(df)

结果输出：
(49063, 14018, 7010, 70091)

查看一下列数：

num_features = df.shape[1]
num_features

结果输出：19

数据正规化处理：

train_mean = train_df.mean()
train_std = train_df.std()

train_df = (train_df - train_mean)/train_std
val_df = (val_df - train_mean)/train_std
test_df = (test_df - train_mean)/train_std

用小提琴图查看一下数据:

df_std = (df-train_mean)/train_std
df_std = df_std.melt(var_name='Column', value_name='Normalized')
plt.figure(figsize=(12, 6))
ax = sns.violinplot(x='Column', y='Normalized', data=df_std)
_ = ax.set_xticklabels(df.keys(), rotation=90)

四，定义一个切片生成器

class WindowGenerator():
    def __init__(self, input_width, label_width, shift,
                train_df=train_df, val_df= val_df, test_df=test_df,
                label_columns=None):
        # Store the data
        self.train_df = train_df
        self.val_df = val_df
        self.test_df = test_df
        
        #Work out the label column 
        self.label_columns = label_columns
        if label_columns is not None:
            self.label_columns_indices = {
                name: i for i , name in enumerate(label_columns)
            }
        self.column_indices = {name: i for i, name in enumerate(train_df.columns)}
        
        # Work out the window parameters
        self.input_width = input_width
        self.label_width = label_width
        self.shift = shift
        
        self.total_window_size = input_width + shift
        
        self.input_slice = slice(0, input_width)
        self.input_indices = np.arange(self.total_window_size)[self.input_slice]
        
        self.label_start = self.total_window_size -self.label_width
        self.labels_slice = slice(self.label_start, None)
        self.label_indices = np.arange(self.total_window_size)[self.labels_slice]
        
    def __repr__(self):
        return '\n'.join([
            f'Total window size: {self.total_window_size}',
            f'Input indices: {self.input_indices}',
            f'Label indices: {self.label_indices}',
            f'Label column name(s): {self.label_columns}'
        ])

实例化看一下效果：

w1 = WindowGenerator(input_width=24, label_width=1, shift=24, label_columns=['T (degC)'])
w1

Total window size: 48
Input indices: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]
Label indices: [47]
Label column name(s): [‘T (degC)’]

再做一个实例化:

w2 = WindowGenerator(input_width=6, label_width=1, shift=1, label_columns=['T (degC)'])
w2

Total window size: 7
Input indices: [0 1 2 3 4 5]
Label indices: [6]
Label column name(s): [‘T (degC)’]

定义一个结构化数据的函数

def split_window(self, features):
    inputs = features[:, self.input_slice, :]
    labels = features[:, self.labels_slice, :]
    if self.label_columns is not None:
        labels = tf.stack(
            [labels[:, :, self.column_indices[name]] for name in self.label_columns],
            axis=-1)

    # Slicing doesn't preserve static shape information, so set the shapes
    # manually. This way the `tf.data.Datasets` are easier to inspect.
    inputs.set_shape([None, self.input_width, None])
    labels.set_shape([None, self.label_width, None])

    return inputs, labels

来实例化一个来看下：

example_window = tf.stack([np.array(train_df[:w2.total_window_size]),
                          np.array(train_df[100:100+w2.total_window_size]),
                          np.array(train_df[200:200+w2.total_window_size])])
example_window.shape                          

结果输出：
TensorShape([3, 7, 19])

继续增加属性：

example_inputs, example_labels = w2.split_window(example_window)
example_inputs.shape，example_labels.shape ， 

结果输出
TensorShape([3, 6, 19])， TensorShape([3, 1, 1])

看一下维度：

print('All shapes are:(batch, time, features)')
print(f'Window shape: {example_window.shape} ')
print(f'Inputs shape: {example_inputs.shape}')
print(f'Label, shape: {example_labels.shape}')

结果输出：
All shapes are:(batch, time, features)
Window shape: (3, 7, 19)
Inputs shape: (3, 6, 19)
Label, shape: (3, 1, 1)

做一个图形化显示的类：

def plot(self, model=None, plot_col='T (degC)', max_subplots=3):
    inputs, labels = self.example
    plt.figure(figsize=(12, 8))
    plot_col_index = self.column_indices[plot_col]
    max_n = min(max_subplots, len(inputs))
    for n in range(max_n):
        plt.subplot(max_n, 1, n+1)
        plt.ylabel(f'{plot_col}[normed]')
        plt.plot(self.input_indices, inputs[n, :, plot_col_index],
                label='Inputs', marker='.', zorder=-10)
        if self.label_columns:
            label_col_index = self.label_columns_indices.get(plot_col, None)
        else:
            label_col_index = plot_col_index
        
        if label_col_index is None:
            continue
        plt.scatter(self.label_indices, labels[n, :, label_col_index],
                   edgecolors='k', label='Labels', c='#2ca02c', s=64)
        if model is not None:
            predictions = model(inputs)
            plt.scatter(self.label_indices, predictions[n, :, label_col_index],
                       marker='X', edgecolors='k', label='Predictions',
                       c='#ff7f0e', s=64)
        if n==0:
            plt.legend()
    plt.xlabel('Time [h]')

WindowGenerator.plot = plot

        

w2.plot()

w2.plot(plot_col='p (mbar)')

将数据按split_window方法进行影射

def make_dataset(self, data):
    data = np.array(data, dtype=np.float32)
    ds = tf.keras.utils.timeseries_dataset_from_array(
        data=data,
        targets=None,
        sequence_length=self.total_window_size,
        sequence_stride=1,
        shuffle=True,
        batch_size=32,
    )
    
    ds = ds.map(self.split_window)
    return ds
WindowGenerator.make_dataset = make_dataset

增加几个生成训练据， 测试数据， 预测数据的方法

@property
def train(self):
    return self.make_dataset(self.train_df)
@property
def val(self):
    return  self.make_dataset(self.val_df)
@property
def test(self):
    return self.make_dataset(self.test_df)
@property
def example(self):
    result = getattr(self, '_example', None)
    if result is None:
        result = next(iter(self.train))
        self._exmaple = result
    return result
WindowGenerator.train = train
WindowGenerator.val = val
WindowGenerator.test = test
WindowGenerator.example = example

实例化一下来看看情况：

w2.train.element_spec

结果输出：

(TensorSpec(shape=(None, 6, 19), dtype=tf.float32, name=None),
TensorSpec(shape=(None, 1, 1), dtype=tf.float32, name=None))

用循环的方式来把数据中的参数列出来：

for example_inputs, example_labels in w2.train.take(1):
    print(f'Inputs shape (batch, time, features):{example_inputs.shape}')
    print(f'Labels shape (batch, time, features):{example_labels.shape}')

结果输出：
Inputs shape (batch, time, features):(32, 6, 19)
Labels shape (batch, time, features):(32, 1, 1)

单一维度数据：

single_step_window = WindowGenerator(
    input_width=1,
    label_width=1,
    shift=1,
    label_columns=['T (degC)']
)
single_step_window

结果输出：
Total window size: 2
Input indices: [0]
Label indices: [1]
Label column name(s): [‘T (degC)’]

多维度的：

for example_inputs, example_labels in w2.train.take(1):
    print(f'Inputs shape (batch, time, features):{example_inputs.shape}')
    print(f'Labels shape (batch, time, features):{example_labels.shape}')

生成一个线性模型：

class Baseline(tf.keras.Model):
    def __init__(self, label_index=None):
        super().__init__()
        self.label_index = label_index
        
    def call(self, inputs):
        if self.label_index is None:
            return inputs
        result = inputs[:, :, self.label_index]
        return result[:, :, tf.newaxis]

用线性模型来训练一下：

baseline = Baseline(label_index=column_indices['T (degC)'])

baseline.compile(loss=tf.keras.losses.MeanSquaredError(),
                metrics=[tf.keras.metrics.MeanAbsoluteError()])
val_performance = {}
performance = {}
val_performance['Baseline'] = baseline.evaluate(single_step_window.val)
performance['Baseline'] = baseline.evaluate(single_step_window.test, verbose=0)

结果输出：
439/439 [==============================] - 1s 3ms/step - loss: 0.0128 - mean_absolute_error: 0.0785

来外多维度的：

wide_window = WindowGenerator(
    input_width=24,
    label_width=24,
    shift=1,
    label_columns=['T (degC)']
)
wide_window

结果输出：
Total window size: 25
Input indices: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]
Label indices: [ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]
Label column name(s): [‘T (degC)’]

来看一下形状：

print('Input shape:', wide_window.example[0].shape)
print('Output shape:',baseline(wide_window.example[0]).shape)

结果输出：
Input shape: (32, 24, 19)
Output shape: (32, 24, 1)

linear = tf.keras.Sequential([
    tf.keras.layers.Dense(units=1)
])

linear

结果输出：
<keras.engine.sequential.Sequential at 0x29c3368ff40>

查看一下数据形状：

print('Input shape:', single_step_window.example[0].shape)
print('Output shape:', linear(single_step_window.example[0]).shape)

结果输出：
Input shape: (32, 1, 19)
Output shape: (32, 1, 1)

做一个训练用的函数：

MAX_EPOCHS = 20
def compile_and_fit(model, window, patience=2):
    early_stopping = tf.keras.callbacks.EarlyStopping(monitor='val_loss',
                                                     patience=patience,
                                                     mode='min')
    
    model.compile(loss=tf.keras.losses.MeanSquaredError(),
                 optimizer=tf.keras.optimizers.Adam(),
                 metrics=[tf.keras.metrics.MeanAbsoluteError()])
    
    history = model.fit(window.train, epochs=MAX_EPOCHS,
                       validation_data=window.val,
                       callbacks=[early_stopping])
    
    return history

用刚才的函数试用一下：

history = compile_and_fit(linear, single_step_window)

val_performance['Linear'] = linear.evaluate(single_step_window.val)
performance['Linear'] = linear.evaluate(single_step_window.test, verbose=0)

结果输出：
Epoch 1/20
1534/1534 [] - 6s 4ms/step - loss: 0.2958 - mean_absolute_error: 0.3149 - val_loss: 0.0152 - val_mean_absolute_error: 0.0887
Epoch 2/20
1534/1534 [] - 6s 4ms/step - loss: 0.0124 - mean_absolute_error: 0.0813 - val_loss: 0.0091 - val_mean_absolute_error: 0.0697
Epoch 3/20
1534/1534 [] - 6s 4ms/step - loss: 0.0092 - mean_absolute_error: 0.0702 - val_loss: 0.0087 - val_mean_absolute_error: 0.0686
Epoch 4/20
1534/1534 [] - 6s 4ms/step - loss: 0.0090 - mean_absolute_error: 0.0694 - val_loss: 0.0088 - val_mean_absolute_error: 0.0694
Epoch 5/20
1534/1534 [] - 6s 4ms/step - loss: 0.0090 - mean_absolute_error: 0.0696 - val_loss: 0.0086 - val_mean_absolute_error: 0.0686
Epoch 6/20
1534/1534 [] - 6s 4ms/step - loss: 0.0091 - mean_absolute_error: 0.0696 - val_loss: 0.0088 - val_mean_absolute_error: 0.0693
Epoch 7/20
1534/1534 [] - 6s 4ms/step - loss: 0.0090 - mean_absolute_error: 0.0696 - val_loss: 0.0087 - val_mean_absolute_error: 0.0690
439/439 [] - 1s 3ms/step - loss: 0.0087 - mean_absolute_error: 0.0690

查看一下形状：

print('Input shape:', wide_window.example[0].shape)
print('Output shape', baseline(wide_window.example[0]).shape)

结果输出：
Input shape: (32, 24, 19)
Output shape (32, 24, 1)

看一下图形：

wide_window.plot(linear)

柱状图显示一下：

plt.bar(x = range(len(train_df.columns)),
       height=linear.layers[0].kernel[:, 0].numpy())
axis = plt.gca()
axis.set_xticks(range(len(train_df.columns)))
_ = axis.set_xticklabels(train_df.columns, rotation=90)

做一个全链接层函数， 训练后并做预测：

dense = tf.keras.Sequential([
    tf.keras.layers.Dense(units=64, activation='relu'),
    tf.keras.layers.Dense(units=64, activation='relu'),
    tf.keras.layers.Dense(units=1)
])

history = compile_and_fit(dense, single_step_window)

val_performance['Dense'] = dense.evaluate(single_step_window.val)
performance['Dense'] = dense.evaluate(single_step_window.test, verbose=0)

Epoch 1/20
1534/1534 [] - 7s 4ms/step - loss: 0.0193 - mean_absolute_error: 0.0822 - val_loss: 0.0082 - val_mean_absolute_error: 0.0668
Epoch 2/20
1534/1534 [] - 6s 4ms/step - loss: 0.0079 - mean_absolute_error: 0.0645 - val_loss: 0.0072 - val_mean_absolute_error: 0.0606
Epoch 3/20
1534/1534 [] - 6s 4ms/step - loss: 0.0075 - mean_absolute_error: 0.0626 - val_loss: 0.0072 - val_mean_absolute_error: 0.0612
Epoch 4/20
1534/1534 [] - 6s 4ms/step - loss: 0.0072 - mean_absolute_error: 0.0606 - val_loss: 0.0070 - val_mean_absolute_error: 0.0596
Epoch 5/20
1534/1534 [] - 6s 4ms/step - loss: 0.0071 - mean_absolute_error: 0.0603 - val_loss: 0.0068 - val_mean_absolute_error: 0.0586
Epoch 6/20
1534/1534 [] - 6s 4ms/step - loss: 0.0069 - mean_absolute_error: 0.0593 - val_loss: 0.0070 - val_mean_absolute_error: 0.0614
Epoch 7/20
1534/1534 [] - 6s 4ms/step - loss: 0.0068 - mean_absolute_error: 0.0589 - val_loss: 0.0067 - val_mean_absolute_error: 0.0585
Epoch 8/20
1534/1534 [] - 7s 4ms/step - loss: 0.0068 - mean_absolute_error: 0.0586 - val_loss: 0.0065 - val_mean_absolute_error: 0.0566
Epoch 9/20
1534/1534 [] - 7s 4ms/step - loss: 0.0067 - mean_absolute_error: 0.0578 - val_loss: 0.0065 - val_mean_absolute_error: 0.0575
Epoch 10/20
1534/1534 [] - 7s 4ms/step - loss: 0.0066 - mean_absolute_error: 0.0574 - val_loss: 0.0068 - val_mean_absolute_error: 0.0588
439/439 [==============================] - 1s 3ms/step - loss: 0.0068 - mean_absolute_error: 0.0588

采用卷积的网络：

CONV_WIDTH = 3
conv_window = WindowGenerator(
    input_width=CONV_WIDTH,
    label_width=1,
    shift=1,
    label_columns=['T (degC)']
)
conv_window

图形化显示一下

conv_window.plot()

多层全链接:

multi_step_dense = tf.keras.Sequential([
    # shape: (time, features) => (time*features)
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(units=32, activation='relu'),
    tf.keras.layers.Dense(units=32, activation='relu'),
    tf.keras.layers.Dense(units=1),
    # Add back the time dimension.
    # Shape: (outputs) => (1,outputs)
    tf.keras.layers.Reshape([1, -1]),
])

看一下数据形状：

print('Input shape:', conv_window.example[0].shape)
print('Output shape:', multi_step_dense(conv_window.example[0]).shape)

结果输出：
Input shape: (32, 3, 19)
Output shape: (32, 1, 1)

训练一下：

history = compile_and_fit(multi_step_dense, conv_window)

IPython.display.clear_output()
val_performance['Multi step dense'] = multi_step_dense.evaluate(conv_window.val)
performance['Multi step dense'] = multi_step_dense.evaluate(conv_window.test, verbose=0)

显示一下结果：

conv_window.plot(multi_step_dense)

查看一下形状：

print('Input shape:', wide_window.example[0].shape)
try:
    print('Output shape:', multi_step_dense(wide_window.example[0]).shape)
except Exception as e:
    print(f'\n{type(e).__name__}:{e}')

结果出错：
Input shape: (32, 24, 19)

ValueError:Exception encountered when calling layer “sequential_27” (type Sequential).

Input 0 of layer “dense_49” is incompatible with the layer: expected axis -1 of input shape to have value 57, but received input with shape (32, 456)

Call arguments received by layer “sequential_27” (type Sequential):
• inputs=tf.Tensor(shape=(32, 24, 19), dtype=float32)
• training=None
• mask=None

做一个卷积模型：

conv_model = tf.keras.Sequential([
    tf.keras.layers.Conv1D(filters=32,
                           kernel_size=(CONV_WIDTH,),
                           activation='relu'),
    tf.keras.layers.Dense(units=32, activation='relu'),
    tf.keras.layers.Dense(units=1),
])

查看一下形状：

print('Conv model on "conv_window"')
print('Input shape:', conv_window.example[0].shape)
print('Output shape:', conv_model(conv_window.example[0]).shape)

结果输出：
Conv model on “conv_window”
Input shape: (32, 3, 19)
Output shape: (32, 1, 1)

训练并预测：

history = compile_and_fit(conv_model, conv_window)

IPython.display.clear_output()
val_performance['Conv'] = conv_model.evaluate(conv_window.test, verbose=0)
performance['Conv'] = conv_model.evaluate(conv_window.test, verbose=0)

查看一下形状：

print('Wide window')
print('Input shape:', wide_window.example[0].shape)
print('Labels shape:', wide_window.example[1].shape)
print('Output shape:', conv_model(wide_window.example[0]).shape)

结果输出：
Wide window
Input shape: (32, 24, 19)
Labels shape: (32, 24, 1)
Output shape: (32, 22, 1)

切片函数：

LABEL_WIDTH = 24
INPUT_WIDTH = LABEL_WIDTH + (CONV_WIDTH - 1)
wide_conv_window = WindowGenerator(
    input_width=INPUT_WIDTH,
    label_width=LABEL_WIDTH,
    shift=1,
    label_columns=['T (degC)']
)
wide_conv_window

查看一下形状：

print('Wide conv window')
print('Input shape:', wide_conv_window.example[0].shape)
print('Labels shape:', wide_conv_window.example[1].shape)
print('Output shape:', conv_model(wide_conv_window.example[0]).shape)

看一下图形化结果：

wide_conv_window.plot(conv_model)

做一个LSTM 模型：

lstm_model = tf.keras.models.Sequential([
    # Shape [batch, time, fetures] => [batch, time, lstm_units]
    tf.keras.layers.LSTM(32, return_sequences=True),
    # Shape => [batch, time, features]
    tf.keras.layers.Dense(units=1)
    
])

看一下数据形状:

print('Input shape:', wide_window.example[0].shape)
print('Output shape:', lstm_model(wide_window.example[0]).shape)

结果输出：
Input shape: (32, 24, 19)
Output shape: (32, 24, 1)

训练并预测：

historys = compile_and_fit(lstm_model, wide_window)

IPython.display.clear_output()
val_performance['LSTM'] = lstm_model.evaluate(wide_window.val)
performance['LSTM'] = lstm_model.evaluate(wide_window.test, verbose=0)

图形化显示：

wide_window.plot(lstm_model)

用柱状图看一下各模型的效果：

x = np.arange(len(performance))
width = 0.3
metric_name = 'mean_absolute_error'
metric_index = lstm_model.metrics_names.index('mean_absolute_error')
val_mae = [v[metric_index] for v in val_performance.values()]
test_mae = [v[metric_index] for v  in performance.values()]

plt.ylabel('mean_absolute_error [T (degC), normalized]')
plt.bar(x - 0.17, val_mae, width, label='validation')
plt.bar(x + 0.17, test_mae, width, label='Test')
plt.xticks(ticks=x, labels=performance.keys(),
          rotation=45)`for name, value in performance.items():
    print(f'{name:12s}:{value[1]:0.4f}')`
_ = plt.legend()

看一下具体结果：

for name, value in performance.items():
    print(f'{name:12s}:{value[1]:0.4f}')

Baseline :0.0852
Linear :0.0671
Dense :0.0602
Multi step dense:0.0683
Conv :0.0638
LSTM :0.0520

从上面看出， LSTM的效果最好

五， 多标签输出

更新一下切片函数：

single_step_window = WindowGenerator(
    # 'WindowGenerator' returns all features as labels if you
    # don't set the 'label_colums' argument
    input_width=1, 
    label_width=1,
    shift=1
)

wide_window = WindowGenerator(
    input_width=24,
    label_width=24,
    shift=1
)

for example_inputs, example_labels in wide_window.train.take(1):
    print(f'Inputs shape (batch, time, features):{example_inputs.shape}')
    print(f'Labels shape (batch, time, feature):{example_labels.shape}')
    

结果输出：
Inputs shape (batch, time, features):(32, 24, 19)
Labels shape (batch, time, feature):(32, 24, 19)

实例化一个baseline模型：

baseline.compile(loss=tf.keras.losses.MeanSquaredError(),
                metrics=[tf.keras.metrics.MeanAbsoluteError()])

清空一下效果字典：

val_performance = {}
performance = {}

做一下测试：

val_performance['Baseline'] = baseline.evaluate(wide_window.val)
performance['Baseline'] = baseline.evaluate(wide_window.test, verbose=0)

结果输出：

438/438 [==============================] - 1s 3ms/step - loss: 0.0886 - mean_absolute_error: 0.1589
更新一下函数：

val_performance['Baseline'] = baseline.evaluate(wide_window.val)
performance['Baseline'] = baseline.evaluate(wide_window.test, verbose=0)dense = tf.keras.Sequential([
    tf.keras.layers.Dense(units=64, activation='relu'),
    tf.keras.layers.Dense(units=64, activation='relu'),
    tf.keras.layers.Dense(units=num_features)
])

训练并预测：

history = compile_and_fit(dense, single_step_window)

IPython.display.clear_output()
val_performance['Dense'] = dense.evaluate(single_step_window.val)
performance['Dense'] = dense.evaluate(single_step_window.test, verbose=0)

结果输出：
439/439 [==============================] - 1s 3ms/step - loss: 0.0691 - mean_absolute_error: 0.1348

看一下lstm的预测效果：

%%time
wide_window = WindowGenerator(
input_width=24, label_width=24, shift=1)

lstm_model = tf.keras.models.Sequential([
    # Shape [batch, time , features] => [batch, time, lstm_units]
    tf.keras.layers.LSTM(32, return_sequences=True),
    # Shape => [batch, time , features]
    tf.keras.layers.Dense(units=num_features)
])

history = compile_and_fit(lstm_model, wide_window)

IPython.display.clear_output()
val_performance['LSTM'] = lstm_model.evaluate(wide_window.val)
performance['LSTM'] = lstm_model.evaluate(wide_window.test, verbose=0)

结果输出：
438/438 [==============================] - 3s 6ms/step - loss: 0.0615 - mean_absolute_error: 0.1209
CPU times: total: 12min 49s
Wall time: 4min 10s

定义一个wrapper 函数：

class ResidualWrapper(tf.keras.Model):
    def __init__(self, model):
        super().__init__()
        self.model = model
        
    def call(self, inputs, *args, **kwargs):
        delta = self.model(inputs, *args, **kwargs)
        
        # The prediction for each time step is the input
        # from the previous time step plus the delta
        # calculated by the model
        return inputs + delta

看一下优化后的LSTM模型：

%%time 
residual_lstm = ResidualWrapper(
    tf.keras.Sequential([
        tf.keras.layers.LSTM(32, return_sequences=True),
        tf.keras.layers.Dense(
            num_features,
            # The predicted delta should stard small.
            # Therefore, initialize the output layer with zeros.
            kernel_initializer=tf.initializers.zeros()
        )
    ])
)

history = compile_and_fit(residual_lstm, wide_window)

IPython.display.clear_output()
val_performance['Residual LSTM'] = residual_lstm.evaluate(wide_window.val)
performance['Residual LSTM'] = residual_lstm.evaluate(wide_window.test, verbose=0)

结果输出：
38/438 [==============================] - 3s 6ms/step - loss: 0.0620 - mean_absolute_error: 0.1180
CPU times: total: 7min 9s
Wall time: 2min 19s

看一下几个模型效果图：

x = np.arange(len(performance))
width = 0.3

metric_name = 'mean_absolute_error'
metric_index = lstm_model.metrics_names.index('mean_absolute_error')
val_mae = [v[metric_index] for v in val_performance.values()]
test_mae = [v[metric_index] for v in performance.values()]

plt.bar(x - 0.17, val_mae, width, label='Validation')
plt.bar(x + 0.17, test_mae, width, label='Test')
plt.xticks(ticks=x, labels=performance.keys(),
          rotation=45)
plt.ylabel('MAE (average over all outputs)')
_ = plt.legend()

从图上可以看， 最后种模型效果最好

六，多层阶模型：

更新一下能数：

OUT_STEPS = 24
multi_window = WindowGenerator(input_width=24,
                              label_width=OUT_STEPS,
                              shift=OUT_STEPS)
multi_window.plot()
multi_window

结果输出：
Total window size: 48
Input indices: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]
Label indices: [24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47]
Label column name(s): None

用线性模型测试一下：

class MultiStepLastBaseline(tf.keras.Model):
    def call(self, inputs):
        return tf.tile(inputs[:, -1:, :], [1, OUT_STEPS, 1])
last_baseline = MultiStepLastBaseline()
last_baseline.compile(loss=tf.keras.losses.MeanSquaredError(),
                     metrics=[tf.keras.metrics.MeanAbsoluteError()])

multi_val_performance = {}
multi_performance = {}

multi_val_performance['Last'] = last_baseline.evaluate(multi_window.val)
multi_performance['Last'] =last_baseline.evaluate(multi_window.test, verbose=0)
multi_window.plot(last_baseline)

结果输出：
437/437 [==============================] - 1s 3ms/step - loss: 0.6285 - mean_absolute_error: 0.5007

优化一下线性模型 ：

class RepeatBaseline(tf.keras.Model):
    def call(self, inputs):
        return inputs
repeat_baseline = RepeatBaseline()
repeat_baseline.compile(loss=tf.keras.losses.MeanSquaredError(),
                       metrics=[tf.keras.metrics.MeanAbsoluteError()])

multi_val_performance['Repeat'] = repeat_baseline.evaluate(multi_window.val)
multi_performance['Repeat'] = repeat_baseline.evaluate(multi_window.test, verbose=0)
multi_window.plot(repeat_baseline)

结果输出：
437/437 [==============================] - 1s 2ms/step - loss: 0.4270 - mean_absolute_error: 0.3959

另一种线性模型：

multi_linear_model = tf.keras.Sequential([
    # Take the last time-step
    # Shape [batch, time , features] => [batch, 1, features]
    tf.keras.layers.Lambda(lambda x: x[:, -1, :]),
    # Shape => [batch, out_steps*features]
    tf.keras.layers.Dense(OUT_STEPS*num_features,
                         kernel_initializer=tf.initializers.zeros()),
    # Shape => [batch, out_steps, features]
    tf.keras.layers.Reshape([OUT_STEPS, num_features])
])

historyt = compile_and_fit(multi_linear_model, multi_window)

IPython.display.clear_output()
multi_val_performance['Linear'] = multi_linear_model.evaluate(multi_window.val)
multi_performance['Linear'] = multi_linear_model.evaluate(multi_window.test, verbose=0)
multi_window.plot(multi_linear_model)

全链接层模型 ：

multi_dense_model = tf.keras.Sequential([
    # Take the last time step.
    # Shape [batch, time , features] => [batch, 1, features]
    tf.keras.layers.Lambda(lambda x: x[:, -1:, :]),
    # Shape => [batch, 1, dense_units]
    tf.keras.layers.Dense(512, activation='relu'),
    # Shape => [batch, out_steps*features]
    tf.keras.layers.Dense(OUT_STEPS*num_features,
                         kernel_initializer=tf.initializers.zeros()),
    # Shape => [batch, out_steps, feratures]
    tf.keras.layers.Reshape([OUT_STEPS, num_features])
])

history = compile_and_fit(multi_dense_model, multi_window)

IPython.display.clear_output()
multi_val_performance['Dense'] = multi_dense_model.evaluate(multi_window.val)
multi_performance['Dense'] = multi_dense_model.evaluate(multi_window.test, verbose=0)
multi_window.plot(multi_dense_model)

结果输出：

用一下卷积模型：

CONV_WIDTH = 3
multi_conv_model = tf.keras.Sequential([
    # Shape [batch, CONV_WIDTH, features]
    tf.keras.layers.Lambda(lambda x: x[:, -CONV_WIDTH:, :]),
    # Shape => [batch, 1, conv_units]
    tf.keras.layers.Conv1D(256, activation='relu', kernel_size=(CONV_WIDTH)),
    # Shape => [batch, 1, out_steps*num_feature]
    tf.keras.layers.Dense(OUT_STEPS*num_features,
                         kernel_initializer=tf.initializers.zeros()),
    # Shape => [batch, out_steps, features]
    tf.keras.layers.Reshape([OUT_STEPS, num_features])
])

history = compile_and_fit(multi_conv_model, multi_window)

IPython.display.clear_output()

multi_val_performance['Conv'] = multi_conv_model.evaluate(multi_window.val)
multi_performance['Conv'] = multi_conv_model.evaluate(multi_window.test, verbose=0)
multi_window.plot(multi_conv_model)

结果输出：

lstm模型：

multi_lstm_model = tf.keras.Sequential([
    # Shape [batch, time, features] => [batch, lstm_units].
    # Add more 'lstm_units' just overfits more quickly.
    tf.keras.layers.LSTM(32, return_sequences=False),
    # Shape => [batch, out_steps*num_features]
    tf.keras.layers.Dense(OUT_STEPS*num_features,
                         kernel_initializer=tf.initializers.zeros()),
    # Shape => [batch, out_steps, features].
    tf.keras.layers.Reshape([OUT_STEPS, num_features])
])

history = compile_and_fit(multi_lstm_model, multi_window)

IPython.display.clear_output()

multi_val_performance['LSTM'] = multi_lstm_model.evaluate(multi_window.val)
multi_performance['LSTM'] = multi_lstm_model.evaluate(multi_window.test, verbose=0)

结果输出:
437/437 [==============================] - 3s 6ms/step - loss: 0.2140 - mean_absolute_error: 0.2839

定义一个回馈函数：

class FeedBack(tf.keras.Model):
    def __init__(self, units, out_steps):
        super().__init__()
        self.out_steps = out_steps
        self.units = units
        self.lstm_cell = tf.keras.layers.LSTMCell(units)
        # Also wrap the LSTMCell in a RNN to simplify the 'warmup' methon.
        self.lstm_rnn = tf.keras.layers.RNN(self.lstm_cell, return_state=True)
        self.dense = tf.keras.layers.Dense(num_features)
        

实例化一下：

feedback_model = FeedBack(units=32, out_steps=OUT_STEPS)

定义一个打包函数：

def warmup(self, inputs):
    # inputs.shape => (batch, time, features)
    # X.shape => (batch, lstm_units)
    x, *state = self.lstm_rnn(inputs)
    
    # predictions.shape => (batch, features)
    prediction = self.dense(x)
    return prediction, state
FeedBack.warmup = warmup

看一下数据形状：

prediction, state = feedback_model.warmup(multi_window.example[0])
prediction.shape

结果输出：
TensorShape([32, 19])

定义一个回调函数：

def call(self, inputs, training=None):
    # Use a TensorArray to  capture dynamically unrolled outputs.
    predictions=[]
    # Initialize the LSTM state
    prediction, state = self.warmup(inputs)
    
    # Insert the first prediction steps.
    predictions.append(prediction)
    
    # Run the rest of the prediction steps
    for n in range(1, self.out_steps):
        # Use the last prediction as input.
        x = prediction
        # Execute one lstm step.
        x, state = self.lstm_cell(x, states=state,
                                 training=training)
        # Convert the lstm output to a prediction.
        prediction = self.dense(x)
        # Add the prediction to the output.
        predictions.append(prediction)
        
    # predictions.shape => (time, batch, features)
    predictions = tf.stack(predictions)
    # predictions.shape => (batch, time, features)
    predictions = tf.transpose(predictions, [1, 0, 2])
    return predictions

FeedBack.call = call

看一下形状：

print('Output shape (batch, time, features):', feedback_model(multi_window.example[0]).shape)

验证并测试一下：

history = compile_and_fit(feedback_model, multi_window)

IPython.display.clear_output()

multi_val_performance['AR LSTM'] = feedback_model.evaluate(multi_window.val)
multi_performance['AR LSTM'] = feedback_model.evaluate(multi_window.test, verbose=0)
multi_window.plot(feedback_model)

用柱状图看一下各模型 的表现效果：

x = np.arange(len(multi_performance))
width = 0.3

metric_name = 'mean_absolute_error'
metric_index = lstm_model.metrics_names.index('mean_absolute_error')
val_mae = [v[metric_index] for v in multi_val_performance.values()]
test_mae = [v[metric_index] for v in multi_performance.values()]

plt.bar(x - 0.17, val_mae, width, label='Validation')
plt.bar(x + 0.17, test_mae, width, label='Test')
plt.xticks(ticks=x, labels=multi_performance.keys(),
          rotation=45)
plt.ylabel(f'MAE (average over all times and outputs)')
_ = plt.legend()

看一下具体的数据：

for name, value in multi_performance.items():
    print(f'{name:8s}: {value[1]:0.4f}')

结果输出：
Last : 0.5157
Repeat : 0.3774
Linear : 0.2983
Dense : 0.2762
Conv : 0.2745
LSTM : 0.2759
AR LSTM : 0.2846

从上面可以看出， 卷积模型与LSTM的效果最好

本案例从以下地方搬运：https://tensorflow.google.cn/tutorials/structured_data/time_series?hl=zh-cn