阅读前思考
在你的应用场景中，哪些特征适合放在Wide侧，哪些特征适合放在Deep侧，为什么呢？
为什么Wide部分要用L1 FTRL训练？
为什么Deep部分不特别考虑稀疏性的问题？
系列导读
- 深度学习推荐系统之deepcrossing简单介绍与代码实现
- 深度学习推荐系统之wide & deep介绍和代码实现
- 深度学习推荐系统之deepFM介绍和代码实现
- NFM（2017）结构与原理简介（代码）
介绍
Wide&Deep推荐算法出自一篇论文《Wide&Deep Learning for RecommenderSystems》，Wide&Deep由两部分组成，分别是Wide和Deep。Wide&Deep 模型同时具备较强的 “记忆能力” 和“泛化能力”，模型能够从历史数据中学习到高频共现的特征组合的能力，称为是模型的Memorization。能够利用特征之间的传递性去探索历史数据中从未出现过的特征组合，称为是模型的Generalization。Wide&Deep兼顾Memorization与Generalization并在Google Play store的场景中成功落地。
Wide & Deep原理
首先我们来看下业内的常用的模型的结构图：
这张图源于论文，从左到右分别展示了Wide模型，Wide & Deep模型以及Deep模型。从图上我们也看得出来所谓的Wide模型呢其实就是线性模型，Deep模型是深度神经网络模型。下面结合这张图对这两个部分做一个详细一点的介绍。
Wide部分
Wide部分就是一个广义的线性模型，表示为y=WX+b。X特征部分包括基础特征和交叉特征(cross-product transformation)。交叉特征在wide部分很重要，可以捕捉到特征间的交互，起到添加非线性的作用。
交叉特征可表示为：$$\phi{k}(x)=\prod{i=1}^d x_i^{c{ki}}, c{ki}\in {0,1}$$
$c{ki}是一个布尔变量，当第个特征属于第个特征组合时，是一个布尔变量，当第i个特征属于第k个特征组合时，c{ki}的值为，否则为，的值为1，否则为0，x_i$是第i个特征的值，大体意思就是两个特征都同时为1这个新的特征才能为1，否则就是0，说白了就是一个特征组合。用原论文的例子举例：
AND(user_installed_app=QQ, impression_app=WeChat)，当特征user_installed_app=QQ,和特征impression_app=WeChat取值都为1的时候，组合特征AND(user_installed_app=QQ, impression_app=WeChat)的取值才为1，否则为0。
对于wide部分训练时候使用的优化器是带L_1正则的FTRL算法(Follow-the-regularized-leader)，而L1 FTLR是非常注重模型稀疏性质的，也就是说W&D模型采用L1 FTRL是想让Wide部分变得更加的稀疏，即Wide部分的大部分参数都为0，这就大大压缩了模型权重及特征向量的维度。Wide部分模型训练完之后留下来的特征都是非常重要的，那么模型的“记忆能力”就可以理解为发现"直接的"，“暴力的”，“显然的”关联规则的能力。例如Google W&D期望wide部分发现这样的规则：用户安装了应用A，此时曝光应用B，用户安装应用B的概率大。
优点： 模型结构简单，具有较强的“记忆能力” （memorization） ，可直接学习并利用历史数据中物品或者特征的“共现频率”的能力，原始数据往往可以直接影响推荐结果。
缺点：依赖人工特征工程,需要人工经验，这种推荐只对用户操作过的商品有效。
Deep部分
Deep部分就是个前馈网络模型(深度神经网络模型)。通过深度学习学习出来的一些向量，这些向量是隐性特征，往往是没有明确可解释性的。这些向量也可以作为特征的一部分参与到训练中。这部分特征是深度学习框架自动生成的，无需人力干预。 特征首先转换为低维稠密向量，维度通常O（10）-O（100）。向量随机初始化，经过最小化随时函数训练模型。激活函数采用Relu。前馈部分表示如下： 公式 $$a^{(l+1)} = f(W^{l}a^{(l)} + b^{l})$$
优点： 模型具有 “泛化能力” generalization ，找到不易于发现的规律
缺点：不易于解释，计算量大
Wide&Deep联合训练
在联合模型中，Wide和Deep部分的输出通过加权方式合并到一起，并通过logistic loss function进行最终输出。
联合训练和模型集成要进行区分，他们有着以下两点区别：
训练方式。 集成模型的子模型部分是独立训练，只在inference阶段合并预测。而联合训练模型是同时训练同时产出的。 模型规模。集成模型独立训练，模型规模要大一些才能达到可接受的效果。而联合训练模型中，Wide部分只需补充Deep模型的缺点，即记忆能力，这部分主要通过小规模的交叉特征实现。因此联合训练模型的Wide部分的模型特征较小。 联合模型求解采用FTRL算法，L1正则。深度部分用AdaGrad优化算法。
优点：同时学习低阶和高阶组合特征，它混合了一个线性模型（Wide part）和Deep模型(Deep part)，能够快速处理并记忆大量历史行为特征，并 且具有强大的表达能力。
缺点：这两部分模型需要不同的输入，而Wide part部分的输入，依旧依赖人工特征工程。
代码实现：
完整数据集:crite
# coding=utf-8
# Author:Jo Choi
# Date:2021-03-18
# Email:cai_oo@sina.com.cn
# Blog: *
'''
数据集：criteo_sample
------------------------------
运行结果：
----------------------------
loss: 0.4625 - binary_crossentropy: 0.4625 - auc: 0.822 - 0s 36ms/step - loss: 0.4164 -
binary_crossentropy: 0.4164 - auc: 0.8191 - val_loss: 0.6470 -
val_binary_crossentropy: 0.6470 - val_auc: 0.6595
----------------------------
'''
import itertools
import pandas as pd
import numpy as np
from tqdm import tqdm
from collections import namedtuple
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import *
from tensorflow.keras.models import *
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler,LabelEncoder
from utils import SparseFeat, DenseFeat, VarLenSparseFeat
def data_process(data_df,dense_features,sparse_features):
"""
数据预处理，包括填充缺失值，数值处理，类别编码
:param data_df: Data_Frame格式的数据
:param dense_features: 数值特征名称列表
:param sparse_features: 离散特征名称列表
"""
#数值型特征缺失值填充0.0
data_df[dense_features] = data_df[dense_features].fillna(0.0)
for f in dense_features:
data_df[f] = data_df[f].apply(lambda x: np.log(x + 1) if x > -1 else -1)
#离散型特征缺失值填充-1
data_df[sparse_features] = data_df[sparse_features].fillna("-1")
for f in sparse_features:
#标准化
lbe = LabelEncoder()
data_df[f] = lbe.fit_transform(data_df[f])
#返回
return data_df[dense_features + sparse_features]
def build_input_layers(feature_columns):
"""
构建输入层
:param feature_columns : 数据集中的所有特征对应的特征标记
"""
# 构建input 层字典，并以dense 和 sparse 两类字典的形式返回
dense_input_dict,sparse_input_dict = {} ,{}
for fc in feature_columns:
if isinstance(fc,SparseFeat):
sparse_input_dict[fc.name] = Input(shape = (1,), name = fc.name)
elif isinstance(fc,DenseFeat):
dense_input_dict[fc.name] = Input(shape = (fc.dimension, ), name = fc.name)
return dense_input_dict, sparse_input_dict
def build_embedding_layers(feature_columns, input_layers_dict, is_linear):
# 定义一个embedding层对应的字典
embedding_layers_dict = dict()
# 将特征中的sparse特征筛选出来
sparse_feature_columns = list(filter(lambda x: isinstance(x, SparseFeat), feature_columns)) if feature_columns else []
# 如果是用于线性部分的embedding层，其维度为1，否则维度就是自己定义的embedding维度
if is_linear:
for fc in sparse_feature_columns:
embedding_layers_dict[fc.name] = Embedding(fc.vocabulary_size + 1, 1, name = '1d_emb_' + fc.name)
else:
for fc in sparse_feature_columns:
embedding_layers_dict[fc.name] = Embedding(fc.vocabulary_size + 1, fc.embedding_dim , name = 'kd_emb_' + fc.name)
return embedding_layers_dict
def get_linear_logits(dense_input_dict, sparse_input_dict, sparse_feature_columns):
# 将所有的dense特征的Input层，然后经过一个全连接层得到dense特征的logits
concat_dense_inputs = Concatenate(axis=1)(list(dense_input_dict.values()))
dense_logits_output = Dense(1)(concat_dense_inputs)
# 获取linear部分sparse特征的embedding层，这里使用embedding的原因是：
# 对于linear部分直接将特征进行onehot然后通过一个全连接层，当维度特别大的时候，计算比较慢
# 使用embedding层的好处就是可以通过查表的方式获取到哪些非零的元素对应的权重，然后在将这些权重相加，效率比较高
linear_embedding_layers = build_embedding_layers(sparse_feature_columns, sparse_input_dict, is_linear=True)
# 将一维的embedding拼接，注意这里需要使用一个Flatten层，使维度对应
sparse_1d_embed = []
for fc in sparse_feature_columns:
feat_input = sparse_input_dict[fc.name]
embed = Flatten()(linear_embedding_layers[fc.name](feat_input)) # B x 1
sparse_1d_embed.append(embed)
# embedding中查询得到的权重就是对应onehot向量中一个位置的权重，所以后面不用再接一个全连接了，本身一维的embedding就相当于全连接
# 只不过是这里的输入特征只有0和1，所以直接向非零元素对应的权重相加就等同于进行了全连接操作(非零元素部分乘的是1)
sparse_logits_output = Add()(sparse_1d_embed)
# 最终将dense特征和sparse特征对应的logits相加，得到最终linear的logits
linear_logits = Add()([dense_logits_output, sparse_logits_output])
return linear_logits
# 将所有的sparse特征embedding拼接
def concat_embedding_list(feature_columns, input_layer_dict, embedding_layer_dict, flatten = False):
# 将sparse特征筛选出来
sparse_feature_columns = list(filter(lambda x: isinstance(x, SparseFeat), feature_columns))
embedding_list = []
for fc in sparse_feature_columns:
# 获取输入层
_input = input_layer_dict[fc.name]
# B x 1 x dim 获取对应的embedding层
_embed = embedding_layer_dict[fc.name]
# B x dim 将input层输入到embedding层中
embed = _embed(_input)
# 是否需要flatten , 如果embedding列表最终是直接输入到Dense层中，需要进行Flatten,否则不需要
if flatten:
embed = Flatten()(embed)
embedding_list.append(embed)
return embedding_list
def get_dnn_logits(dense_input_dict, sparse_input_dict, sparse_feature_columns, dnn_embedding_layers):
concat_dense_inputs = Concatenate(axis=1)(list(dense_input_dict.values())) # B x n1 (n表示的是dense特征的维度)
sparse_kd_embed = concat_embedding_list(sparse_feature_columns, sparse_input_dict, dnn_embedding_layers, flatten=True)
concat_sparse_kd_embed = Concatenate(axis=1)(sparse_kd_embed) # B x n2k (n2表示的是Sparse特征的维度)
dnn_input = Concatenate(axis=1)([concat_dense_inputs, concat_sparse_kd_embed]) # B x (n2k + n1)
# dnn层，这里的Dropout参数，Dense中的参数及Dense的层数都可以自己设定
dnn_out = Dropout(0.5)(Dense(1024, activation='relu')(dnn_input))
dnn_out = Dropout(0.3)(Dense(512, activation='relu')(dnn_out))
dnn_out = Dropout(0.1)(Dense(256, activation='relu')(dnn_out))
dnn_logits = Dense(1)(dnn_out)
return dnn_logits
# Wide&Deep 模型的wide部分及Deep部分的特征选择，应该根据实际的业务场景去确定哪些特征应该放在Wide部分，哪些特征应该放在Deep部分
def WideNDeep(linear_feature_columns, dnn_feature_columns):
# 构建输入层，即所有特征对应的Input()层，这里使用字典的形式返回，方便后续构建模型
dense_input_dict, sparse_input_dict = build_input_layers(linear_feature_columns + dnn_feature_columns)
# 将linear部分的特征中sparse特征筛选出来，后面用来做1维的embedding
linear_sparse_feature_columns = list(filter(lambda x: isinstance(x, SparseFeat), linear_feature_columns))
# 构建模型的输入层，模型的输入层不能是字典的形式，应该将字典的形式转换成列表的形式
# 注意：这里实际的输入与Input()层的对应，是通过模型输入时候的字典数据的key与对应name的Input层
input_layers = list(dense_input_dict.values()) + list(sparse_input_dict.values())
# Wide&Deep模型论文中Wide部分使用的特征比较简单，并且得到的特征非常的稀疏，所以使用了FTRL优化Wide部分（这里没有实现FTRL）
# 但是是根据他们业务进行选择的，我们这里将所有可能用到的特征都输入到Wide部分，具体的细节可以根据需求进行修改
linear_logits = get_linear_logits(dense_input_dict, sparse_input_dict, linear_sparse_feature_columns)
# 构建维度为k的embedding层，这里使用字典的形式返回，方便后面搭建模型
embedding_layers = build_embedding_layers(dnn_feature_columns, sparse_input_dict, is_linear=False)
dnn_sparse_feature_columns = list(filter(lambda x: isinstance(x, SparseFeat), dnn_feature_columns))
# 在Wide&Deep模型中，deep部分的输入是将dense特征和embedding特征拼在一起输入到dnn中
dnn_logits = get_dnn_logits(dense_input_dict, sparse_input_dict, dnn_sparse_feature_columns, embedding_layers)
# 将linear,dnn的logits相加作为最终的logits
output_logits = Add()([linear_logits, dnn_logits])
# 这里的激活函数使用sigmoid
output_layer = Activation("sigmoid")(output_logits)
model = Model(input_layers, output_layer)
return model
if __name__ == "__main__":
# 读取数据
data = pd.read_csv('./data/criteo_sample.txt')
# 划分dense和sparse特征
columns = data.columns.values
dense_features = [feat for feat in columns if 'I' in feat]
sparse_features = [feat for feat in columns if 'C' in feat]
# 简单的数据预处理
train_data = data_process(data, dense_features, sparse_features)
train_data['label'] = data['label']
# 将特征分组，分成linear部分和dnn部分(根据实际场景进行选择)，并将分组之后的特征做标记（使用DenseFeat, SparseFeat）
linear_feature_columns = [SparseFeat(feat, vocabulary_size=data[feat].nunique(),embedding_dim=4)
for i,feat in enumerate(sparse_features)] + [DenseFeat(feat, 1,)
for feat in dense_features]
dnn_feature_columns = [SparseFeat(feat, vocabulary_size=data[feat].nunique(),embedding_dim=4)
for i,feat in enumerate(sparse_features)] + [DenseFeat(feat, 1,)
for feat in dense_features]
# 构建WideNDeep模型
history = WideNDeep(linear_feature_columns, dnn_feature_columns)
history.summary()
history.compile(optimizer="adam",
loss="binary_crossentropy",
metrics=["binary_crossentropy", tf.keras.metrics.AUC(name='auc')])
# 将输入数据转化成字典的形式输入
train_model_input = {name: data[name] for name in dense_features + sparse_features}
# 模型训练
history.fit(train_model_input, train_data['label'].values,
batch_size=64, epochs=8, validation_split=0.2, ) Model: "functional_3"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
C1 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C2 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C3 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C4 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C5 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C6 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C7 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C8 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C9 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C10 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C11 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C12 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C13 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C14 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C15 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C16 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C17 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C18 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C19 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C20 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C21 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C22 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C23 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C24 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C25 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
C26 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
kd_emb_C1 (Embedding) (None, 1, 4) 112 C1[0][0]
__________________________________________________________________________________________________
kd_emb_C2 (Embedding) (None, 1, 4) 372 C2[0][0]
__________________________________________________________________________________________________
kd_emb_C3 (Embedding) (None, 1, 4) 692 C3[0][0]
__________________________________________________________________________________________________
kd_emb_C4 (Embedding) (None, 1, 4) 632 C4[0][0]
__________________________________________________________________________________________________
kd_emb_C5 (Embedding) (None, 1, 4) 52 C5[0][0]
__________________________________________________________________________________________________
kd_emb_C6 (Embedding) (None, 1, 4) 32 C6[0][0]
__________________________________________________________________________________________________
kd_emb_C7 (Embedding) (None, 1, 4) 736 C7[0][0]
__________________________________________________________________________________________________
kd_emb_C8 (Embedding) (None, 1, 4) 80 C8[0][0]
__________________________________________________________________________________________________
kd_emb_C9 (Embedding) (None, 1, 4) 12 C9[0][0]
__________________________________________________________________________________________________
kd_emb_C10 (Embedding) (None, 1, 4) 572 C10[0][0]
__________________________________________________________________________________________________
kd_emb_C11 (Embedding) (None, 1, 4) 696 C11[0][0]
__________________________________________________________________________________________________
kd_emb_C12 (Embedding) (None, 1, 4) 684 C12[0][0]
__________________________________________________________________________________________________
kd_emb_C13 (Embedding) (None, 1, 4) 668 C13[0][0]
__________________________________________________________________________________________________
kd_emb_C14 (Embedding) (None, 1, 4) 60 C14[0][0]
__________________________________________________________________________________________________
kd_emb_C15 (Embedding) (None, 1, 4) 684 C15[0][0]
__________________________________________________________________________________________________
kd_emb_C16 (Embedding) (None, 1, 4) 676 C16[0][0]
__________________________________________________________________________________________________
kd_emb_C17 (Embedding) (None, 1, 4) 40 C17[0][0]
__________________________________________________________________________________________________
kd_emb_C18 (Embedding) (None, 1, 4) 512 C18[0][0]
__________________________________________________________________________________________________
kd_emb_C19 (Embedding) (None, 1, 4) 180 C19[0][0]
__________________________________________________________________________________________________
kd_emb_C20 (Embedding) (None, 1, 4) 20 C20[0][0]
__________________________________________________________________________________________________
kd_emb_C21 (Embedding) (None, 1, 4) 680 C21[0][0]
__________________________________________________________________________________________________
kd_emb_C22 (Embedding) (None, 1, 4) 28 C22[0][0]
__________________________________________________________________________________________________
kd_emb_C23 (Embedding) (None, 1, 4) 44 C23[0][0]
__________________________________________________________________________________________________
kd_emb_C24 (Embedding) (None, 1, 4) 504 C24[0][0]
__________________________________________________________________________________________________
kd_emb_C25 (Embedding) (None, 1, 4) 84 C25[0][0]
__________________________________________________________________________________________________
kd_emb_C26 (Embedding) (None, 1, 4) 364 C26[0][0]
__________________________________________________________________________________________________
I1 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I2 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I3 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I4 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I5 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I6 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I7 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I8 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I9 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I10 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I11 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I12 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
I13 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
flatten_52 (Flatten) (None, 4) 0 kd_emb_C1[0][0]
__________________________________________________________________________________________________
flatten_53 (Flatten) (None, 4) 0 kd_emb_C2[0][0]
__________________________________________________________________________________________________
flatten_54 (Flatten) (None, 4) 0 kd_emb_C3[0][0]
__________________________________________________________________________________________________
flatten_55 (Flatten) (None, 4) 0 kd_emb_C4[0][0]
__________________________________________________________________________________________________
flatten_56 (Flatten) (None, 4) 0 kd_emb_C5[0][0]
__________________________________________________________________________________________________
flatten_57 (Flatten) (None, 4) 0 kd_emb_C6[0][0]
__________________________________________________________________________________________________
flatten_58 (Flatten) (None, 4) 0 kd_emb_C7[0][0]
__________________________________________________________________________________________________
flatten_59 (Flatten) (None, 4) 0 kd_emb_C8[0][0]
__________________________________________________________________________________________________
flatten_60 (Flatten) (None, 4) 0 kd_emb_C9[0][0]
__________________________________________________________________________________________________
flatten_61 (Flatten) (None, 4) 0 kd_emb_C10[0][0]
__________________________________________________________________________________________________
flatten_62 (Flatten) (None, 4) 0 kd_emb_C11[0][0]
__________________________________________________________________________________________________
flatten_63 (Flatten) (None, 4) 0 kd_emb_C12[0][0]
__________________________________________________________________________________________________
flatten_64 (Flatten) (None, 4) 0 kd_emb_C13[0][0]
__________________________________________________________________________________________________
flatten_65 (Flatten) (None, 4) 0 kd_emb_C14[0][0]
__________________________________________________________________________________________________
flatten_66 (Flatten) (None, 4) 0 kd_emb_C15[0][0]
__________________________________________________________________________________________________
flatten_67 (Flatten) (None, 4) 0 kd_emb_C16[0][0]
__________________________________________________________________________________________________
flatten_68 (Flatten) (None, 4) 0 kd_emb_C17[0][0]
__________________________________________________________________________________________________
flatten_69 (Flatten) (None, 4) 0 kd_emb_C18[0][0]
__________________________________________________________________________________________________
flatten_70 (Flatten) (None, 4) 0 kd_emb_C19[0][0]
__________________________________________________________________________________________________
flatten_71 (Flatten) (None, 4) 0 kd_emb_C20[0][0]
__________________________________________________________________________________________________
flatten_72 (Flatten) (None, 4) 0 kd_emb_C21[0][0]
__________________________________________________________________________________________________
flatten_73 (Flatten) (None, 4) 0 kd_emb_C22[0][0]
__________________________________________________________________________________________________
flatten_74 (Flatten) (None, 4) 0 kd_emb_C23[0][0]
__________________________________________________________________________________________________
flatten_75 (Flatten) (None, 4) 0 kd_emb_C24[0][0]
__________________________________________________________________________________________________
flatten_76 (Flatten) (None, 4) 0 kd_emb_C25[0][0]
__________________________________________________________________________________________________
flatten_77 (Flatten) (None, 4) 0 kd_emb_C26[0][0]
__________________________________________________________________________________________________
concatenate_4 (Concatenate) (None, 13) 0 I1[0][0]
I2[0][0]
I3[0][0]
I4[0][0]
I5[0][0]
I6[0][0]
I7[0][0]
I8[0][0]
I9[0][0]
I10[0][0]
I11[0][0]
I12[0][0]
I13[0][0]
__________________________________________________________________________________________________
concatenate_5 (Concatenate) (None, 104) 0 flatten_52[0][0]
flatten_53[0][0]
flatten_54[0][0]
flatten_55[0][0]
flatten_56[0][0]
flatten_57[0][0]
flatten_58[0][0]
flatten_59[0][0]
flatten_60[0][0]
flatten_61[0][0]
flatten_62[0][0]
flatten_63[0][0]
flatten_64[0][0]
flatten_65[0][0]
flatten_66[0][0]
flatten_67[0][0]
flatten_68[0][0]
flatten_69[0][0]
flatten_70[0][0]
flatten_71[0][0]
flatten_72[0][0]
flatten_73[0][0]
flatten_74[0][0]
flatten_75[0][0]
flatten_76[0][0]
flatten_77[0][0]
__________________________________________________________________________________________________
concatenate_6 (Concatenate) (None, 117) 0 concatenate_4[0][0]
concatenate_5[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 1024) 120832 concatenate_6[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 1024) 0 dense_2[0][0]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 512) 524800 dropout[0][0]
__________________________________________________________________________________________________
1d_emb_C1 (Embedding) (None, 1, 1) 28 C1[0][0]
__________________________________________________________________________________________________
1d_emb_C2 (Embedding) (None, 1, 1) 93 C2[0][0]
__________________________________________________________________________________________________
1d_emb_C3 (Embedding) (None, 1, 1) 173 C3[0][0]
__________________________________________________________________________________________________
1d_emb_C4 (Embedding) (None, 1, 1) 158 C4[0][0]
__________________________________________________________________________________________________
1d_emb_C5 (Embedding) (None, 1, 1) 13 C5[0][0]
__________________________________________________________________________________________________
1d_emb_C6 (Embedding) (None, 1, 1) 8 C6[0][0]
__________________________________________________________________________________________________
1d_emb_C7 (Embedding) (None, 1, 1) 184 C7[0][0]
__________________________________________________________________________________________________
1d_emb_C8 (Embedding) (None, 1, 1) 20 C8[0][0]
__________________________________________________________________________________________________
1d_emb_C9 (Embedding) (None, 1, 1) 3 C9[0][0]
__________________________________________________________________________________________________
1d_emb_C10 (Embedding) (None, 1, 1) 143 C10[0][0]
__________________________________________________________________________________________________
1d_emb_C11 (Embedding) (None, 1, 1) 174 C11[0][0]
__________________________________________________________________________________________________
1d_emb_C12 (Embedding) (None, 1, 1) 171 C12[0][0]
__________________________________________________________________________________________________
1d_emb_C13 (Embedding) (None, 1, 1) 167 C13[0][0]
__________________________________________________________________________________________________
1d_emb_C14 (Embedding) (None, 1, 1) 15 C14[0][0]
__________________________________________________________________________________________________
1d_emb_C15 (Embedding) (None, 1, 1) 171 C15[0][0]
__________________________________________________________________________________________________
1d_emb_C16 (Embedding) (None, 1, 1) 169 C16[0][0]
__________________________________________________________________________________________________
1d_emb_C17 (Embedding) (None, 1, 1) 10 C17[0][0]
__________________________________________________________________________________________________
1d_emb_C18 (Embedding) (None, 1, 1) 128 C18[0][0]
__________________________________________________________________________________________________
1d_emb_C19 (Embedding) (None, 1, 1) 45 C19[0][0]
__________________________________________________________________________________________________
1d_emb_C20 (Embedding) (None, 1, 1) 5 C20[0][0]
__________________________________________________________________________________________________
1d_emb_C21 (Embedding) (None, 1, 1) 170 C21[0][0]
__________________________________________________________________________________________________
1d_emb_C22 (Embedding) (None, 1, 1) 7 C22[0][0]
__________________________________________________________________________________________________
1d_emb_C23 (Embedding) (None, 1, 1) 11 C23[0][0]
__________________________________________________________________________________________________
1d_emb_C24 (Embedding) (None, 1, 1) 126 C24[0][0]
__________________________________________________________________________________________________
1d_emb_C25 (Embedding) (None, 1, 1) 21 C25[0][0]
__________________________________________________________________________________________________
1d_emb_C26 (Embedding) (None, 1, 1) 91 C26[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 512) 0 dense_3[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 13) 0 I1[0][0]
I2[0][0]
I3[0][0]
I4[0][0]
I5[0][0]
I6[0][0]
I7[0][0]
I8[0][0]
I9[0][0]
I10[0][0]
I11[0][0]
I12[0][0]
I13[0][0]
__________________________________________________________________________________________________
flatten_26 (Flatten) (None, 1) 0 1d_emb_C1[0][0]
__________________________________________________________________________________________________
flatten_27 (Flatten) (None, 1) 0 1d_emb_C2[0][0]
__________________________________________________________________________________________________
flatten_28 (Flatten) (None, 1) 0 1d_emb_C3[0][0]
__________________________________________________________________________________________________
flatten_29 (Flatten) (None, 1) 0 1d_emb_C4[0][0]
__________________________________________________________________________________________________
flatten_30 (Flatten) (None, 1) 0 1d_emb_C5[0][0]
__________________________________________________________________________________________________
flatten_31 (Flatten) (None, 1) 0 1d_emb_C6[0][0]
__________________________________________________________________________________________________
flatten_32 (Flatten) (None, 1) 0 1d_emb_C7[0][0]
__________________________________________________________________________________________________
flatten_33 (Flatten) (None, 1) 0 1d_emb_C8[0][0]
__________________________________________________________________________________________________
flatten_34 (Flatten) (None, 1) 0 1d_emb_C9[0][0]
__________________________________________________________________________________________________
flatten_35 (Flatten) (None, 1) 0 1d_emb_C10[0][0]
__________________________________________________________________________________________________
flatten_36 (Flatten) (None, 1) 0 1d_emb_C11[0][0]
__________________________________________________________________________________________________
flatten_37 (Flatten) (None, 1) 0 1d_emb_C12[0][0]
__________________________________________________________________________________________________
flatten_38 (Flatten) (None, 1) 0 1d_emb_C13[0][0]
__________________________________________________________________________________________________
flatten_39 (Flatten) (None, 1) 0 1d_emb_C14[0][0]
__________________________________________________________________________________________________
flatten_40 (Flatten) (None, 1) 0 1d_emb_C15[0][0]
__________________________________________________________________________________________________
flatten_41 (Flatten) (None, 1) 0 1d_emb_C16[0][0]
__________________________________________________________________________________________________
flatten_42 (Flatten) (None, 1) 0 1d_emb_C17[0][0]
__________________________________________________________________________________________________
flatten_43 (Flatten) (None, 1) 0 1d_emb_C18[0][0]
__________________________________________________________________________________________________
flatten_44 (Flatten) (None, 1) 0 1d_emb_C19[0][0]
__________________________________________________________________________________________________
flatten_45 (Flatten) (None, 1) 0 1d_emb_C20[0][0]
__________________________________________________________________________________________________
flatten_46 (Flatten) (None, 1) 0 1d_emb_C21[0][0]
__________________________________________________________________________________________________
flatten_47 (Flatten) (None, 1) 0 1d_emb_C22[0][0]
__________________________________________________________________________________________________
flatten_48 (Flatten) (None, 1) 0 1d_emb_C23[0][0]
__________________________________________________________________________________________________
flatten_49 (Flatten) (None, 1) 0 1d_emb_C24[0][0]
__________________________________________________________________________________________________
flatten_50 (Flatten) (None, 1) 0 1d_emb_C25[0][0]
__________________________________________________________________________________________________
flatten_51 (Flatten) (None, 1) 0 1d_emb_C26[0][0]
__________________________________________________________________________________________________
dense_4 (Dense) (None, 256) 131328 dropout_1[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 1) 14 concatenate_3[0][0]
__________________________________________________________________________________________________
add (Add) (None, 1) 0 flatten_26[0][0]
flatten_27[0][0]
flatten_28[0][0]
flatten_29[0][0]
flatten_30[0][0]
flatten_31[0][0]
flatten_32[0][0]
flatten_33[0][0]
flatten_34[0][0]
flatten_35[0][0]
flatten_36[0][0]
flatten_37[0][0]
flatten_38[0][0]
flatten_39[0][0]
flatten_40[0][0]
flatten_41[0][0]
flatten_42[0][0]
flatten_43[0][0]
flatten_44[0][0]
flatten_45[0][0]
flatten_46[0][0]
flatten_47[0][0]
flatten_48[0][0]
flatten_49[0][0]
flatten_50[0][0]
flatten_51[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 256) 0 dense_4[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, 1) 0 dense_1[0][0]
add[0][0]
__________________________________________________________________________________________________
dense_5 (Dense) (None, 1) 257 dropout_2[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, 1) 0 add_1[0][0]
dense_5[0][0]
__________________________________________________________________________________________________
activation (Activation) (None, 1) 0 add_2[0][0]
==================================================================================================
Total params: 788,751
Trainable params: 788,751
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/8
3/3 [==============================] - ETA: 0s - loss: 0.6686 - binary_crossentropy: 0.6686 - auc: 0.660 - 1s 322ms/step - loss: 0.8642 - binary_crossentropy: 0.8642 - auc: 0.5307 - val_loss: 0.9196 - val_binary_crossentropy: 0.9196 - val_auc: 0.6852
Epoch 2/8
3/3 [==============================] - ETA: 0s - loss: 0.6993 - binary_crossentropy: 0.6993 - auc: 0.587 - 0s 33ms/step - loss: 0.7057 - binary_crossentropy: 0.7057 - auc: 0.5780 - val_loss: 0.6952 - val_binary_crossentropy: 0.6952 - val_auc: 0.6667
Epoch 3/8
3/3 [==============================] - ETA: 0s - loss: 0.5635 - binary_crossentropy: 0.5635 - auc: 0.688 - 0s 32ms/step - loss: 0.6362 - binary_crossentropy: 0.6362 - auc: 0.5944 - val_loss: 0.6602 - val_binary_crossentropy: 0.6602 - val_auc: 0.6880
Epoch 4/8
3/3 [==============================] - ETA: 0s - loss: 0.6217 - binary_crossentropy: 0.6217 - auc: 0.591 - 0s 39ms/step - loss: 0.6272 - binary_crossentropy: 0.6272 - auc: 0.5453 - val_loss: 0.6744 - val_binary_crossentropy: 0.6744 - val_auc: 0.7464
Epoch 5/8
3/3 [==============================] - ETA: 0s - loss: 0.5482 - binary_crossentropy: 0.5482 - auc: 0.564 - 0s 34ms/step - loss: 0.5380 - binary_crossentropy: 0.5380 - auc: 0.6284 - val_loss: 0.6426 - val_binary_crossentropy: 0.6426 - val_auc: 0.7265
Epoch 6/8
3/3 [==============================] - ETA: 0s - loss: 0.5465 - binary_crossentropy: 0.5465 - auc: 0.635 - 0s 31ms/step - loss: 0.4800 - binary_crossentropy: 0.4800 - auc: 0.7146 - val_loss: 0.6307 - val_binary_crossentropy: 0.6307 - val_auc: 0.6652
Epoch 7/8
3/3 [==============================] - ETA: 0s - loss: 0.5032 - binary_crossentropy: 0.5032 - auc: 0.699 - 0s 30ms/step - loss: 0.4948 - binary_crossentropy: 0.4948 - auc: 0.6784 - val_loss: 0.6486 - val_binary_crossentropy: 0.6486 - val_auc: 0.6524
Epoch 8/8
3/3 [==============================] - ETA: 0s - loss: 0.4625 - binary_crossentropy: 0.4625 - auc: 0.822 - 0s 36ms/step - loss: 0.4164 - binary_crossentropy: 0.4164 - auc: 0.8191 - val_loss: 0.6470 - val_binary_crossentropy: 0.6470 - val_auc: 0.6595参考：
1） https://www.jianshu.com/p/e73146094478
2） https://zhuanlan.zhihu.com/p/57247478
3） 王喆 《深度学习推荐系统》
扫一扫
790
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
