PaddleRec之Wide&Deep的傻瓜式教程

AI Studio

于 2023-02-25 14:52:38 发布

阅读量376

点赞数 1

分类专栏：人工智能文章标签：机器学习深度学习人工智能

原文链接：https://aistudio.baidu.com/aistudio/projectdetail/5569918?forkThirdPart=1

版权

人工智能专栏收录该内容

180 篇文章 30 订阅

订阅专栏

★★★ 本文源自AlStudio社区精品项目,【点击此处】查看更多精品内容 >>>

PaddleRec之Wide&Deep的傻瓜式教程

wide&deep

项目介绍

训练时，给模型提供一段时间的广告流量，根据模型计算出的广告点击情况与真实的广告点击情况进行对比，得出损失值，反向传播得到梯度，让模型反复学习。

预测时，给模型提供训练数据后一天的广告点击流量，然而后一天的广告点击情况我们是知道的，只是没有提供给模型，待模型预测出后一天的广告点击情况时，再与实际情况做对比，从而得到预测结果中的auc值

模型简介

《Wide & Deep Learning for Recommender Systems》是Google 2016年发布的推荐框架，wide&deep设计了一种融合浅层（wide）模型和深层（deep）模型进行联合训练的框架，综合利用浅层模型的记忆能力和深层模型的泛化能力，实现单模型对推荐系统准确性和扩展性的兼顾。从推荐效果和服务性能两方面进行评价：

效果上，在Google Play 进行线上A/B实验，wide&deep模型相比高度优化的Wide浅层模型，app下载率+3.9%。相比deep模型也有一定提升。
性能上，通过切分一次请求需要处理的app 的Batch size为更小的size，并利用多线程并行请求达到提高处理效率的目的。单次响应耗时从31ms下降到14ms。

数据准备

训练及测试数据集选用Display Advertising Challenge所用的Criteo数据集。该数据集包括两部分：训练集和测试集。训练集包含一段时间内Criteo的部分流量，测试集则对应训练数据后一天的广告点击流量。
每一行数据格式如下所示：
<integer feature 1> … <integer feature 13> <categorical feature 1> … <categorical feature 26>
其中<label>表示广告是否被点击，点击用1表示，未点击用0表示。<integer feature>代表数值特征（连续特征），共有13个连续特征。<categorical feature>代表分类特征（离散特征），共有26个离散特征。相邻两个特征用\t分隔，缺失特征用空格表示。测试集中<label>特征已被移除。
在模型目录的data目录下为您准备了快速运行的示例数据，若需要使用全量数据可以参考下方效果复现部分。

数据展示(训练集)

click:0 dense_feature:0.0 dense_feature:0.00497512437811 dense_feature:0.05 dense_feature:0.08 dense_feature:0.207421875 dense_feature:0.028 dense_feature:0.35 dense_feature:0.08 dense_feature:0.082 dense_feature:0.0 dense_feature:0.4 dense_feature:0.0 dense_feature:0.08 1:737395 2:210498 3:903564 4:286224 5:286835 6:906818 7:906116 8:67180 9:27346 10:51086 11:142177 12:95024 13:157883 14:873363 15:600281 16:812592 17:228085 18:35900 19:880474 20:984402 21:100885 22:26235 23:410878 24:798162 25:499868 26:306163
click:1 dense_feature:0.0 dense_feature:0.932006633499 dense_feature:0.02 dense_feature:0.14 dense_feature:0.0395625 dense_feature:0.328 dense_feature:0.98 dense_feature:0.12 dense_feature:1.886 dense_feature:0.0 dense_feature:1.8 dense_feature:0.0 dense_feature:0.14 1:715353 2:761523 3:432904 4:892267 5:515218 6:948614 7:266726 8:67180 9:27346 10:266081 11:286126 12:789480 13:49621 14:255651 15:47663 16:79797 17:342789 18:616331 19:880474 20:984402 21:242209 22:26235 23:669531 24:26284 25:269955 26:187951
click:0 dense_feature:0.0 dense_feature:0.00829187396352 dense_feature:0.08 dense_feature:0.06 dense_feature:0.14125 dense_feature:0.076 dense_feature:0.05 dense_feature:0.22 dense_feature:0.208 dense_feature:0.0 dense_feature:0.2 dense_feature:0.0 dense_feature:0.06 1:737395 2:952384 3:511141 4:271077 5:286835 6:948614 7:903547 8:507110 9:27346 10:56047 11:612953 12:747707 13:977426 14:671506 15:158148 16:833738 17:342789 18:427155 19:880474 20:537425 21:916237 22:26235 23:468277 24:676936 25:751788 26:363967
click:0 dense_feature:0.0 dense_feature:0.124378109453 dense_feature:0.02 dense_feature:0.04 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.08 dense_feature:0.024 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.04 1:210127 2:286436 3:183920 4:507656 5:286835 6:906818 7:199553 8:67180 9:502607 10:708281 11:809876 12:888238 13:375164 14:202774 15:459895 16:475933 17:555571 18:847163 19:26230 20:26229 21:808836 22:191474 23:410878 24:315120 25:26224 26:26223
click:0 dense_feature:0.1 dense_feature:0.0149253731343 dense_feature:0.34 dense_feature:0.32 dense_feature:0.016421875 dense_feature:0.098 dense_feature:0.04 dense_feature:0.96 dense_feature:0.202 dense_feature:0.1 dense_feature:0.2 dense_feature:0.0 dense_feature:0.32 1:230803 2:817085 3:539110 4:388629 5:286835 6:948614 7:586040 8:67180 9:27346 10:271155 11:176640 12:827381 13:36881 14:202774 15:397299 16:411672 17:342789 18:474060 19:880474 20:984402 21:216871 22:26235 23:761351 24:787115 25:884722 26:904135
click:0 dense_feature:0.0 dense_feature:0.00829187396352 dense_feature:0.13 dense_feature:0.04 dense_feature:0.246203125 dense_feature:0.108 dense_feature:0.05 dense_feature:0.04 dense_feature:0.03 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.04 1:737395 2:64837 3:259267 4:336976 5:515218 6:154084 7:847938 8:67180 9:27346 10:708281 11:776766 12:964800 13:324323 14:873363 15:212708 16:637238 17:681378 18:895034 19:673458 20:984402 21:18600 22:26235 23:410878 24:787115 25:884722 26:355412
click:0 dense_feature:0.0 dense_feature:0.028192371476 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0245625 dense_feature:0.016 dense_feature:0.04 dense_feature:0.12 dense_feature:0.016 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:737395 2:554760 3:661483 4:263696 5:938478 6:906818 7:786926 8:67180 9:27346 10:245862 11:668197 12:745676 13:432600 14:413795 15:751427 16:272410 17:342789 18:422136 19:26230 20:26229 21:452501 22:26235 23:51381 24:776636 25:26224 26:26223
click:0 dense_feature:0.0 dense_feature:0.00497512437811 dense_feature:1.95 dense_feature:0.28 dense_feature:0.092828125 dense_feature:0.57 dense_feature:0.06 dense_feature:0.4 dense_feature:0.4 dense_feature:0.0 dense_feature:0.2 dense_feature:0.0 dense_feature:0.4 1:371155 2:817085 3:773609 4:555449 5:938478 6:906818 7:166117 8:507110 9:27346 10:545822 11:316654 12:172765 13:989600 14:255651 15:792372 16:606361 17:342789 18:566554 19:880474 20:984402 21:235256 22:191474 23:700326 24:787115 25:884722 26:569095
click:0 dense_feature:0.0 dense_feature:0.0912106135987 dense_feature:0.01 dense_feature:0.02 dense_feature:0.06625 dense_feature:0.018 dense_feature:0.05 dense_feature:0.06 dense_feature:0.098 dense_feature:0.0 dense_feature:0.4 dense_feature:0.0 dense_feature:0.04 1:230803 2:531472 3:284417 4:661677 5:938478 6:553107 7:21150 8:49466 9:27346 10:526914 11:164508 12:631773 13:882348 14:873363 15:523948 16:687081 17:342789 18:271301 19:26230 20:26229 21:647160 22:26235 23:410878 24:231695 25:26224 26:26223
click:1 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.01 dense_feature:0.02 dense_feature:0.02153125 dense_feature:0.092 dense_feature:0.05 dense_feature:0.68 dense_feature:0.472 dense_feature:0.0 dense_feature:0.3 dense_feature:0.0 dense_feature:0.02 1:737395 2:532829 3:320762 4:887282 5:286835 6:25207 7:640357 8:67180 9:27346 10:695831 11:739268 12:835325 13:402539 14:873363 15:125813 16:168896 17:342789 18:374414 19:26230 20:26229 21:850229 22:26235 23:410878 24:480027 25:26224 26:26223

模型组网

wide&deep设计了一种融合浅层（wide）模型和深层（deep）模型进行联合训练的框架，综合利用浅层模型的记忆能力和深层模型的泛化能力，实现单模型对推荐系统准确性和扩展性的兼顾。

wide&deep:

效果复现

为了方便使用者能够快速的跑通每一个模型，我们在每个模型下都提供了样例数据。如果需要复现readme中的效果,请按如下步骤依次操作即可。
在全量数据下模型的指标如下：

模型	auc	batch_size	thread_num	epoch_num	Time of each epoch
wide_deep	0.79	512	1	4	约2小时

下载数据集

cd /home/aistudio/PaddleRec/datasets/criteo

!bash run.sh

训练

config.yaml(配置文件)

runner:
  train_data_dir: "data/sample_data/train"
  train_reader_path: "criteo_reader" # importlib format
  use_gpu: False
  use_auc: True
  train_batch_size: 50
  epochs: 4
  print_interval: 2
  # model_init_path: "models/rank/wide_deep/output_model_wide_deep/2" # init model
  model_save_path: "output_model_wide_deep"
  test_data_dir: "data/sample_data/train"
  infer_reader_path: "criteo_reader" # importlib format
  infer_batch_size: 5
  infer_load_path: "output_model_wide_deep"
  infer_start_epoch: 3
  infer_end_epoch: 4
  #use inference save model
  use_inference: False
  save_inference_feed_varnames: ["C1","C2","C3","C4","C5","C6","C7","C8","C9","C10","C11","C12","C13","C14","C15","C16","C17","C18","C19","C20","C21","C22","C23","C24","C25","C26","dense_input"]
  save_inference_fetch_varnames: ["sigmoid_0.tmp_0"]
  #use fleet
  use_fleet: False

# hyper parameters of user-defined network
hyper_parameters:
  # optimizer config
  optimizer:
    class: Adam
    learning_rate: 0.001
    strategy: async
  # user-defined <key, value> pairs
  sparse_inputs_slots: 27
  sparse_feature_number: 1000001
  sparse_feature_dim: 9
  dense_input_dim: 13
  fc_sizes: [512, 256, 128, 32]
  distributed_embedding: 0

net.py(网络搭建)

import paddle
import paddle.nn as nn
import paddle.nn.functional as F
import math


class WideDeepLayer(nn.Layer):
    def __init__(self, sparse_feature_number, sparse_feature_dim,
                 dense_feature_dim, num_field, layer_sizes):
        super(WideDeepLayer, self).__init__()
        self.sparse_feature_number = sparse_feature_number
        self.sparse_feature_dim = sparse_feature_dim
        self.dense_feature_dim = dense_feature_dim
        self.num_field = num_field
        self.layer_sizes = layer_sizes

        self.wide_part = paddle.nn.Linear(
            in_features=self.dense_feature_dim,
            out_features=1,
            weight_attr=paddle.ParamAttr(
                initializer=paddle.nn.initializer.TruncatedNormal(
                    mean=0.0, std=1.0 / math.sqrt(self.dense_feature_dim))))

        self.embedding = paddle.nn.Embedding(
            self.sparse_feature_number,
            self.sparse_feature_dim,
            sparse=True,
            weight_attr=paddle.ParamAttr(
                name="SparseFeatFactors",
                initializer=paddle.nn.initializer.Uniform()))

        sizes = [sparse_feature_dim * num_field + dense_feature_dim
                 ] + self.layer_sizes + [1]
        acts = ["relu" for _ in range(len(self.layer_sizes))] + [None]
        self._mlp_layers = []
        for i in range(len(layer_sizes) + 1):
            linear = paddle.nn.Linear(
                in_features=sizes[i],
                out_features=sizes[i + 1],
                weight_attr=paddle.ParamAttr(
                    initializer=paddle.nn.initializer.Normal(
                        std=1.0 / math.sqrt(sizes[i]))))
            self.add_sublayer('linear_%d' % i, linear)
            self._mlp_layers.append(linear)
            if acts[i] == 'relu':
                act = paddle.nn.ReLU()
                self.add_sublayer('act_%d' % i, act)
                self._mlp_layers.append(act)

    def forward(self, sparse_inputs, dense_inputs):
        # wide part
        wide_output = self.wide_part(dense_inputs)

        # deep part
        sparse_embs = []
        for s_input in sparse_inputs:
            emb = self.embedding(s_input)
            emb = paddle.reshape(emb, shape=[-1, self.sparse_feature_dim])
            sparse_embs.append(emb)

        deep_output = paddle.concat(x=sparse_embs + [dense_inputs], axis=1)
        for n_layer in self._mlp_layers:
            deep_output = n_layer(deep_output)

        prediction = paddle.add(x=wide_output, y=deep_output)
        pred = F.sigmoid(prediction)
        print("模型预测结果为:\n{}\n".format(pred))
        return pred

# 静态图训练
# !python -u ../../../tools/static_trainer.py -m config.yaml # 全量数据运行config_bigdata.yaml
# 静态图预测
# !python -u ../../../tools/static_infer.py -m config.yaml

训练部分数据

cd /home/aistudio/PaddleRec/models/rank/wide_deep

如果要调整学习率等超参，需要在config.yaml或config_bigdata.yaml里面调整

!python -u ../../../tools/trainer.py -m config.yaml

训练全部数据

cd /home/aistudio/PaddleRec/models/rank/wide_deep

!python -u ../../../tools/trainer.py -m config_bigdata.yaml # 全量数据运行config_bigdata.yaml

推理

推理部分数据

cd /home/aistudio/PaddleRec/models/rank/wide_deep

!python -u ../../../tools/infer.py -m config.yaml

推理全部数据

cd /home/aistudio/PaddleRec/models/rank/wide_deep

!python -u ../../../tools/infer.py -m config_bigdata.yaml

AI Studio

关注

1
点赞
踩
2

收藏

觉得还不错? 一键收藏
0
评论
PaddleRec之Wide&Deep的傻瓜式教程

训练时，给模型提供一段时间的广告流量，根据模型计算出的广告点击情况与真实的广告点击情况进行对比，得出损失值，反向传播得到梯度，让模型反复学习。预测时，给模型提供训练数据后一天的广告点击流量，然而后一天的广告点击情况我们是知道的，只是没有提供给模型，待模型预测出后一天的广告点击情况时，再与实际情况做对比，从而得到预测结果中的auc值。
复制链接

扫一扫