《Wide and Deep Learning for Recommender Systems》学习笔记

最新推荐文章于 2022-09-13 02:21:35 发布

kingzone_2008

最新推荐文章于 2022-09-13 02:21:35 发布

阅读量301

点赞数

分类专栏：推荐算法深度学习

深度学习同时被 2 个专栏收录

27 篇文章 4 订阅

订阅专栏

推荐算法

22 篇文章 4 订阅

订阅专栏

顾名思义，Google提出的这篇文章将Wide Model和Deep Model结合起来进行，思路非常值得学习。

1.Wide Model

首先说一下Wide Model,就是上图中左边的那一部分，是一个简单的逻辑回归模型。这一部分比较简单，不多有一个新的思路就是交叉特征：

论文中一个比较形象的例子

For binary features, a cross-product transformation (e.g.,AND(gender=female, language=en)”) is 1 if and only if the constituent features (\gender=female” and \language=en”) are all 1, and 0 otherwise.

对应的开源代码如下：

<span style="color:#000000"><code>wide_columns = [
  gender, native_country, education, occupation, workclass, relationship, age_buckets,
  tf<span style="color:#009900">.contrib</span><span style="color:#009900">.layers</span><span style="color:#009900">.crossed</span>_column([education, occupation], hash_bucket_size=int(<span style="color:#006666">1e4</span>)),
  tf<span style="color:#009900">.contrib</span><span style="color:#009900">.layers</span><span style="color:#009900">.crossed</span>_column([native_country, occupation], hash_bucket_size=int(<span style="color:#006666">1e4</span>)),
  tf<span style="color:#009900">.contrib</span><span style="color:#009900">.layers</span><span style="color:#009900">.crossed</span>_column([age_buckets, education, occupation], hash_bucket_size=int(<span style="color:#006666">1e6</span>))]</code></span>

2.Deep Model

Deep Model是最上图中右边的部分，首先是一个embedding层，然后是两层的神经网络，最后是一个softmax。整体上看上去清晰明了，开源的代码中实现的也比较简单，有时间的话还是要研究研究源码。

公式表示如下，其中l是层数，W是系数，b是偏置，f是激活函数

对应的开源代码如下：

<span style="color:#000000"><code>deep_columns = [
  tf<span style="color:#009900">.contrib</span><span style="color:#009900">.layers</span><span style="color:#009900">.embedding</span>_column(workclass, dimension=<span style="color:#006666">8</span>),
  tf<span style="color:#009900">.contrib</span><span style="color:#009900">.layers</span><span style="color:#009900">.embedding</span>_column(education, dimension=<span style="color:#006666">8</span>),
  tf<span style="color:#009900">.contrib</span><span style="color:#009900">.layers</span><span style="color:#009900">.embedding</span>_column(gender, dimension=<span style="color:#006666">8</span>),
  tf<span style="color:#009900">.contrib</span><span style="color:#009900">.layers</span><span style="color:#009900">.embedding</span>_column(relationship, dimension=<span style="color:#006666">8</span>),
  tf<span style="color:#009900">.contrib</span><span style="color:#009900">.layers</span><span style="color:#009900">.embedding</span>_column(native_country, dimension=<span style="color:#006666">8</span>),
  tf<span style="color:#009900">.contrib</span><span style="color:#009900">.layers</span><span style="color:#009900">.embedding</span>_column(occupation, dimension=<span style="color:#006666">8</span>),
  age, education_num, capital_gain, capital_loss, hours_per_week]</code></span>

3.Wide & Deep Model

将上面两个基本Model进行结合，得到最终的模型(二分类问题)

对应的开源代码如下：

<span style="color:#000000"><code>import tempfile
model_dir = tempfile<span style="color:#009900">.mkdtemp</span>()
m = tf<span style="color:#009900">.contrib</span><span style="color:#009900">.learn</span><span style="color:#009900">.DNNLinearCombinedClassifier</span>(
    model_dir=model_dir,
    linear_feature_columns=wide_columns,
    dnn_feature_columns=deep_columns,
    dnn_hidden_units=[<span style="color:#006666">100</span>, <span style="color:#006666">50</span>])</code></span>

4.完整实现代码

<span style="color:#000000"><code><span style="color:#880000"># -*- coding: utf-8 -*-</span>

<span style="color:#000088">import</span> tensorflow <span style="color:#000088">as</span> tf
<span style="color:#000088">import</span> tempfile
<span style="color:#000088">import</span> pandas <span style="color:#000088">as</span> pd
<span style="color:#000088">import</span> urllib
<span style="color:#000088">import</span> numpy <span style="color:#000088">as</span> np
<span style="color:#000088">import</span> warnings

<span style="color:#000088">from</span> __future__ <span style="color:#000088">import</span> print_function

warnings.filterwarnings(<span style="color:#009900">"ignore"</span>)

<span style="color:#880000"># Categorical base columns.</span>
gender = tf.contrib.layers.sparse_column_with_keys(column_name=<span style="color:#009900">"gender"</span>, keys=[<span style="color:#009900">"Female"</span>, <span style="color:#009900">"Male"</span>])
race = tf.contrib.layers.sparse_column_with_keys(column_name=<span style="color:#009900">"race"</span>, keys=[<span style="color:#009900">"Amer-Indian-Eskimo"</span>, <span style="color:#009900">"Asian-Pac-Islander"</span>, <span style="color:#009900">"Black"</span>, <span style="color:#009900">"Other"</span>, <span style="color:#009900">"White"</span>])
education = tf.contrib.layers.sparse_column_with_hash_bucket(<span style="color:#009900">"education"</span>, hash_bucket_size=<span style="color:#006666">1000</span>)
relationship = tf.contrib.layers.sparse_column_with_hash_bucket(<span style="color:#009900">"relationship"</span>, hash_bucket_size=<span style="color:#006666">100</span>)
workclass = tf.contrib.layers.sparse_column_with_hash_bucket(<span style="color:#009900">"workclass"</span>, hash_bucket_size=<span style="color:#006666">100</span>)
occupation = tf.contrib.layers.sparse_column_with_hash_bucket(<span style="color:#009900">"occupation"</span>, hash_bucket_size=<span style="color:#006666">1000</span>)
native_country = tf.contrib.layers.sparse_column_with_hash_bucket(<span style="color:#009900">"native_country"</span>, hash_bucket_size=<span style="color:#006666">1000</span>)

<span style="color:#880000"># Continuous base columns.</span>
age = tf.contrib.layers.real_valued_column(<span style="color:#009900">"age"</span>)
age_buckets = tf.contrib.layers.bucketized_column(age, boundaries=[<span style="color:#006666">18</span>, <span style="color:#006666">25</span>, <span style="color:#006666">30</span>, <span style="color:#006666">35</span>, <span style="color:#006666">40</span>, <span style="color:#006666">45</span>, <span style="color:#006666">50</span>, <span style="color:#006666">55</span>, <span style="color:#006666">60</span>, <span style="color:#006666">65</span>])
education_num = tf.contrib.layers.real_valued_column(<span style="color:#009900">"education_num"</span>)
capital_gain = tf.contrib.layers.real_valued_column(<span style="color:#009900">"capital_gain"</span>)
capital_loss = tf.contrib.layers.real_valued_column(<span style="color:#009900">"capital_loss"</span>)
hours_per_week = tf.contrib.layers.real_valued_column(<span style="color:#009900">"hours_per_week"</span>)

wide_columns = [
  gender, native_country, education, occupation, workclass, relationship, age_buckets,
  tf.contrib.layers.crossed_column([education, occupation], hash_bucket_size=int(<span style="color:#006666">1e4</span>)),
  tf.contrib.layers.crossed_column([native_country, occupation], hash_bucket_size=int(<span style="color:#006666">1e4</span>)),
  tf.contrib.layers.crossed_column([age_buckets, education, occupation], hash_bucket_size=int(<span style="color:#006666">1e6</span>))]

deep_columns = [
  tf.contrib.layers.embedding_column(workclass, dimension=<span style="color:#006666">8</span>),
  tf.contrib.layers.embedding_column(education, dimension=<span style="color:#006666">8</span>),
  tf.contrib.layers.embedding_column(gender, dimension=<span style="color:#006666">8</span>),
  tf.contrib.layers.embedding_column(relationship, dimension=<span style="color:#006666">8</span>),
  tf.contrib.layers.embedding_column(native_country, dimension=<span style="color:#006666">8</span>),
  tf.contrib.layers.embedding_column(occupation, dimension=<span style="color:#006666">8</span>),
  age, education_num, capital_gain, capital_loss, hours_per_week]

model_dir = tempfile.mkdtemp()
m = tf.contrib.learn.DNNLinearCombinedClassifier(
    model_dir=model_dir,
    linear_feature_columns=wide_columns,
    dnn_feature_columns=deep_columns,
    dnn_hidden_units=[<span style="color:#006666">100</span>, <span style="color:#006666">50</span>])

<span style="color:#880000"># Define the column names for the data sets.</span>
COLUMNS = [<span style="color:#009900">"age"</span>, <span style="color:#009900">"workclass"</span>, <span style="color:#009900">"fnlwgt"</span>, <span style="color:#009900">"education"</span>, <span style="color:#009900">"education_num"</span>,
  <span style="color:#009900">"marital_status"</span>, <span style="color:#009900">"occupation"</span>, <span style="color:#009900">"relationship"</span>, <span style="color:#009900">"race"</span>, <span style="color:#009900">"gender"</span>,
  <span style="color:#009900">"capital_gain"</span>, <span style="color:#009900">"capital_loss"</span>, <span style="color:#009900">"hours_per_week"</span>, <span style="color:#009900">"native_country"</span>, <span style="color:#009900">"income_bracket"</span>]
LABEL_COLUMN = <span style="color:#009900">'label'</span>
CATEGORICAL_COLUMNS = [<span style="color:#009900">"workclass"</span>, <span style="color:#009900">"education"</span>, <span style="color:#009900">"marital_status"</span>, <span style="color:#009900">"occupation"</span>,
                       <span style="color:#009900">"relationship"</span>, <span style="color:#009900">"race"</span>, <span style="color:#009900">"gender"</span>, <span style="color:#009900">"native_country"</span>]
CONTINUOUS_COLUMNS = [<span style="color:#009900">"age"</span>, <span style="color:#009900">"education_num"</span>, <span style="color:#009900">"capital_gain"</span>, <span style="color:#009900">"capital_loss"</span>,
                      <span style="color:#009900">"hours_per_week"</span>]

<span style="color:#880000"># Download the training and test data to temporary files.</span>
<span style="color:#880000"># Alternatively, you can download them yourself and change train_file and</span>
<span style="color:#880000"># test_file to your own paths.</span>
train_file = tempfile.NamedTemporaryFile()
test_file = tempfile.NamedTemporaryFile()
urllib.urlretrieve(<span style="color:#009900">"http://mlr.cs.umass.edu/ml/machine-learning-databases/adult/adult.data"</span>, train_file.name)
urllib.urlretrieve(<span style="color:#009900">"http://mlr.cs.umass.edu/ml/machine-learning-databases/adult/adult.test"</span>, test_file.name)

<span style="color:#880000"># Read the training and test data sets into Pandas dataframe.</span>
df_train = pd.read_csv(train_file, names=COLUMNS, skipinitialspace=<span style="color:#000088">True</span>)
df_test = pd.read_csv(test_file, names=COLUMNS, skipinitialspace=<span style="color:#000088">True</span>, skiprows=<span style="color:#006666">1</span>)
df_train[LABEL_COLUMN] = (df_train[<span style="color:#009900">'income_bracket'</span>].apply(<span style="color:#000088">lambda</span> x: <span style="color:#009900">'>50K'</span> <span style="color:#000088">in</span> x)).astype(int)
df_test[LABEL_COLUMN] = (df_test[<span style="color:#009900">'income_bracket'</span>].apply(<span style="color:#000088">lambda</span> x: <span style="color:#009900">'>50K'</span> <span style="color:#000088">in</span> x)).astype(int)

<span style="color:#000088">def</span> <span style="color:#009900">input_fn</span><span style="color:#4f4f4f">(df)</span>:
  <span style="color:#880000"># Creates a dictionary mapping from each continuous feature column name (k) to</span>
  <span style="color:#880000"># the values of that column stored in a constant Tensor.</span>
  continuous_cols = {k: tf.constant(df[k].values)
                     <span style="color:#000088">for</span> k <span style="color:#000088">in</span> CONTINUOUS_COLUMNS}
  <span style="color:#880000"># Creates a dictionary mapping from each categorical feature column name (k)</span>
  <span style="color:#880000"># to the values of that column stored in a tf.SparseTensor.</span>
  categorical_cols = {k: tf.SparseTensor(
      indices=[[i, <span style="color:#006666">0</span>] <span style="color:#000088">for</span> i <span style="color:#000088">in</span> range(df[k].size)],
      values=df[k].values,
      dense_shape=[df[k].size, <span style="color:#006666">1</span>])
                      <span style="color:#000088">for</span> k <span style="color:#000088">in</span> CATEGORICAL_COLUMNS}
  <span style="color:#880000"># Merges the two dictionaries into one.</span>
  feature_cols = dict(continuous_cols.items() + categorical_cols.items())
  <span style="color:#880000"># Converts the label column into a constant Tensor.</span>
  label = tf.constant(df[LABEL_COLUMN].values)
  <span style="color:#880000"># Returns the feature columns and the label.</span>
  <span style="color:#000088">return</span> feature_cols, label

<span style="color:#000088">def</span> <span style="color:#009900">train_input_fn</span><span style="color:#4f4f4f">()</span>:
  <span style="color:#000088">return</span> input_fn(df_train)

<span style="color:#000088">def</span> <span style="color:#009900">eval_input_fn</span><span style="color:#4f4f4f">()</span>:
  <span style="color:#000088">return</span> input_fn(df_test)

print(<span style="color:#009900">'df_train shape:'</span>,np.array(df_train).shape)
print(<span style="color:#009900">'df_test shape:'</span>,np.array(df_test).shape)

m.fit(input_fn=train_input_fn, steps=<span style="color:#006666">200</span>)
results = m.evaluate(input_fn=eval_input_fn, steps=<span style="color:#006666">1</span>)
<span style="color:#000088">for</span> key <span style="color:#000088">in</span> sorted(results):
    print(<span style="color:#009900">"%s: %s"</span> % (key, results[key]))
</code></span>

参考文献
（1）《Wide & Deep Learning for Recommender Systems》
（2）https://www.tensorflow.org/tutorials/wide_and_deep
（3）https://www.tensorflow.org/api_docs/python/tf/contrib/learn/DNNLinearCombinedClassifier#fit
（4）https://research.googleblog.com/2016/06/wide-deep-learning-better-together-with.html

kingzone_2008

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
《Wide and Deep Learning for Recommender Systems》学习笔记

顾名思义，Google提出的这篇文章将Wide Model和Deep Model结合起来进行，思路非常值得学习。 1.Wide Model首先说一下Wide Model,就是上图中左边的那一部分，是一个简单的逻辑回归模型。这一部分比较简单，不多有一个新的思路就是交叉特征：论文中一个比较形象的例子For binary features, a cross...
复制链接

扫一扫

专栏目录