Recurrent Recommender Networks

流行，非参数估计，梯度消失，长短记忆模型，概率矩阵分解

http://dl.acm.org/citation.cfm?doid=3018661.3018689

ABSTRACT

Recommender systems traditionally assume that user pro-files and movie attributes are static. Temporal dynamics arepurely reactive, that is, they are inferred after they are observed,e.g. after a user’s taste has changed or based on handengineeredtemporal bias corrections for movies. We proposeRecurrent Recommender Networks (RRN) that are able topredict future behavioral trajectories. This is achieved byendowing both users and movies with a Long Short-Term Memory (LSTM)autoregressive model that capturesdynamics, in addition to a more traditional low-rank factorization.On multiple real-world datasets, our model offersexcellent prediction accuracy and it is very compact, sincewe need not learn latent state but rather just the state transitionfunction.

1. INTRODUCTION

Given the overall structure of the model, we are at liberty to posit a specific type of state for the latent variable. Popular choices are to assume a discrete latent state, as used e.g. when modeling web browsing behavior [9]. Likewise, we could resort to spectral methods [22, 1] or nonparametric estimators, such as Recurrent Neural Networks (RNNs). To address the vanishing gradient problem we resort to Long Short-Term Memory [14] in our model. Our contributions are as follows:

1.Nonlinear nonparametric recommender systems have proven to be somewhat elusive. In particular, nonlinear substitutes of the inner product formulation showed only limited promise in our experiments. To the best of our knowledge this is the first paper addressing movie recommendation in a fully causal and integrated fashion. That is, we believe that this is the first model which attempts to capture the dynamics of both users and movies. Moreover, our model is nonparametric. This allows us to model the data rather than having to assume a specific form of a state space.

2.Recurrent Recommender Networks are very concise since we only learn the dynamics rather than the state. This is one of the key differences to typical latent variable models where considerable effort is spent on estimating the latent state. In this aspect it closely resembles the neural autoencoder for recommendation of [21]. In fact, one could view the latter as a special case of our model.

2. RELATED WORK

2.1 Recommender Systems
Probably one of the most popular variants is Probabilistic Matrix Factorization (PMF)

2.2 Recurrent Deep Networks

One of the key challenges in a graphical model described in Figure 3 is that it requires us to infer future states given observations, e.g. via message passing or particle filtering. This is costly and fraught with difficulty, since we need to match the emission model (of ratings) to the latent state. In
other words, the only way to feed information about ratings back into the latent state is via the likelihood p(r_ij |u_it, m_jt). Common strategies for achieving this are via message passing
or via particle filtering, i.e. via Sequential Monte Carlo sampling [2]. They are approximate and can be nontrivial to implement accurately at scale. Alternatively we can simply learn the mapping as part of a nonparametric state update, i.e. via a Recurrent Neural Network (RNN). See [12] for an extensive overview and tutorial. The key idea is to use a latent variable autoregressive model as follows:


They control how information flows through the sequence. For simplicity, in the following sections we use ht = LSTM(h_t−1, zt) to denote these operations. Note that LSTMs are not the only option. For instance, [7] propose the use of a Gated Recurrent Unit (GRU). It is very similar in spirit, albeit computationally cheaper and often similar in its results. For the purpose of the current paper we limit ourselves to LSTMs since they are slightly more general. [13] propose to use RNN for session-based recommendation, but it does not consider personalization, nor does it attempt to model user or item state transitions.

3. MODEL

…待整理