A Pareto-Efficient Algorithm for Multiple Objective Optimization in E-Commerce Recommendation阅读翻译

备注：大部分通过百度翻译的中文

ABSTRACT 摘要

Recommendation with multiple objectives is an important but difficult problem, where the coherent difficulty lies in the possible conflicts between objectives. In this case, multi-objective optimization is expected to be Pareto efficient, where no single objective can be further improved without hurting the others. However existing approaches to Pareto efficient multi-objective recommendation still lack good theoretical guarantees.
In this paper, we propose a general framework for generating Pareto efficient recommendations. Assuming that there are formal differentiable formulations for the objectives, we coordinate these objectives with a weighted aggregation. Then we propose a condition ensuring Pareto efficiency theoretically and a two-step Pareto efficient optimization algorithm. Meanwhile the algorithm can be easily adapted for Pareto Frontier generation and fair recommendation selection. We specifically apply the proposed framework on E-Commerce recommendation to optimize GMV and CTR simultaneously. Extensive online and offline experiments are conducted on the real-world E-Commerce recommender system and the results validate the Pareto efficiency of the framework.
To the best of our knowledge, this work is among the first to provide a Pareto efficient framework for multi-objective recommendation with theoretical guarantees. Moreover, the framework can be applied to any other objectives with differentiable formulations and any model with gradients, which shows its strong scalability.

多目标推荐是一个重要而困难的问题，其难点在于目标之间可能存在冲突。在这种情况下，我们期望多目标优化是帕累托有效的，即，其中没有一个目标可以在不伤害其他目标的情况下得到进一步的改进。然而，现有的帕累托有效多目标推荐方法仍然缺乏很好的理论保证。

本文提出了一个产生帕累托有效推荐的一般框架。假设目标有形式可微的公式，我们用加权集合来协调这些目标。然后从理论上提出了保证Pareto有效的条件和两步Pareto有效优化算法。同时，该算法可以很好地适应Pareto边界的生成和公平的推荐选择。我们把所提出的框架应用到电子商务推荐来同时优化GMV和CTR。在实际的电子商务推荐系统上进行了大量的在线和离线实验，实验结果验证了该框架的帕累托效率。

据我们所知，这项工作是第一个为多目标推荐提供帕累托有效框架并提供理论保证的工作。此外，该框架还可以应用于任何其他具有可微公式的目标和任何具有梯度的模型，具有很强的可扩展性。

1 INTRODUCTION 简介

Recommender systems are emerging as a crucial role in online services and platforms, which prevent users from information overload. The recommendation algorithms (for example Learning To Rank) generate personalized rankings of items and the top-ranked items are recommended to users. Usually, the algorithms need very careful designs to fulfill multiple objectives. However, it is difficult to optimize multiple objectives simultaneously, where the core difficulty lies in the conflicts between different objectives. In E-Commerce recommendation, CTR (Click Through Rate) and GMV (Gross Merchandise Volume) are two important objectives that are not entirely consistent. To validate this inconsistency, we collect one-week online data from a real-world E-Commerce platform and plot the trends of GMV when CTR . According to the trends reflected in Fig. 1, CTR is not entirely consistent with GMV , and a CTR-optimal or GMV-optimal recommendation can be rather sub-optimal or even bad in terms of the other objective.
推荐系统在网络服务和平台中扮演着重要的角色，它可以防止用户的信息过载。推荐算法（例如排序学习LTR）生成个性化的物品排名，排名靠前的物品推荐给用户。通常，算法需要非常仔细的设计来实现多个目标。但同时优化多个目标是困难的，其核心难点在于不同目标之间存在冲突。在电子商务推荐中，点击率CTR和成交总额GMV是两个不完全一致的重要目标。为了验证这种不一致性，我们从一个真实的电子商务平台收集了一周的在线数据，并绘制了按CTR排序时GMV的趋势图。根据图1所示的趋势，CTR与GMV并不完全一致，就另一个目标而言，按CTR最优或GMV最优进行推荐可能是相当次优的，甚至是不好的。

Therefore, a solution is considered as optimal for two objectives in the sense that no objective can be further improved without hurting the other one. This optimality is widely acknowledged in multiple objective optimization and named as Pareto efficiency or Pareto optimality. In the context of Pareto efficiency, solution A is considered to dominate solution B only when A outperforms B on all the objectives. And the aim of Pareto efficiency is to find solutions that are not dominated by any others.
因此，如果没有一个目标可以在不伤害另一个目标的情况下得到进一步的改进，那么这个解被认为是两个目标的最优解。这种最优性在多目标优化中得到了广泛的认可，被称为帕累托效率或帕累托最优性。在帕累托效率的背景下，只有当A在所有目标上都优于B时，才认为A方案优于B方案。帕累托效率的目标是找到最优的解决方案。

Existing approaches for Pareto optimization can be categorized into two categories: heuristic search and scalarization. Evolutionary algorithms are popular choices in heuristic search approaches. However, heuristic search can not guarantee Pareto efficiency, it only ensures the resulting solutions are not dominated by each other (but still can be dominated by the Pareto efficient solutions) [45]. Unlike heuristic search, scalarization transforms multiple objectives into a single one with a weighted sum of all the objective functions. With proper scalarization, the Pareto efficient solutions can be achieved by optimizing the reformulated objective function. However, the scalarization weights of objective functions are usually determined manually and Pareto efficiency is still not guaranteed. To summarize, it is very difficult for existing evolutionary algorithms and scalarization algorithms to find Pareto efficient solutions with a guarantee. Recently, it is pointed out that the Karush-Kuhn-Tucker (KKT) conditions can be used to guide the scalarization [11]. We build our algorithm upon the KKT conditions and propose a novel algorithmic framework that generates the scalarization weights with theoretical guarantees.
现有的帕累托优化方法可分为两类：启发式搜索和标量化。进化算法是启发式搜索方法中的常用选择。然而，启发式搜索不能保证Pareto有效性，它只能保证得到的解在搜索到的解中最优（但仍然可能不如Pareto有效解）[45]。与启发式搜索不同，标量化将多个目标转换为一个单一的目标，并对所有目标函数进行加权和。在适当的尺度化条件下，通过优化目标函数可以得到Pareto有效解。然而，目标函数的标量化权值通常由人工确定，帕累托效率仍然得不到保证。综上所述，现有的进化算法和标量化算法很难找到具有保证的帕累托有效解。最近，有人指出Karush-Kuhn-Tucker（KKT）条件可以用来指导标度化[11]。我们在KKT条件下建立了算法，并提出了一种新的算法框架，该框架在理论上保证了量化权重的生成。

Specifically, we propose a Pareto-Efficient algorithmic framework “PE-LTR” that optimizes multiple objectives with an LTR procedure. Given the candidate items generated for each user, PE-LTR ranks the candidates so that the ranking is Pareto efficient with respect to multiple objectives. Assuming that there exist differentiable formulations for each objective correspondingly, we adopt the scalarization technique to coordinate different objectives into a single objective function. As stated before, the scalarization technique can not guarantee Pareto efficiency unless the weights are carefully chosen. Therefore, we first propose a condition for the scalarization weights that ensures the solution is Pareto efficient. The condition is equivalent to a constrained optimization problem, and we propose an algorithm that solves the problem in two steps. First we simplify the problem by relaxing the constraints so that an analytic solution is achieved; then we get the feasible solution by conducting a projection procedure. With PE-LTR as the cornerstone, we provide methods to generate the Pareto Frontier and a specific recommendation, depending on the needs of service providers. To generate the Pareto Frontier, one can run PE-LTR by evenly set the bounds of the objective scalarization weights. To generate a specific recommendation, one can either run PE-LTR once with proper bounds or generate the Pareto Frontier first and choose a “fair” solution with specific fairness metric.
具体地说，我们提出了一个帕累托有效的算法框架“PE-LTR”，用LTR过程优化多个目标。给定为每个用户生成的候选项，PE-LTR对候选项进行排序，以便对多个目标进行帕累托有效排序。假设每个目标都有相应的可微公式，采用标量化技术将不同目标协调成一个目标函数。如前所述，除非仔细选择权重，否则标量化技术不能保证帕累托效率。因此，我们首先提出了一个可确保解是帕累托有效的尺度化权重的条件。该条件等价于一个约束优化问题，提出了一种分两步求解的算法。首先通过放松约束使问题简化，得到解析解；然后通过投影过程得到可行解。以PE-LTR为基础，根据服务提供商的需求，我们提供了生成Pareto边界的方法和具体的建议。要生成Pareto边界，可以通过均匀设置目标尺度化权重的边界来运行PE-LTR。为了产生一个特定的推荐，他们可以使用适当的边界运行一次PE-LTR，或者首先生成Pareto边界，然后选择一个具有特定公平度量的“公平”解决方案。

In this paper we apply this framework to optimize two important objectives for E-Commerce recommendation, i.e. GMV and CTR. For E-Commerce platforms, the primary objective is to improve the GMV, but too much sacrifice of CTR may cause a severe decrease of daily active users (DAU) in the long term. Therefore we aim to find Pareto efficient solutions with respect to both objectives. We propose two differentiable formulations for GMV and CTR respectively and apply the PE-LTR framework for generating Pareto-optimal solutions. We conduct extensive experiments on a real-world E-Commerce recommender system and compare the results with state-of-the-art approaches. The online and offline experimental results both indicate that our solution outperforms other baselines significantly and the solutions are nearly Pareto efficient.
本文应用该框架对电子商务推荐的两个重要目标GMV和CTR进行了优化。对于电子商务平台来说，首要的目标是提高GMV，但长期来看，过度舍弃CTR可能会导致日常活跃用户（DAU）的严重减少。因此，我们的目的是找到关于这两个目标的Pareto有效解。我们分别提出了GMV和CTR的两个可微公式，并应用PE-LTR框架生成帕累托最优解。我们在一个真实的电子商务推荐系统上进行了大量的实验，并将结果与最新的方法进行了比较。离线和在线实验结果均表明，我们的解明显优于其他基线，且解接近帕累托效率。

The contributions of this work are:

• We propose a general Pareto efficient algorithmic framework
  (PE-LTR) for multi-objective recommendation. The framework is both model and objective agnostic, which shows its great scalability.
• We propose a two-step algorithm which theoretically guarantees the Pareto efficiency. Despite the algorithm is built upon scalarization technique, it differs from other scalarization approaches with its theoretical guarantee and its automatic learning of scalarization weights rather than manually assignment.
• With PE-LTR as the cornerstone, we present how to generate the Pareto Frontier and a specific recommendation. Specifically, we propose to select a fair recommendation from the Pareto Frontier with proper fairness metrics.
• We use E-Commerce recommendation as a specification of PE-LTR, and conduct extensive online and offline experiments on a real-world recommender system. The results indicate that our algorithm outperforms other state-of-the-art approaches significantly and the solutions generated are Pareto efficient.
• We open-source a large-scale E-Commerce recommendation dataset EC-REC, which contains the real records of impressions, clicks and purchases. To the best of our knowledge, no public dataset includes all three labels and enough features, this dataset can be used for further studies.
  这项工作的贡献是：
• 我们提出了一个通用的帕累托有效算法框架（PE-LTR）用于多目标推荐。该框架既不依赖于模型，也不依赖于目标，显示了其强大的可伸缩性。
• 我们提出一个两步算法，从理论上保证帕累托效率。尽管该算法是建立在标量化技术的基础上的，但它不同于其他标量化方法，它具有理论保证和标量化权值的自动学习，而不是手工赋值。
• 以PE-LTR为基础，我们介绍了如何生成帕累托前沿和具体推荐。具体来说，我们提出从帕累托边界中选择一个公平的推荐，并使用适当的公平度量。
• 我们使用电子商务推荐作为PE-LTR的规范，并在现实世界的推荐系统上进行广泛的在线和离线实验。结果表明，我们的算法明显优于其他最新方法，并且生成的解是帕累托有效的。
• 我们开源了一个大型电子商务推荐数据集EC-REC，其中包含展现、点击和购买的真实记录。据我们所知，这是第一个包含所有三个标签和足够特征的公共数据集，这个数据集可以用于进一步的研究。

2 RELATED WORK 相关工作

In this section, we provide a detailed introduction to the related studies from the following aspects: recommendation with multiple objectives, E-Commerce recommendation and learning to rank.
本部分从多目标推荐、电子商务推荐、排序学习等方面对相关研究进行了详细介绍。

2.1 Recommendation with Multiple Objectives 多目标推荐

We look at the studies on multi-objective recommendation from two aspects, i.e. the objectives concerned and the approaches for multi-objective recommendations.
Despite the recommendation accuracy is the main concern, some studies argue that other characteristics such as the availability, profitability, or usefulness should be considered simultaneously [15, 22]. Some studies attempt to model the trade-off s between relevance and diversity in recommendation [14, 17, 41]. When multiple objectives are concerned, it is expected to get a Pareto efficient recommendation [27, 28]. Recently, it is pointed out that some multiple objectives are related to users [7, 16, 23, 29]. On one hand, different objectives are related to different user behaviors. For example, both clicks and hides are considered in LinkedIn feeds [33]. On the other hand, the objectives are related to different user statuses, for example different stakeholders [8, 23].
本文从多目标推荐的目标和方法两个方面对多目标推荐进行了研究。

尽管推荐的准确性是主要问题，但一些研究认为，其他特征，如可用性、盈利能力或有用性，应同时考虑[15，22]。一些研究试图在推荐[14、17、41]中对相关性和多样性之间的权衡进行建模。当涉及多个目标时，期望得到帕累托有效的推荐[27，28]。最近有人指出，有些多目标与用户有关[7、16、23、29]。一方面，不同的目标与不同的用户行为有关。例如，在LinkedIn订阅源[33]中，点击和隐藏都被考虑在内。另一方面，目标与不同的用户状态相关，例如不同的涉众[8，23]。

The approaches on recommendation with multiple objectives can be categorized into evolutionary algorithm [45] and scalarization [38]. The evolutionary algorithm has been used for long- tail recommendation [35], diversified recommendation [10], and novelty-aware recommendation [28]. And it has also been used for Pareto efficient hybridization [28] of multiple recommendation algorithms. Scalarization technique is also used for recommendation with multiple objectives [38]. However, existing studies mostly depend on manually assigned weights for scalarization, whose Pareto efficiency can not be guaranteed. Recently, the KKT conditions are used for guiding scalarization techniques [11, 32]. However, existing algorithms based on these conditions are limited to the unconstrained cases and can not t the requirements in real-world scenarios.
多目标推荐方法可分为进化算法[45]和标量化[38]。进化算法已用于长尾推荐[35]、多样化推荐[10]和新颖性感知推荐[28]。它也被用于多重推荐算法的帕累托有效混合[28]。标量化技术也用于多目标推荐[38]。然而，现有的研究大多是针对手工赋权的标量化问题，其帕累托效率不能得到保证。最近，KKT条件被用来指导标量化技术[11，32]。然而，现有的基于这些条件的算法仅限于无约束情况，不能满足真实场景中的要求。

2.2 E-Commerce Recommendation 电子商务推荐

E-Commerce recommendation is also a popular research topic. Some studies adopt economic theory models and Markov chains for recommendation [12, 19, 42, 43]. While some other studies focus on other aspects in E-Commerce recommendation [1, 30, 34, 40], such as feature learning and diversification. It is pointed out that a good practice in E-Commerce searching is learning to rank [18], which also coincides with the motivation of our framework. Usually there are multiple stages in E-Commerce recommendation, for example clicks and purchases. Therefore the learning-to-rank algorithms need to jointly optimize multiple stages [36]. Some studies focus on the post-click stage in searching and recommendation. For example, the bidding price and revenue are jointly considered with relevance [26, 44]. Recently, two studies focus on the connection between clicks and purchases in E-Commerce searching and advertising [21, 36]. As optimizing clicks and purchases are not entirely consistent, it is necessary to find a Pareto efficient trade-off between them, which is not considered in previous studies on purchase optimization [21, 36].
电子商务推荐也是一个热门的研究课题。一些研究侧重于推荐中的经济理论模型和马尔可夫链[12，19，42，43]。而其他一些研究则侧重于电子商务推荐中的其他方面，如特征学习和多样化。电子商务搜索的一个良好实践是排序学习[18]，这也符合我们框架的动机。电子商务推荐通常有多个阶段，如点击和购买。因此，排序学习算法需要联合优化多个阶段[36]。一些研究集中在搜索和推荐中的点击后阶段。例如，投标价格和收入与相关性一起考虑[26，44]。最近，有两项研究关注电子商务搜索和广告中点击和购买之间的关系[21，36]。由于优化点击和购买并不完全一致，因此有必要在它们之间找到帕累托有效的折衷，这在以往的购买优化研究中是没有考虑到的[21，36]。

2.3 Learning to Rank 排序学习

Learning To Rank (LTR) has been a popular research topic for quite a long time. The studies on LTR can be categorized into point-wise, pair-wise and list-wise approaches. The point-wise scheme [20] predicts the individual instance separately; the pair-wise scheme [4, 13] is approximated as a binary classification problem, which focuses on the relative order of a pair of instances; while the list-wise scheme [5, 6, 37, 39] directly optimizes the metric of a ranking list. Usually, list-wise LTR achieves superior performances than other schemes.
ranking methods have been proposed, such as RankNet [4], Rank- Boost [13], AdaRank [39], LambdaRank [5], ListNet [6] and LambdaMART [37]. Due to the similarity between searching and recommendation in ranking, LTR approaches are widely used in both scenarios. Recently, it is pointed out that LTR is a key component in E-Commerce searching [18], which is able to exploit multiple user feedback signals for relevance modeling, including clicks, add-to-cart ratios, and revenue.
According to the previous studies, LambdaMART is one of the best performing algorithms [36]. As focus of this paper is not about ranking model, we choose a simple point-wise ranking model for the proposed framework.
长期以来，排序学习一直是一个热门的研究课题。LTR的研究可分为point-wise、pair-wise和list-wise。point-wise方案[20]分别预测个体实例；pair-wise方案[4, 13]被近似为二分类问题，其集中于一对实例的相对顺序；而list-wise方案[5, 6, 37，39]直接优化排序列表的度量。通常，list-wise LTR比其他方案具有更好的性能。

目前已有多种排序方法，如RankNet[4]、Rank-Boost[13]、AdaRank[39]、LambdaRank[5]、ListNet[6]和LambdaMART[37]。由于搜索和推荐排序的相似性，LTR方法在这两种情况下都得到了广泛的应用。最近，有人指出LTR是电子商务搜索中的一个关键组件[18]，它能够利用多个用户反馈信号进行相关性建模，包括点击率、购物车添加比率和收入。

根据之前的研究，LambdaMART是性能最好的算法之一[36]。由于本文的研究重点不是排序模型，因此我们选择了一个简单的point-wise排序模型作为框架。

3 PROPOSED FRAMEWORK 提出的框架

In this section, we first provide a brief introduction to the concept of Pareto efficiency. Then we introduce the details of the proposed framework, i.e. Pareto-Efficient Learning-to-Rank (PE-LTR). Assuming that there are differentiable loss functions for multiple objectives correspondingly, we propose a condition that guarantees the Pareto efficiency of the solution. We show that the proposed condition is equivalent to a constrained Quadratic Programming problem. Then we propose a two-step algorithm to solve this problem. Moreover, we provide methods to generate both Pareto Frontier and specific single recommendation with PE-LTR.
在这一节中，我们首先简要介绍帕累托效率的概念。然后详细介绍了该框架，即Pareto有效学习排序（PE-LTR）。假设多目标对应有可微损失函数，我们提出了保证解的帕累托效率的条件。证明了该条件等价于一个约束二次规划问题。然后我们提出了一个两步算法来解决这个问题。此外，我们还提供了利用PE-LTR生成Pareto前沿和特定单一推荐的方法。

3.1 Preliminary 前言

First, we provide a brief introduction to Pareto efficiency and some related concepts. Pareto efficiency is an important concept in multiple objective optimization. Given a system which aims to minimize a series of objective functions $f_1,...,f_K$ , Pareto efficiency is a state when it is impossible to improve one objective without hurting other objectives in terms of multi-objective optimization.
首先，我们简要介绍了帕累托效率和相关概念。帕累托效率是多目标优化中的一个重要概念。假定一个系统的目标是最小化一系列目标函数 $f_1,...,f_K$，Pareto效率是指在多目标优化中，一个目标不可能在不影响其他目标的情况下得到改善的状态。

Definition 3.1. Denote the outcomes of two solutions as $s_i=(f_1^i,...,f_K^i)$ and $s_j=(f_1^j,...,f_K^j)$, $s_i$ dominates $s_j$ if and only if $f_1^i\le f_1^j,f_2^i\le f_2^j,...,f_K^i\le f_K^j$ (for minimization objectives).
The concept of Pareto efficiency is built upon the definition of domination:
Definition 3.2. A solution $s_i=(f_1^i,...,f_K^i)$ is Pareto efficient if there is no other solution $sj=(f_1^j,...,f_K^j)$ that dominates $s_i$.
Therefore, a solution that is not Pareto efficient can still be improved for at least one objective without hurting the others, and it is always expected to achieve Pareto efficient solutions in multi-objective optimization. It is worth mentioning that Pareto efficient solutions are not unique and the set of all such solutions is named as the “Pareto Frontier”
定义3.1。将两个方案的结果表示为$s_i=(f_1^i,...,f_K^i)$和$s_j=(f_1^j,...,f_K^j)$，当且仅当$f_1^i\le f_1^j,f_2^i\le f_2^j,...,f_K^i\le f_K^j$（用于最小化目标）时，$s_i$ 支配 $s_j$。

帕累托效率的概念建立在支配的定义之上：

定义3.2。一个解$s_i=(f_1^i,...,f_K^i)$是帕累托有效的，如果没有其他解决方案$s_j=(f_1^j,...,f_K^j)$支配$s_i$。

因此，在多目标优化中，一个非帕累托有效解仍然可以在不损害其他目标的情况下至少对一个目标进行改进，并且总是期望能得到帕累托有效解。值得一提的是，Pareto有效解并不是唯一的，所有这些解的集合称为"帕累托边界"。

3.2 Pareto-Efficient Learning to Rank 帕累托有效的排序学习

To achieve a Pareto efficient solution, we propose a Learning-to-Rank scheme that optimizes multiple objectives with the scalarization technique. Assuming that there are K objectives in a given recommender system, a model F(θ) needs to optimize these objectives simultaneously, where θ denotes the model parameters. Without loss of generality, we assume that there exist K differentiable loss functions $L_i(\theta ),\forall i\in 1,...,K$ for the K objectives correspondingly.
为了得到帕累托有效解，我们提出了一个排序学习的方案，利用标量化技术优化多个目标。假设在给定的推荐系统中存在K个目标，则F(θ)模型需要同时优化这些目标，其中θ表示模型参数。在不损失一般性的情况下，我们假设K个目标存在K个可微损失函数$L_i(\theta ),\forall i\in 1,...,K$。

Given the formulations, optimizing i-th objective is equal to minimizing $L_i$ . However, optimizing these K objectives simultaneously is non-trivial, since the optimal solution to one objective is usually sub-optimal for another one. Therefore, we use the scalarization technique to merge multiple objectives into a single one. Specifically, we aggregate the loss functions $L_i$ with ${\omega }_i,\forall i\in 1,...,K$:

$L(\theta )={\sum }_{i=1}^K{w}_i{L}_i(\theta )$

where ${\sum }_{i=1}^K{w}_i=1$ and