[参文]bike_rebalance_vehicle（二）

从论文《A Deep Reinforcement Learning Framework for Rebalancing Dockless Bike Sharing Systems》中获取的参考文献。

一、基于车辆的方法

1.1Data analysis and optimization for (citi) bike sharing.

摘要：
自行车共享系统在城市环境中越来越普遍。它们为城市提供了一种低成本、环保的交通选择。这些系统的管理带来了许多优化问题。这些问题中最主要的是自行车再平衡问题。用户以不对称模式创造需求，从而使系统失衡。这就需要采取行动，使系统与每个车站所需的自行车数量保持平衡，以便于将来使用。在本文中，我们解决的问题是在高峰时段维持系统平衡，以及夜间重新平衡，为高峰时段的使用做好准备。我们提供了新颖的问题公式，其动机是与纽约市自行车共享中心（Citibike）的密切合作，以及对系统使用数据的仔细分析。我们分析系统数据，找出自行车的最佳位置，以方便使用。我们解决夜班的路线问题，以及处理高峰时段使用的集群问题。根据这项研究开发的工具目前在Citibike的运营商纽约自行车共享有限责任公司日常使用。
abtract:
Bike-sharing systems are becoming increasingly prevalent in urban environments. They provide a low-cost, environmentally-friendly transportation alternative for cities. The management of these systems gives rise to many optimization problems. Chief among these problems is the issue of bicycle rebalancing. Users imbalance the system by creating demand in an asymmetric pattern. This necessitates action to put the system back in balance with the requisite levels of bicycles at each station to facilitate future use. In this paper, we tackle the problem of maintaing system balance during peak rush-hour usageas well as rebalancing overnight to prepare the systemfor rush-hour usage. We provide novel problem formulationsthat have been motivated by both a close collaborationwith the New York City bike share (Citibike) and a careful analysisof system usage data. We analyze system data to discover the best placement of bikes tofacilitate usage. We solve routing problems forovernight shifts as well as clustering problems for handlingmid rush-hour usage. The tools developed from this research are currently in daily use at NYC Bike Share LLC, operators of Citibike.

bib:

@article{2015Data,
  title={Data Analysis and Optimization for (Citi)Bike Sharing},
  author={ O'Mahony, E.  and  Shmoys, D. B. },
    booktitle={AAAI},
  year={2015},
}

1.2 Dynamic bike reposition: A spatio-temporal reinforcement learning approach.

解析链接：https://www.dazhuanlan.com/kuangzi/topics/1115189
ACM链接
pdf链接

摘要：
共享单车系统在许多主要城市广泛部署，而其中的站点拥堵和空置导致严重的客户流失。目前，运营商尝试在系统运行时不断在站点之间重新定位自行车。然而，如何有效地重新定位以最大限度地减少长期客户流失仍然没有解决。我们提出了一种基于时空强化学习的自行车重新定位模型来解决这个问题。首先，提出了一种相互独立的内平衡聚类算法，将站点聚类。获得的集群有两个属性，即每个集群都是内部平衡的并且相互独立。由于在一个非常大的系统中同时有许多三轮车重新定位，因此需要聚类以降低问题的复杂性。其次，我们为每个集群分配多辆三轮车进行集群内自行车重新定位。为每个集群设计了一个时空强化学习模型，以学习其中的重新定位策略，旨在最大限度地减少其长期客户流失。为了学习每个模型，我们设计了一个深度神经网络来估计其最优长期价值函数，从中可以很容易地推断出最优策略。除了以多代理方式构建模型外，我们还通过两个时空修剪规则进一步降低了其训练复杂度。第三，我们设计了一个基于两个预测器的系统模拟器来训练和评估重新定位模型。对来自 Citi Bike 的真实世界数据集进行了实验，以确认我们模型的有效性。
abstract:
Bike-sharing systems are widely deployed in many major cities, while the jammed and empty stations in them lead to severe customer loss. Currently, operators try to constantly reposition bikes among stations when the system is operating. However, how to effectively reposition to minimize the customer loss in a long period remains unsolved. We propose a spatio-temporal reinforcement learning based bike reposition model to deal with this problem. Firstly, an inter-independent inner-balance clustering algorithm is proposed to cluster stations into groups. Clusters obtained have two properties, i.e. each cluster is innerbalanced and independent from the others. As there are many trikes repositioning in a very large system simultaneously, clustering is necessary to reduce the problem complexity. Secondly, we allocate multiple trikes to each cluster to conduct inner-cluster bike reposition. A spatio-temporal reinforcement learning model is designed for each cluster to learn a reposition policy in it, targeting at minimizing its customer loss in a long period. To learn each model, we design a deep neural network to estimate its optimal long-term value function, from which the optimal policy can be easily inferred. Besides formulating the model in a multi-agent way, we further reduce its training complexity by two spatio-temporal pruning rules. Thirdly, we design a system simulator based on two predictors to train and evaluate the reposition model. Experiments on real-world datasets from Citi Bike are conducted to confirm the effectiveness of our model.

bib：

@inproceedings{li2018dynamic,
author = {Li, Yexin and Zheng, Yu and Yang, Qiang},
title = {Dynamic Bike Reposition: A Spatio-Temporal Reinforcement Learning Approach},
year = {2018},
isbn = {9781450355520},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3219819.3220110},
doi = {10.1145/3219819.3220110},
booktitle = {Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining},
pages = {1724–1733},
numpages = {10},
keywords = {reinforcement learning, dynamic bike reposition, bike-sharing system},
location = {London, United Kingdom},
series = {KDD '18}
}

1.3Robust Repositioning to Counter Unpredictable Demand in Bike Sharing Systems

摘要：
自行车共享系统（BSS）由于饥饿（基站空置，无法取回自行车）或拥堵（基站满，无法归还自行车），客户需求大幅下降。因此，BSSs运营商在运输工具的帮助下，在车站之间重新定位自行车。由于需求的不可预测和动态变化的性质，目光短浅的推理通常产生低于标准数据的性能。我们提出了一种在线且稳健的重新定位方法，以最大限度地减少客户需求的损失，同时考虑未来需求中可能存在的不确定性目标。具体地说，我们开发了一种基于迭代两人博弈的场景生成方法，通过假设环境可以针对当前的重新定位解决方案生成更糟糕的需求场景（在可行的需求场景之外），来计算重新定位策略 假设的条件。基于自行车共享公司真实数据集的大量模拟计算结果表明，与现有的基准方法相比，我们的方法可以显著减少预期的需求损失。
abstract:
Bike Sharing Systems (BSSs) experience a significant loss in customer demand due to starvation (empty base stations precluding bike pickup) or congestion (full base stations precluding bike return). Therefore, BSSs operators reposition bikes between stations with the help of carrier vehicles. Due to unpredictable and dynamically changing nature of the demand, myopic reasoning typically provides a below par performance. We propose an online and robust repositioning approach to minimise the loss in customer demand while considering the possible uncertainty in future demand. Specifically, we develop a scenario generation approach based on an iterative two player game to compute a strategy of repositioning by assuming that the environment can generate a worse demand scenario (out of the feasible demand scenarios) against the current repositioning solution. Extensive computational results from a simulation built on real world data set of bike sharing company demonstrate that our approach can significantly reduce the expected lost demand over the existing benchmark approaches.

链接
bib:

@inproceedings{2016Robust,
author = {Ghosh, Supriyo and Trick, Michael and Varakantham, Pradeep},
title = {Robust Repositioning to Counter Unpredictable Demand in Bik