2021-09-27论文

Deep reinforcement learning based computation offloading and resource allocation for MEC

We formulated the computation offloading decision and MEC computation resource allocation problems in this framework
state: the sum cost of the entire system and the available computational capacity of the MEC server
action: the offloading decision and resource allocation
reward: negatively correlated to the size of the sum cost,
本文提到的考虑多用户解决问题，好像是在MDP建模的时候考虑了多个元素

Deep Reinforcement Learning for Offloading and Resource Allocation in Vehicle Edge Computing and Networks

服务器作为agent 使用DQN算法

proposed the offloading policies and the resource allocation scheme
the vehicles can provide computation services for UEs as well as the traditional edge server
We propose an efficient offloading scheme for the vehicle edge computing network while considering both delay and limited computation capabilities of vehicles and edge servers. Accordingly, we formulate an optimization problem to maximize the total utility of the vehicle edge computing network.
we reformulated the proposed problem as a semi-Markov process and propose Q-learning based reinforcement learning method

state: 由FES(fixed edge server) n提供服务的UE i的数据速率和计算能力，由VES(vehicle edge server) n提供服务的UE i的数据速率和计算能力
action: 是否卸载，卸载到FES，卸载到VES，FES和VES的通信和计算资源的分配
服务器作为agent，

Integrated Networking, Caching, and Computing for Connected Vehicles: A Deep Reinforcement Learning Approach

服务器MVNO虚拟网络运营商（Mobile Virtual Network Operator)作为agent Double-Dueling-DeepQ-network algorithm 没有看到用dueling

In this paper, we propose an integrated framework that can enable dynamic orchestration of networking, caching, and computing resources to improve the performance of next generation vehicular networks. We formulate the resource allocation strategy in this framework as a joint optimization problem, where the gains of not only networking but also caching and computing are taken into consideration in the proposed framework.
state: 信道状态，计算能力在不同时刻被看作是不同的，是否会缓存，这三个变量都是一个数组
action: 哪个BS分配给车辆，请求内容是否缓存在BS上，计算任务是否卸载到MEC上
The basic principle of SDN is to separate the control plane from the data plane, enabling the ability of programming the network via a centralized software-defined controller with a global view of the network. Software defined and virtualized vehicular networks enable direct programmability of vehicular network controls and abstraction of the underlying infrastructure for a variety of applications of connected vehicles, with improved efficiency and great flexibility in vehicular network management
本文提出的BS是虚拟的，当有计算任务时，它先发送给BS，如果BS中有相对就该请求的缓存内容，则BS把返回结果传给车辆，否则会把任务信息发送给MEC中进行处理。

Joint Computing and Caching in 5G-Envisioned Internet of V ehicles: A Deep Reinforcement Learning-Based Traffic Control System

服务器作为agent 使用DDPG

state: The vehicular status setDi(t)includes vehicle’s location, velocity, the total size of computing tasks and requested contents, the popularity of requested contents, the size of residual contents, remained computing tasks and requiredcomputation resources, and required CPU cycles for the task computing. In addition,Fj(t),Gj(t)andBj(t)denote the available resources of computation, caching and bandwidth of each MEC server, respectively.
action: the amount of computational resource, caching resource and bandwidth that MEC serverjallocates to vehiclei,
动作是连续值，所以用DDPG