文献笔记 - Application of Task Allocation Algorithms in Multi-UAV Intelligent Transportation Systems

Unmanned aerial vehicles (UAVs), commonly known as drones, are being seen as the most promising type of autonomous vehicles in the context of intelligent transportation system (ITS) technology. A key enabling factor for the current development of ITS technology based on autonomous vehicles is the task allocation architecture. This approach allows tasks to be efficiently assigned to robots of a multi-agent system, taking into account both the robots’ capabilities and service requirements. Consequently, this study provides an overview of the application of drones in ITSs, focusing on the applications of task allocation algorithms for UAV networks. Currently, there are different types of algorithms that are employed for task allocation in drone-based intelli gent transportation systems, including market-based approaches, game-theory-based algorithms, optimization-based algorithms, machine learning techniques, and other hybrid methodologies. This paper offers a comprehensive literature review of how such approaches are being utilized to optimize the allocation of tasks in UAV-based ITSs. The main characteristics, constraints, and limitations are detailed to highlight their advantages, current achievements, and applicability to different types of UAV-based ITSs.

无人驾驶飞行器（UAV），通常被称为无人机，被视为智能交通系统（ITS）技术背景下最有前景的自动驾驶汽车类型。当前基于自动驾驶汽车的智能交通系统技术发展的一个关键推动因素是任务分配架构。这种方法允许将任务有效地分配给多智能体系统的机器人，同时考虑到机器人的能力和服务要求。因此，本研究概述了无人机在ITSs中的应用，重点介绍了任务分配算法在无人机网络中的应用。目前，在基于无人机的智能运输系统中，有不同类型的算法用于任务分配，包括基于市场的方法、基于博弈论的算法、基于优化的算法、机器学习技术和其他混合方法。本文对如何利用这些方法优化基于无人机的信息技术服务中的任务分配进行了全面的文献综述。详细介绍了其主要特征、约束和局限性，以突出其优势、当前成就以及对不同类型无人机智能交通系统的适用性。

1. Introduction

With the advancement of the Industry 5.0 paradigm, intelligent transportation system (ITS) technology is poised to enter the next stage: Transportation 5.0 will be dedicated to solving urban transportation challenges through intelligent technology and a multitude of autonomous robots, thereby enhancing the efficiency and safety of transportation sys tems [1]. Currently, unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs), i.e., drones, are being extensively utilized in ITSs as the two most promising types of autonomous vehicles [2]. Consequently, this study provides an overview of the appli cation of drones in ITSs, with a particular focus on reviewing task allocation algorithms for UAVs.

随着工业5.0范式的推进，智能交通系统（ITS）技术即将进入下一阶段：交通5.0将致力于通过智能技术和大量自主机器人解决城市交通挑战，从而提高交通系统的效率和安全性[1]。目前，无人地面车辆（UGV）和无人驾驶飞行器（UAV），即无人机，作为两种最有前景的自动驾驶车辆，在信息技术服务中得到了广泛的应用[2]。因此，本研究概述了无人机在ITSs中的应用，特别侧重于审查无人机的任务分配算法。

UAVs are enhancing the level of automation in the context of intelligent transportation systems because they can either operate as an “eye in the sky” to support road transporta tion vehicles or carry out tasks in complete autonomy, thereby fully revolutionizing the concept of transportation service itself. For instance, in a smart city context, a multi-UAV network equipped with cameras can cooperate to report accidents to a mobility service center, thus reducing the accident response time [3]. Another example is the usage of UAVs to carry out parcel delivery tasks in traffic congested areas or difficult-to-reach areas, which results in the reduction in delivery times and traffic congestion due to delivery vans. A scalable and efficient task allocation architecture represents a crucial enabling factor for optimally coordinating the fleet of drones of the ITS, allowing the fleet to be strategically assigned to the tasks, improving the overall efficiency of the system and reducing costs. The scalability of the allocation architecture with respect to the number of agents and tasks is also a crucial factor for the deployment of large-scale UAV networks.

无人机正在提高智能交通系统的自动化水平，因为它们既可以作为“空中之眼”来支持道路运输车辆，也可以完全自主地执行任务，从而彻底改变了运输服务本身的概念。例如，在智慧城市环境中，配备摄像头的多无人机网络可以合作向移动服务中心报告事故，从而缩短事故响应时间[3]。另一个例子是使用无人机在交通拥堵地区或难以到达的地区执行包裹递送任务，这导致了递送时间的减少和送货车造成的交通拥堵。可扩展且高效的任务分配架构是优化协调智能交通系统无人机机队的关键因素，使机队能够战略性地分配任务，提高系统的整体效率并降低成本。分配架构在代理和任务数量方面的可扩展性也是部署大规模无人机网络的关键因素。

Generally, task allocation aims to minimize the execution time of tasks, maximize the number of completed tasks, and increase the reliability of the task allocation procedure. Presently, the applications of UAVs in ITSs primarily include delivery [4], communica tion [5], search and rescue, traffic monitoring, and data collection [6], to name a few.

一般来说，任务分配的目的是尽量减少任务的执行时间，最大限度地增加已完成任务的数量，并提高任务分配过程的可靠性。目前，无人机在信息技术服务中的应用主要包括交付[4]、通信[5]、搜救、交通监控和数据收集[6]等。

The UAV task allocation problem in the context of intelligent transportation systems can be divided into four main categories. First, based on whether UAVs can perform multiple different tasks simultaneously, they are classified as either Single-Task-UAVs (S-T UAVs) or Multi-Task-UAVs (M-T-UAVs). Second, depending on whether a task requires multi-UAVs to work together, it is categorized as either a Single-UAV-Task (S-UAV-T) or a Multi-UAV-Task (M-UAV-T). Third, depending on whether the drone task allocation is completed in real time, it can be categorized into Online-Task-Allocation (O-T-A) and Offline-Task-Allocation (OF-T-A). Fourth, based on the presence of dependencies between tasks, tasks can be divided into two types: Independent-Tasks (I-T) and Dependent-Tasks (D-T).

智能交通系统背景下的无人机任务分配问题可分为四大类。首先，根据无人机是否可以同时执行多种不同的任务，它们被分为单任务无人机（S-T无人机）或多任务无人机。其次，根据任务是否需要多架无人机协同工作，它被分为单架无人机任务（S-UAV-T）或多架无飞行器任务（M-UAV-T）。第三，根据无人机任务分配是否实时完成，可分为在线任务分配（O-T-A）和离线任务分配（OF-T-A）。第四，根据任务之间是否存在依赖关系，任务可分为两类：独立任务（I-T）和依赖任务（D-T）。

The two most used drones in the six task allocation models (S-UAV-T, M-UAV-T, O-T-A, OF-T-A, I-T, and D-T) mentioned above are S-T-UAV and M-T-UAV, and all six task allocation models involve several common objectives, including maximizing the total revenue of the task set, minimizing the flight distance, and minimizing the total cost of the fleet [7]. Some UAV task allocation issues in ITS technology are the same as those previously defined. For example, in distribution systems, due to the large-scale characteristics of some distribution problems, a fleet composed of multiple UAVs needs to cooperate to complete the set of tasks. This M-UAV-T allocation problem is defined as the Vehicle Routing Problem (VRP) [8]. In small-scale delivery systems using a single UAV, the S-UAV-T allocation problem is defined as the Traveling Salesman Problem (TSP). The task allocation problem of a UAV-based ITS is a Non-deterministic Polynomial time (NP-hard) problem. In synthesis, the UAV task allocation problem is the determination of the task sequences for a single UAV or a UAV fleet based on the scope and objectives of the entire task set, thereby ensuring its smooth and efficient completion [9]. At the same time, for the UAVtosuccessfully complete its mission, various constraints of both the task and the UAV need to be considered, including the payload capacity, operational speed, task due date, and the maximumflight distance of the UAV.

上述六种任务分配模型（S-UAV-T、M-UAV-T、O-T-A、OF-T-A、I-T和D-T）中使用最多的两种无人机是S-T-UAV和M-T-UAV，所有六种任务配置模型都涉及几个共同目标，包括最大化任务集的总收入、最小化飞行距离和最小化机队的总成本[7]。智能交通系统技术中的一些无人机任务分配问题与之前定义的问题相同。例如，在配送系统中，由于一些配送问题的大规模特性，由多架无人机组成的机队需要协同完成一组任务。这个M-UAV-T分配问题被定义为车辆路线问题（VRP）[8]。在使用单个无人机的小型交付系统中，S-UAV-T分配问题被定义为旅行商问题（TSP）。基于无人机的智能交通系统的任务分配问题是一个非确定性多项式时间（NP-hard）问题。综合来看，无人机任务分配问题是根据整个任务集的范围和目标确定单个无人机或无人机机队的任务序列，从而确保其顺利高效地完成[9]。同时，为了使无人机成功完成任务，需要考虑任务和无人机的各种约束，包括有效载荷能力、作战速度、任务到期日和无人机最大飞行距离。

With the rise of robotic systems technology, the concept of multi-robot task allocation has been established as a dynamic research area in the broad context of operations research applications, and some literature reviews have been recently proposed that also consider UAV-basedsystems[7,10–12]. But, to the best of our knowledge, the literature lacks a critical survey of the application of multi-agent system (MAS)-based task allocation paradigms to a fleet of UAVs conceptualized as an intelligent transportation network. This paper presents a survey of MAS task allocation techniques and their application to drone-based networks for intelligent transportation applications. The main contributions of this work are threefold:

随着机器人系统技术的兴起，多机器人任务分配的概念已被确立为运筹学应用广泛背景下的一个动态研究领域，最近有一些文献综述也考虑了基于无人机的系统[7,10-12]。但是，据我们所知，文献中缺乏对基于多智能体系统（MAS）的任务分配范式在概念化为智能交通网络的无人机机队中的应用的批判性调查。本文对MAS任务分配技术及其在基于无人机的智能交通应用网络中的应用进行了调查。这项工作的主要贡献有三方面：

• Thedevelopment of a critical review about MAS task allocation methodologies fo cusing on multi-UAV networks. This review paper is for engineers, researchers, and scholars who need a critical overview of these emerging topics;

• Thediscussion of state-of-the-art allocation strategies for UAV-based ITSs, focusing on their suitability to the most established applications;

• Thediscussion of the challenges of task allocation algorithms for UAV-based ITSs as well as the gaps in the literature for informing future trends.

•针对多无人机网络，对MAS任务分配方法进行了批判性回顾。本综述论文面向需要对这些新兴主题进行批判性概述的工程师、研究人员和学者；
•讨论基于无人机的信息技术服务的最新分配策略，重点讨论其对最成熟应用的适用性；
•讨论了基于无人机的信息技术服务的任务分配算法的挑战，以及为未来趋势提供信息的文献中的空白。

This paper is organized as follows. Game-theory-based approaches are presented in Section 2. Learning-based algorithms, auction-based algorithms, and optimization-based allocation algorithms are presented in Section 3, Section 4, and Section 5, respectively. Other hybrid approaches are discussed in Section 6. Finally, a comprehensive discussion of UAVstate-of-the-art allocation techniques as well as their pros and cons, their applicability to multi-UAV ITSs, and the current gaps in this field are presented in Section 7. Our conclusions are drawn in Section 8.

本文的结构如下。第2节介绍了基于博弈论的方法。第3节、第4节和第5节分别介绍了基于学习的算法、基于拍卖的算法和基于优化的分配算法。第6节讨论了其他混合方法。最后，第7节全面讨论了无人机最先进的分配技术及其优缺点、其对多无人机ITSs的适用性以及该领域的当前差距。我们的结论见第8节。

Challenges of Task Allocation Algorithms

According to the works of [13,14], UAV task allocation for ITSs can mainly be divided into two categories: OF-T-A (also known as static task allocation) and O-T-A (also known as dynamic task allocation). Unlike static task allocation, dynamic task allocation typically requires the use of fewer computing resources to generate real-time solutions. Centralized algorithms and distributed algorithms are the mainstream algorithms applied to static task allocation and dynamic task allocation, respectively. Currently, algorithms used for static task allocation mainly rely on biologically inspired operators, such as genetic algorithms (GAS) [15], particle swarm optimization (PSO) approaches [16], and differential evolution (DE) algorithms [17], aiming to find approximate optimal solutions in a short period of time. After more than two decades of development, although centralized algorithms have become mature, the aspects of computational time and convergence accuracy still remain significant challenges.

根据[13,14]的工作，ITSs的无人机任务分配主要分为两类：of-T-A（也称为静态任务分配）和O-T-A（也称为动态任务分配）。与静态任务分配不同，动态任务分配通常需要使用较少的计算资源来生成实时解决方案。集中式算法和分布式算法分别是应用于静态任务分配和动态任务分配的主流算法。目前，用于静态任务分配的算法主要依赖于生物启发的算子，如遗传算法（GAS）[15]、粒子群优化（PSO）方法[16]和差分进化（DE）算法[17]，旨在在短时间内找到近似最优解。经过二十多年的发展，尽管集中式算法已经成熟，但计算时间和收敛精度方面仍然存在重大挑战。

In comparison to OF-T-A algorithms, the development of O-T-A algorithm faces other significant challenges. Firstly, real-time task allocation increases the computational demand of solution algorithms, requiring them to solve NP-hard problems with fewer computa tional resources. It is well known that real-time algorithms often sacrifice decision quality to ensure their real-time performance; thus, balancing decision quality and algorithmic real-time performance is also a significant challenge. In addition, the generalization capa bility of the task allocation algorithms poses significant challenges in the task scheduling context of UAV-based ITSs. Determining how algorithms that perform satisfactorily in small-scale networks can adapt to large-scale drone networks has also become an emerging issue for researchers. Finally, the algorithms’ robustness presents further challenges. In unexpected situations, such as the loss of control of a drone or inadequate communication network coverage, real-time algorithms should be able to make immediate decisions to ensure that the completion of the task set is not compromised. Therefore, task reallocation is also being addressed in the literature. There are different types of algorithms that are employed in state-of-the-art drone-based intelligent transportation systems, including auction (market)-based approaches, game-theory-based algorithms, optimization-based algorithms, and machine learning (ML) techniques. These approaches and their application to UAV-based ITSs are thoughtfully presented and discussed in the next sections.

与OF-T-A算法相比，O-T-A算法的发展还面临着其他重大挑战。首先，实时任务分配增加了求解算法的计算需求，要求它们用较少的计算资源解决NP难问题。众所周知，实时算法往往牺牲决策质量来确保其实时性能；因此，平衡决策质量和算法实时性能也是一个重大挑战。此外，任务分配算法的泛化能力在基于无人机的ITSs的任务调度环境中提出了重大挑战。确定在小规模网络中表现良好的算法如何适应大规模无人机网络也成为研究人员面临的一个新问题。最后，算法的鲁棒性带来了进一步的挑战。在意外情况下，例如无人机失控或通信网络覆盖不足，实时算法应该能够立即做出决定，以确保任务集的完成不受影响。因此，任务再分配也在文献中得到了解决。最先进的基于无人机的智能交通系统中采用了不同类型的算法，包括基于拍卖（市场）的方法、基于博弈论的算法、基于优化的算法和机器学习（ML）技术。这些方法及其在基于无人机的ITSs中的应用将在下一节中进行深入的介绍和讨论。

2. Game-Theory-Based Algorithms

2.1. Non-Cooperative-Game-Based Task Allocation

2.2. Cooperative Game-Based Task Allocation

3. Learning-Based Algorithms

For real-time task allocation (O-T-A), the learning-based algorithm is another good approach. Compared to traditional artificial neural networks and deep neural networks, reinforcement learning can handle complex tasks and continuously optimize strategies from the optimization process, making it widely used by researchers in real-time task allocation problems for multi-UAVs. In order to solve the M-UAV-T allocation problem, a deep reinforcement learning algorithm is proposed in [27] with the aim of improving the computational efficiency and the convergence accuracy of the task allocation algorithm. Unlike game-theory-based methods, reinforcement-learning-based algorithms typically establish a nonlinear model based on the task allocation problem, as shown in Equation (1).

对于实时任务分配（O-T-A），基于学习的算法是另一种很好的方法。与传统的人工神经网络和深度神经网络相比，强化学习可以处理复杂的任务，并从优化过程中不断优化策略，使其在多无人机的实时任务分配问题中得到了研究人员的广泛应用。为了解决M-UAV-T分配问题，[27]中提出了一种深度强化学习算法，旨在提高任务分配算法的计算效率和收敛精度。与基于博弈论的方法不同，基于强化学习的算法通常基于任务分配问题建立非线性模型，如方程式（1）所示。

The objective function aims to minimize the cost of the problem, which is the same as the reward function in game-based models. b ≤ Bmax denotes a set of constraints considering the boundary conditions.

目标函数旨在最小化问题的成本，这与基于博弈的模型中的奖励函数相同。b≤Bmax表示考虑边界条件的一组约束。

In addition, the work of [28] also aims at improving the convergence accuracy of the algorithm, thus developing an improved reinforcement learning algorithm. The reinforce ment learning algorithm introduces the transfer learning theory. After finding a similar UAVtask allocation model in the policy library, the algorithm transfers the training parameter results of the previous source task to the new model through transfer learning. The simulation results show that the algorithm not only effectively improves the performance of UAVtask allocation schemes, but also has a strong generalization capability. The authors of [29] developed a multi-agent reinforcement learning method aimed at generating task allocation schemes for heterogeneous UAV fleets. This algorithm can run in locally known environments and has strong robustness.

此外，[28]的工作还旨在提高算法的收敛精度，从而开发出一种改进的强化学习算法。强化学习算法引入了迁移学习理论。在策略库中找到类似的无人机任务分配模型后，该算法通过迁移学习将先前源任务的训练参数结果转移到新模型中。仿真结果表明，该算法不仅有效地提高了无人机任务分配方案的性能，而且具有很强的泛化能力。[29]的作者开发了一种多智能体强化学习方法，旨在为异构无人机机队生成任务分配方案。该算法可以在本地已知的环境中运行，具有很强的鲁棒性。

4. Market-Based Algorithms

5. Optimization-Based Algorithms

6. Hybrid Allocation Algorithms

7. Discussion

For the sake of completeness, before delving into a discussion of the reviewed method ologies along with the applicability of the task allocation methods to the most established applications of UAVs in the context of ITS technology, the main applications of UAVs in the context of ITS technology are listed as follows:

为了完整起见，在深入讨论所审查的方法论以及任务分配方法在智能交通系统技术背景下对无人机最成熟应用的适用性之前，无人机在智能交通技术背景下的主要应用如下：

• Searchandrescue(SR) [83–86];

• Delivery(D)[87–89];

• Trafficmonitoring (TM) [90];

• Inspection (I) [91–93];

• Disasterresponse (DR) [94–96];

• Surveillance (S) [97,98];

• Coverage(C)[99,100];

• Datacollection (DC) [101];

• Smartmobility (SM) [102];

• Agriculture (A) [103–105].

8. Conclusions

The use of unmanned aerial vehicles has gained significant attention in the context of intelligent transportation. The use of different sensors and high-resolution cameras enables the drones to support road transportation vehicles and to be used for a variety of parcel delivery tasks, among other applications. However, a scalable and efficient task allocation architecture must be designed for optimizing the coordination of the fleet of drones of an intelligent transportation system. Generally, task allocation is used to minimize the execution time of the tasks with a reliable and well-defined procedure. A categorization can be defined depending on the number and type of vehicles and tasks employed, including single-task UAVs or multi-task UAVs, single-UAV tasks or multi-UAV tasks, online or offline task allocation, and independent or dependent tasks. In addition, a combination of multiple UAVs and trucks as well as the inclusion of several constraints can significantly improve the overall efficiency. Therefore, the constant development of task allocation enables us to create more efficient methodologies that cover a large variety of scenarios.

在智能交通的背景下，无人机的使用受到了广泛关注。使用不同的传感器和高分辨率摄像头使无人机能够支持道路运输车辆，并可用于各种包裹递送任务以及其他应用。然而，必须设计一个可扩展且高效的任务分配架构，以优化智能交通系统中无人机机队的协调。通常，任务分配用于通过可靠和定义良好的程序来最小化任务的执行时间。可以根据所使用的车辆和任务的数量和类型来定义分类，包括单任务无人机或多任务无人机、单无人机任务或多无人机任务、在线或离线任务分配以及独立或依赖任务。此外，多架无人机和卡车的组合以及几个约束条件的加入可以显著提高整体效率。因此，任务分配的不断发展使我们能够创建更有效的方法，涵盖各种场景。

In this sense, this paper provides a comprehensive literature review of how such approaches are being utilized to optimize the allocation of tasks in UAV-based ITSs. Market based algorithms, game-theory-based algorithms, optimization-based algorithms, machine learning techniques, and other hybrid methodologies are reviewed and discussed. Fur thermore, the main applications of unmanned aerial vehicles in ITSs are presented as well as the suitability of the task allocation algorithms presented throughout the paper with respect to the different applications. The main characteristics of, limitations of, and differences between the algorithms are highlighted, showing their main uses over the last few years. Understanding the main characteristics and the applicability of each type of allocation enables engineers and researchers to properly choose the most appropriate type of task scheduling logic. Moreover, the emerging trends and gaps in the literature are also discussed.

从这个意义上讲，本文对如何利用这些方法优化基于无人机的信息技术服务中的任务分配进行了全面的文献综述。对基于市场的算法、基于博弈论的算法、优化算法、机器学习技术和其他混合方法进行了回顾和讨论。此外，本文还介绍了无人机在ITSs中的主要应用，以及本文中提出的任务分配算法对不同应用的适用性。突出了这些算法的主要特征、局限性和差异，展示了它们在过去几年中的主要用途。了解每种分配类型的主要特征和适用性，使工程师和研究人员能够正确选择最合适的任务调度逻辑类型。此外，还讨论了文献中的新趋势和差距。

In conclusion, we stress the importance of considering the requirements of the service as well as the environmental conditions and the operational capability of the UAV-based intelligent transportation system when designing a task allocation strategy.

总之，我们强调在设计任务分配策略时，考虑服务要求以及基于无人机的智能交通系统的环境条件和运行能力的重要性。

As afurther consideration, it is worth noticing that the design of communication chan nels and their security are fundamental for both implementing (if the allocation architecture is fully decentralized) and validating (if the allocation architecture is centralized or dis tributed) the allocation of tasks in a fleet of UAVs. Also, the security of the communication channels is a significant challenge for achieving a safe, regulatory-compliant, and resilient real-world deployment of a UAV-based ITS. The efficiency of the task allocation process can be heavily influenced by aspects such as the security of the communication channels as well as their fallibility.

作为进一步的考虑，值得注意的是，通信通道的设计及其安全性对于实现（如果分配架构是完全分散的）和验证（如果分配体系结构是集中式或分布式的）无人机机队中的任务分配至关重要。此外，通信信道的安全性对于实现基于无人机的智能交通系统的安全、合规和有弹性的现实部署来说是一个重大挑战。任务分配过程的效率会受到通信信道的安全性及其不可靠性等方面的严重影响。

Future survey-based research will focus on investigating how the UAV communication protocol can influence the efficiency of the task allocation architecture in terms of both security and robustness to fallible communication networks. Also, conceptual modelling frameworks used to implement task allocation algorithms in UAV-based ITSs may also be discussed.

未来基于调查的研究将侧重于调查无人机通信协议如何在安全性和对易出错通信网络的鲁棒性方面影响任务分配架构的效率。此外，还可以讨论用于在基于无人机的ITSs中实现任务分配算法的概念建模框架。