论文：Automated Vehicle Detection and Classification: Models, Methods, and Techniques翻译（一）

要开始写开题报告了，得看一些外文文献。就在此处做做笔记吧，翻译水平有限，就这样吧。

[1] Boukerche A, Siddiqui A J, Mammeri A, et al. Automated Vehicle Detection and Classification: Models, Methods, and Techniques[J]. AcmComputing Surveys, 2017, 50(5):1-39.

Automated Vehicle Classification (AVC) based on vision sensors has received active attention from researchers, due to heightened security concerns in Intelligent Transportation Systems. In this work, we propose a categorization of AVC studies based on the granularity of classification, namely Vehicle Type Recognition, Vehicle Make Recognition, and Vehicle Make and Model Recognition. For each category of AVC systems, we present a comprehensive review and comparison of features extraction, global representation, and classification techniques. We also present the accuracy and speed-related performance metrics and discuss how they can be used to compare and evaluate different AVC works. The various datasets proposed over the years for AVC are also compared in light of the real-world challenges they represent, and those they do not. The major challenges involved in each category of AVC systems are presented, highlighting open problems in this area of research. Finally, we conclude by providing future directions of research in this area, paving the way toward efficient large-scale AVC systems. This survey shall help researchers interested in the area to analyze works completed so far in each category of AVC, focusing on techniques proposed for each module, and to chalk out strategies to enhance state-of-the-art technology.

基于视觉传感器的自动车辆分类（AVC）得到了研究人员的积极关注，原因是智能交通系统中的安全问题日益严重。在这项工作中，我们提出了基于分类粒度的AVC研究分类，即车辆类型识别，品牌（徽标）或品牌和型号识别。对于每个类别的AVC系统，我们提供全面的评论和特征提取，全局表示和分类技术的比较。我们还提供准确性和速度相关的性能指标，并讨论如何将它们用于比较和评估不同的AVC作品。考虑到他们所代表的现实世界的挑战以及他们所没有的挑战，对多年来提出的各种数据集进行了比较。介绍了各类AVC系统所面临的主要挑战，突出了这一领域的研究中存在的问题。最后，我们通过提供未来这一领域的研究方向来结束研究，为高效的大规模AVC系统铺平道路。这项调查将有助于对该地区感兴趣的研究人员分析目前在各类AVC中完成的工作，重点关注为每个模块提出的技术，并提出战略以加强最先进的技术。

CCS Concepts: • Computing methodologies→Computer vision; Visual inspection; Object recognition;• Information systems→Information retrieval;

Additional Key Words and Phrases: Vehicle classification, vehicle detection, vehicle recognition, vehicle verification, vehicle identification, vehicle localization, target tracking, intelligent transportation system, video surveillance, smart city surveillance

CCS概念：•计算方法学→计算机视觉; 视力检查; 物体识别;•信息系统→信息检索;

其他关键词和短语：车辆分类，车辆检测，车辆识别，车辆验证，车辆识别，车辆定位，目标跟踪，智能交通系统，视频监控，智能城市监

1 INTRODUCTION

Over recent years, a plethora of innovative technologies and solutions are bringing Intelligent Transportation Systems (ITSs) closer to reality. The ITS Society of Canada defines ITS as “the application of advanced and emerging technologies (computers, sensors, control, communications, and electronic devices) in transportation to save lives, time, money, energy and the environment” (ITS Canada 2012a). The development of digital image sensors and computer vision techniques offer a significant advantage in enabling many important Smart City and ITS applications, such as automated vehicular surveillance, traffic and activity monitoring, traffic behavior analysis, traffic management, and so on. Identification and classification of vehicles by type, make, or model is of great interest in these applications (ITS Canada 2012b), giving rise to many research attempts to develop robust and real-time Automated Vehicle Classification (AVC) systems. However, automated classification of vehicles into fine categories such as these has gained attention only recently, and many challenges remain yet to be addressed.

近年来，大量的创新技术和解决方案正在使智能交通系统（ITS）更接近现实。加拿大ITS协会将ITS定义为“先进技术（计算机，传感器，控制，通信和电子设备）在交通行业的应用，用以挽救生命，时间，金钱，能源和环境”（ITS Canada 2012a）。数字图像传感器和计算机视觉技术的发展为启用许多重要的智能城市和ITS应用提供了显着的优势，如自动车辆监控，交通和活动监控，交通行为分析，交通管理等。根据类型，制造商或型号对车辆进行识别和分类对于这些应用（ITS Canada 2012b）非常感兴趣，从而引发了许多研究尝试来开发强健的实时自动车辆分类（AVC）系统。然而，将车辆自动分类为诸如此类的优良类别直到最近才得到关注，许多挑战仍有待解决。

AVC systems are designed to assist human operators or security personnel to efficiently extract relevant information. Security is a great concern in vulnerable areas, such as public parking lots at malls, stadiums, airports, and so on. In such critical scenarios, an AVC system running over surveillance cameras’ images can greatly assist the human operators or security personnel in identifying vehicles of certain colors, types, makes (manufacturers), and models. In the literature, the term “make” has been used to refer to the “auto manufacturer,” for example, Toyota, Dodge, Ford, and so on. Dlagnekov (2005), Pearce and Pears (2011), AbdelMaseeh et al. (2012), M. Fraz et al. (2014), and Chen et al. (2015). Moreover, in cases where the police are searching for a target vehicle of a specific make and model, AVC systems would save a considerable amount of time, resources, and manpower. In addition, for applications such as electronic toll collection, where different charges are applied to different types of vehicles, vision-based AVC systems could serve as a complementary tool in improving the efficiency of existing systems. 

AVC系统旨在帮助操作人员或安全人员去有效地提取相关信息。在诸如商场，体育场，机场等公共停车场等易受攻击的地区，安全性是一个非常重要的问题。在这种危急情况下，运行监控摄像机图像的AVC系统可以极大地帮助操作人员或安全人员识别某些颜色，类型，制造商和型号的车辆。在文献中，“make”一词被用来指代“汽车制造商”，例如丰田，道奇，福特等。 Dlagnekov（2005），Pearce和Pears（2011），AbdelMaseeh等人。 （2012），M.Fraz等人（2014）和Chen等人（2015年）。此外，如果警方正在搜索特定品牌和型号的目标车辆，AVC系统将节省大量时间，资源和人力。此外，对于不同类型的车辆收取不同费用的电子收费系统等应用，基于视觉的AVC系统可作为提高现有系统效率的补充工具。

Traditional vehicle classification systems recognize makes and models of vehicles by relying on manual human observations or automated license plate recognition (ALPR) systems, hardly meeting the real-time constraints.Both approaches are failure-prone and have several limitations.First, it is practically difficult for human observers to remember and efficiently distinguish between the wide variety of vehicle makes and models. Second, it becomes a laborious and time-consuming task for a human observer tomonitor and observe them ultitude of screens and record the incoming or outgoing makes and models, or to even spot the make and model being looked for.

传统的车辆分类系统依靠手动人工观察或自动车牌识别（ALPR）系统来识别车辆的制造商和型号，难以满足实时要求。两种方法都很容易出故障并且有几个局限性。首先，人类观察员难以记住并且有效地区分各种各样的车辆制造商和模型。 其次，对于观察人员来说，这是一项费时费力的任务。需要观察它们的屏幕高度并记录进入或离开的车辆的制造商和型号，或者甚至找出正在寻找的制造商和型号所代表的车辆。

On the other hand, the VC systems that rely on license-plates suffer from the following disadvantages. License plates can easily be forged, damaged, modified, or occluded. In addition, some license plates can be ambiguous (e.g., between “0” and “O”). Consequently, when ALPR systems fail to correctly read the detected license plates due to the above issues, the wrong make and model information could be retrieved from the license plate registry or database. The type, make, and model of a vehicle recognized by the AVC system can be cross-checked with the license plate registry to screen for any fraud. In this way, AVC augments traditional ALPR-based vehicle identification and classification systems to further enhance security and prevent fraud.

另一方面，依赖牌照的VC系统存在以下缺点。 车牌很容易被伪造，损坏，修改或堵塞。 另外，一些牌照可能不明确（例如，在“0”和“O”之间）。 因此，当ALPR系统由于上述问题而未能正确读取检测到的车牌时，可从车牌登记处或数据库中检索出错的车牌和车型信息。 AVC系统识别的车辆类型，品牌和型号可以与车牌注册表进行交叉检查，以筛选任何欺诈行为。 通过这种方式，AVC增强了传统的基于ALPR的车辆识别和分类系统，以进一步增强安全性并防止欺诈行为。

To overcome the above shortcomings, though the AVC systems have recently gained attention, much research is still needed to develop systems that are robust to real-world conditions and that perform in real time. The idea of automatically recognizing the type, make, or model of a vehicle seen by a camera may seem very simple, especially with the advancements in human face recognition and image search technology. Despite the perceived simplicity, AVC has its own set of challenges and issues, summarized below:

虽然AVC系统最近受到了关注，为了克服上述缺陷，但仍然需要进行大量研究，以开发对现实世界条件稳健且实时执行的系统。 自动识别相机看到的车辆的类型，结构或型号可能看起来非常简单，尤其是随着人脸识别和图像搜索技术的进步。 尽管简单易懂，但AVC有其自身的一系列挑战和问题，总结如下：

—Diversity: there exists a large variety of vehicle types (sedans, vans, SUVs, trucks, etc.), makes and models, with diverse colors
—Multiplicity: when vehicles of the same class exhibit different shapes, designs, or sizes
—Ambiguity: when vehicles in different classes have similar shapes or designs
—Heterogeneous Views: varying viewpoints, scales, rotation, camera position, or camera pitch angle
—Lighting Conditions: images taken under different illumination conditions (bright sunlight, glare on vehicles, shadows of objects upon vehicles, dusk, night, etc.)
—Environmental Conditions: different weather conditions such as: sunny, cloudy, rainy, foggy, snowy, and so on.
—Occlusions: challenging scenarios, such as low-to-heavy occlusions (due to pedestrians or other objects), partially hidden vehicle views, closely adjacent vehicles, and so on.
—Modified Appearance: when a vehicle's original appearance may be modified, for example, by painting a different pattern on the hood or trunk, changing the bumpers or grill, adding spoilers, adding neon lights below bumpers, and so on.

—Dataset Availability: lack of publicly accessible benchmark datasets to enable researchers to evaluate different AVC proposals.

- 多样性：有各种各样的车型（轿车，货车，SUV，卡车等），品牌和型号，多种颜色
- 多样性：同一类别的车辆呈现不同的形状，设计或尺寸
- 含糊不清：当不同类别的车辆具有相似的形状或设计时
- 异构视图：不同的视点，缩放，旋转，相机位置或相机俯仰角
- 照明条件：在不同照明条件下拍摄的图像（明亮的阳光，车辆上的眩光，车辆上的物体的阴影，黄昏，夜间等）
- 环境条件：不同的天气条件，例如：晴天，多云，多雨，多雾，多雪等等。
- 遮挡：具有挑战性的情景，例如由于行人或其他物体而产生的低重度遮挡（occlusion），部分隐藏的车辆视野，紧密相邻的车辆等等。
-修改后的外观：当车辆的原始外观可能被修改时，例如，在发动机罩或后备箱上涂抹不同的图案，更换保险杠或格栅，添加扰流板，在保险杠下面添加霓虹灯等等。

-数据集可用性：缺乏可公开访问的基准数据集，使研究人员能够评估不同的AVC提案。

In addition to the above-mentioned challenges, an AVC system must have high processing speeds and be capable of performing in real-time. To have an AVC system working from a moving vehicle (i.e., an On-Board AVC, or OB-AVC) is an even more difficult problem, due to added effects of ego-motion, rapidly changing views and lighting conditions, heavy occlusions, or obstructed views of targets. To the best of our knowledge, OB-AVC systems have received very little attention, possibly none, by the research community.

除了上述挑战之外，AVC系统必须具有高处理速度并且能够实时执行。 由于附加的自我运动效应，快速变化的视图和照明条件，严重的遮挡物，或目标视野阻挡等不利因素，要使移动车辆（即，车载AVC或OB-AVC）具有AVC系统工作是更加困难的问题。 。 就我们所知，OB-AVC系统很少受到研究界的关注，可能没有。

Based on our comprehensive survey of different works in AVC, to the best of our knowledge there is no single proposal that is acceptable for all the real-world challenges mentioned above. AVC is an emerging area of research that calls for intensive efforts to tackle the above-mentioned issues.

根据我们对AVC不同作品的综合调查，据我们所知，没有任何单一提案可以接受上述所有现实世界的挑战。 AVC是一个新兴的研究领域，需要作出密集的努力来解决上述问题。

This article is organized as follows. In Section 2, we present the general architecture of AVC systems, which will then be used for a systematic comparison of different AVC approaches in light of the constituting modules. Here, we introduce the proposed taxonomy of AVC works. We then describe the typical target environments that have been the focus of most AVC works.We believe that this will assist in better comprehension of the goals and methods of various AVC works. Based on the  roposed taxonomy of AVC works, we dedicate a section for each category of AVC: Section 3 for Vehicle Type Recognition, Section 4 for Vehicle Make Recognition, and Section 5 for Vehicle Make and Model Recognition. In each of these sections, we will present the respective challenges and issues, reviewthe proposalsmade over the years, and  iscuss potential directions of future work in these areas. Although vehicle detection (VD) is not the focus of our review, for the purpose of completeness,we present in Section 6 a brief reviewof VD techniques employed in AVC systems.We provide the accuracy and speed related performance metrics in Section 7 and discuss how they can be used to compare and evaluate different AVC works. In Section 8, we present the requirements for a comprehensive AVC dataset, review and analyze the various datasets proposed for AVC, and highlight the dataset issues that remain to be addressed. In Section 9, we then discuss open challenges in the area of AVC, and present our vision for enhancing the state-of-the-art.

这篇文章的结构安排如下。在第2节中，我们将介绍AVC系统的总体架构，然后根据构成的模块将其用于不同AVC方法的系统比较。在这里，我们介绍AVC作品的建议分类。然后我们描述一直是大多数AVC作品关注的典型目标场景。我们相信这将有助于更好地理解各种AVC作品的目标和方法。根据AVC作品的分类标准，我们为AVC的每个类别分配了一个部分：第3部分为车型识别，第4部分为车辆品牌（徽标）识别，第5部分为车辆品牌和型号识别。在这些部分中，我们将分别介绍各自的挑战和问题，回顾多年来提出的建议，并探讨这些领域未来工作的潜在方向。尽管车辆检测（VD）并不是我们评论的重点，但为了完整起见，我们在第6节简要回顾了AVC系统中采用的VD技术。我们在第7节中提供了准确性和速度相关的性能指标，并讨论了他们可以用来比较和评估不同的AVC作品。在第8节中，我们提出了一个全面的AVC数据集的要求，审查和分析了针对AVC提出的各种数据集，并强调了尚待解决的数据集问题。在第9节中，我们将讨论AVC领域的公开挑战，并展示我们提升最新技术的愿景。

2 GENERAL OVERVIEW OF AVC SYSTEMS

The problem of AVC is an important consideration for traffic/vehicular surveillance and other ITS applications. Almost all AVC works can be described according to the architecture shown in Figure 2. Most works first adopt a Vehicle Detection system that produces Regions of Interest (ROIs) containing the vehicles’ faces (front or rear), segmented from the background. The Vehicle Classification system then works on these ROIs. Depending on the granularity of classification, AVC systems can be grouped into three categories: Type, Make (Logo), or Make-and-Model recognition, as depicted in Figure 1.

AVC的问题是交通/车辆监控和其他ITS应用的重要考虑因素。 几乎所有的AVC作品都可以根据图2所示的架构进行描述。大多数作品首先采用车辆检测系统，该系统会生成包含车辆背面（前部或后部）的感兴趣区域（ROI）。 车辆分类系统然后处理这些ROI。 根据分类的粒度，AVC系统可以分为三类：类型，品牌（徽标）或品牌和型号识别，如图1所示。

Within an AVC system, the three major modules or components are: (1) Features Extractor, (2) Global Representation Generator, and (3) Classifier, as shown in Figure 2. In the following sections, we provide a review of works done in different categories of AVC systems, to show how research is evolving in this challenging area. In particular, we review and compare various proposals for the three modules of AVC systems, discuss the issues and challenges there with, and present future research directions.

在AVC系统中，三个主要模块或组件是：（1）特征提取器，（2）全局表示生成器和（3）分类器，如图2所示。在以下各节中，我们在不同类别的AVC系统中提供了对所完成的工作的回顾，去展示如何在这一具有挑战性的领域进行研究。 我们特别回顾并比较了AVC系统三个模块的各种提案，讨论了其中的问题和挑战，并提出了今后的研究方向。

Most AVC works focus on environments with fixed camera(s) that capture incoming and/or outgoing vehicles. Such a case is dominant in scenarios that are highly vulnerable to security threats, such as public areas. Examples include checkpoints, entrances and exits of parking lots at airports, malls, schools, government offices, or stadiums (see Figure 3). Works such as Siddiqui (2015) have attempted working towards an on-board AVC system in a scenario where a camera was installed on a moving vehicle. However, these works were based on a limited number of vehicle classes,and there are no publicly accessible datasets for such scenarios (to the best of our knowledge). Developing mobile or on-board AVC systems seems to be an interesting direction of research.

大多数AVC作品都专注于使用固定摄像头拍摄进入和/或外出车辆的环境。 这种极易受到安全威胁的场景占主导地位，如公共场所。 例如检查站，机场，商场，学校，政府机关或体育场的停车场入口和出口（见图3）。 像Siddiqui（2015）这样的作品曾试图在摄像机安装在移动车辆上的情况下，在车载AVC系统上工作。 但是，这些作品是基于数量有限的车辆类别，并且目前还没有可公开获取的数据集（据我们所知）。 开发移动或机载AVC系统似乎是一个有趣的研究方向。