基于智能和视觉的火灾检测系统:调查 （论文翻译）

---

英文版论文原文：英文版论文链接地址

---

基于智能和视觉的火灾检测系统:调查

Intelligent and Vision-based Fire Detection Systems: a Survey

Fengju Bu& Mohammad Samadi Gharajeh

• 山东农业工程大学
• Shandong Agriculture and Engineering University
• Young Researchers and Elite Club, Tabriz Branch, Islamic Azad University

Abstract

火灾是世界上主要的灾难之一。火灾探测系统应在最短的时间内探测到各种环境（例如建筑物，森林和农村地区）中的火灾，以减少经济损失和人为灾难。实际上，火灾传感器是对传统点传感器（例如，烟雾和热探测器）的补充，后者可为人们提供火灾发生的预警。结合图像处理技术的摄像机比点传感器能够更快地检测到火灾。而且，与常规探测器相比，它们更容易提供火的大小，增长和方向。最初，本文简要介绍了火灾探测系统设计中应考虑的各种环境（包括建筑物，森林和矿山）的主要特征。随后，它描述了研究人员在过去十年中提出的一些基于智能和视觉的火灾探测系统。本文将这些系统分为两类：森林火灾智能检测系统和所有环境的智能火灾检测系统。他们使用各种智能技术（例如卷积神经网络，颜色模型和模糊逻辑）来在各种环境中高精度检测火灾发生。在不同评估方案下，将火灾检测系统的性能在检测率，精度，真阳性率，假阳性率等方面进行比较。

Fire is one of the main disasters in the world. A fire detection system should detect fires in various environments (e.g., buildings, forests, and rural areas) in the shortest time in order to reduce financial losses and humanistic disasters. Fire sensors are, in fact, complementary to conventional point sensors (e.g., smoke and heat detectors), which provide people the early warnings of fire occurrences. Cameras combined with image processing techniques detect fire occurrences more quickly than point sensors. Moreover, they provide the size, growth, and direction of fires more easily than their conventional detectors. This paper, initially, presents a glance view on the main features of various environments including buildings, forests, and mines that should be considered in the design of fire detection systems. Afterwards, it describes some of the intelligent and vision-based fire detection systems that have been presented by researchers in the last decade. These systems are categorized, in this paper, into two groups: intelligent detection systems for forest fires and intelligent fire detection systems for all of the environments. They use various intelligent techniques (e.g., convolutional neural networks, color models, and fuzzy logic) to detect fire occurrences with a high accuracy in various environments. Performances of the fire detection systems are compared to each other in terms of detection rate, precision, true positive rate, false positive rate, etc. under different evaluation scenarios.

1. Introduction

火灾会给人类安全，健康和财产造成更多问题。过去几年中，火灾造成的经济和人文损失已大大增加。它们是由一些不可预料的条件引起的，例如燃料管理政策的变更，气候变化以及农村地区的高速发展[1]。火灾探测系统是监视系统中的重要组成部分。它监视各种环境（例如建筑物）作为预警框架的主要单元，以最好地报告起火情况。现有的大多数火灾探测系统都使用有关颗粒采样，温度采样和烟雾分析的点传感器。它们缓慢地测量着火概率，通常在数分钟之内，并且在户外环境中几乎没有适用性。当有足够数量的烟雾颗粒流入设备或温度显着升高时，烟雾探测器和热探测器就会触发以避免误报。此外，时间是减少火灾损失的主要因素之一。因此，应减少探测系统的响应时间，以增加扑灭火灾的机会，从而减少财务和人文破坏[2]。

Fire causes much more problems to human safety, health, and property. Financial and humanistic losses in fires have been considerably increased over the past years. They are happened by some of the unpredicted conditions such as changing fuel management policies, climate change, and high development in rural areas [1]. Fire detection system is an important component in surveillance systems. It monitors various environments (e.g., buildings) as main unit of an early warning framework to report preferably the start of fire. Most of the existing fire detection systems use point sensor regarding the particle sampling, temperature sampling, and smoke analysis. They measure fire probabilities slowly, usually in minutes, and also have little applicability in outdoor environments. Upon a sufficient amount of smoke particles flows into the device or temperature is increased considerably, the smoke and heat detectors trigger to avoid false alarms. Besides, time is one of the main factors to minimize fire damages. Thus, the response time of detection systems should be decreased to increase the chances to extinguish fires and, thereby, reduce the financial and humanistic damages [2].

在过去的几十年中，研究人员试图实现点检测器的慢响应时间。 在替代的基于火灾传感器的自动火灾探测系统计算中可以观察到此问题。 他们总结说，检测烟雾和火焰的最新有效技术之一是从视觉传感器获得图像和/或视频帧[3]。 基于视觉传感器的技术可以缩短响应时间，提高检测过程的概率，并显着提供大面积的覆盖范围（例如，开放环境）。 视觉传感器可以提供有关火和烟的方向，生长以及大小的更多信息[4]。 此外，我们可以以较低的实施成本升级安装在各种环境（例如工厂和公共场所）中的现有监视系统。 通过使用视频处理和机器视觉技术，可以执行此过程以提供火灾预警，并检测火焰和烟雾。

Researchers have attempted to achieve the slow response time of point detectors in the last decades. This issue can be observed in the alternative, fire sensor based computing for automatic fire detection systems. They have summarized that one of the latest efficient techniques to detect smoke and flame is to achieve images and/or video frames from vision sensors [3]. Vision sensor based techniques can lead response time to be decreased, probability of the detection process to be enhanced, and coverage of large areas (e.g., open environments) to be provided considerably. Vision sensors can give more information about the direction, growth, and size of fire and smoke [4]. In addition, we can upgrade the existing surveillance systems installed in various environments (e.g., factories and public sites) with low implementation costs. This process can be done to provide early warnings of fires and, also, detect flame and smoke by using video processing and machine vision techniques.

由视觉传感器实现的火灾探测系统通常由三个单元组成：运动探测，候选对象分割以及火焰或烟雾斑点探测。在这些系统中，相机通常被认为是固定的，但是基于图像减法和背景建模技术的一些变化[5]，可以检测到初始运动。运动检测过程之后可以使用颜色信息[6，7]，像素可变性度量[8]，纹理提示[9]或光流场分布[10]对候选对象进行分割。可以通过图像分离方法[11，12]或通过以数学术语对火焰或烟雾建模来识别火焰和烟雾斑点[13-15]。在进行运动检测之后，可以通过使用形态图像处理方法作为预处理步骤来识别火灾的地理区域。消防系统检测火灾和烟雾的能力取决于摄像机视野的视点和特定场景的深度。此外，烟雾和火焰具有不同的动态行为和物理特性。因此，与仅定位烟雾区域或火焰区域的系统相比，同时定位烟雾和火焰区域的消防系统可以更早地检测到火灾[16]。

Fire detection systems, implemented by vision sensors, are generally composed of three units: motion detection, candidate object segmentation, and flame or smoke blob detection. In these systems, cameras are usually considered to be fixed but initial motion is detected based on some variations on image subtraction and background modeling techniques [5]. Motion detection process can be followed by candidate object segmentation using color information [6, 7], pixel variability measures [8], texture cues [9], or optical flow field distribution [10]. Flame and smoke blobs can be identified by image separation approaches [11, 12] or by modeling the flame or smoke in mathematical terms [13-15]. After motion detection, geographical regions of fire can be identified by using morphological image processing methods as a preprocessing step. Ability of a fire system for detecting the fire and smoke depends on the viewpoint of camera field and the depth of specific scene. Moreover, smoke and flame have different dynamic behaviors and physical properties. Therefore, a fire system that locates the regions of both smoke and flame can considerably detect a fire earlier than a system that only locates the region of smoke or the region of flame [16].

研究人员已经提出了各种基于智能和视觉的火灾探测系统，尤其是在过去的十年中。他们使用一些方法和系统，例如相机（例如光学相机[17]），智能技术（例如神经网络[18-20]，粒子群优化[21]和模糊逻辑[22]），无线网络（例如，无线传感器网络和卫星系统[17]），机器人系统（例如，无人机系统[23、24]）和图像处理技术（例如，RGB颜色[25]）。本文介绍了一些最新的基于视觉的智能消防系统的主要特征，优点和/或缺点。这些消防系统的设计和实现是为了检测不同环境（例如森林[26]）中的火灾。似乎使用智能和基于视觉的技术已对火灾探测系统的性能产生了显着影响。使用智能技术（例如卷积神经网络），根据火灾的各种特征（例如火焰，烟雾和颜色）来组织最多的火灾检测系统。也就是说，这些系统之间有许多共同的特征。因此，本文根据环境类型对它们进行分类，包括森林火灾检测系统和所有室内和室外环境的火灾检测系统。

Various intelligent and vision-based fire detection systems have been presented by researchers, especially in the last decade. They use some of the methods and systems such as cameras (e.g., optical cameras [17]), intelligent techniques (e.g., neural networks [18-20], particle swarm optimization [21], and fuzzy logic [22]), wireless networks (e.g., wireless sensor networks and satellite systems [17]), robotic systems (e.g., unmanned aerial systems [23, 24]), and image processing techniques (e.g., RGB color [25]). Main features, advantages, and/or drawbacks of some of the latest intelligent and vision-based fire systems are described in this paper. These fire systems are designed and implemented for detecting the fire occurrences in different environments (e.g., forests [26]). It seems that the use of intelligent and vision-based techniques have had noticeable effects on the performance of fire detection systems. The most fire detection systems are organized based on various features of the fire (e.g., flame, smoke, and color) using the intelligent techniques (e.g., convolutional neural networks). That is, there are many common features between these systems. Therefore, they are categorized in this paper based on types of the environments including detection systems for forest fires and fire detection systems for all of the indoor and outdoor environments.

其余论文的结构如下。 第2节介绍了火灾探测系统应考虑的环境类型。 第3节介绍了一些基于智能和视觉的火灾探测系统，它们在不同的环境中使用各种方法（例如卷积神经网络）。 第4节根据不同的参数（例如检测率，真实阳性率和错误警报率）评估最新检测系统的仿真和/或实验结果。 第五部分讨论了智能火灾探测系统的基准数据集。 最后，第6节总结了论文。

The rest of paper is organized as follows. Section 2 presents types of the environments that should be considered by fire detection systems. Section 3 describes some of the intelligent and vision-based fire detection systems that use various methods (e.g., convolutional neural networks) in different environments. Section 4 evaluates the simulation and/or experimental results of the latest detection systems according to different parameters such as detection rate, true positive rate, and false alarm rate. Section 5 discusses about benchmark datasets of the intelligent fire detection systems. Finally, Section 6 concludes the paper.

2.环境类型

2. Types of the environments

火灾可能会破坏各种类型的环境。 应当针对此类环境设计和实施消防系统，以防止严重的生命损失，并且还应保留宝贵的自然和个人财产。 本节介绍了三个重要环境的一些物理特征，包括建筑物，森林和矿山。

There are various types of the environments that may be destroyed by fires. Fire systems should be designed and implemented for such environments to prevent terrible loss of lives and, also, survive valuable natural and individual properties. This section presents some physical features of the three important environments including buildings, forests, and mines.

荒地火灾可能会破坏建筑物组件和系统的暴露。根据暴露条件，随着野火向社区和城市地区蔓延，建筑物可能会着火。因此，通过了解缓解策略和结构的着火潜力，可以完全防止或大大减少着火。社区设计，规划实践，主动/被动抑制机制和基于工程的方法是一些缓解策略，可以减少火灾的财务和财产损失。 Hakes等。 [27]代表了典型的建筑构件和系统，这些构件和系统可以被认为是城市地区潜在的起火途径。他们讨论了一种具有主要组件和系统的结构，包括屋顶，山谷，天窗，檐槽，屋檐，山墙，通风口，壁板，甲板，围栏和覆盖物。当结构暴露于辐射，火斑和火焰接触时，结构上的某些组件将直接开始燃烧或先闷燃，然后过渡到明火。由于摇晃的木瓦和带状瓦的屋顶容易燃烧，有许多缝隙和大面积的易燃材料，因此通常会增加火灾危险[28]。檐沟中使用的碎屑可能会被火印点燃，尤其是在檐沟处。因此，檐槽是引爆房屋的潜在途径之一。在城市家庭中，屋檐和通风口被认为是可能的着火源。此外，通风孔会导致通向燃烧的品牌并渗入建筑物内部[29]。由于暴露于辐射热或直接接触火焰，通常认为壁板材料在起火时会着火。火斑的堆积会点燃附近的植被或其他燃料（例如，木桩和覆盖物），而在结构基础周围没有适当的间隙。它可能会导致辐射热暴露，并使外墙的壁板材料直接接触火焰。甲板，围栏和覆盖物是结构的其余材料，可能会引起最潜在的着火源[30-32]。

Wildland fires can destroy the exposure of building components and systems. Structures may ignite as wildland fire spreads toward the community and urban areas based on the exposure conditions. Thus, the ignition can be fully prevented or considerably reduced by understanding the mitigation strategies and the ignition potential of structures. Community design, planning practices, active/passive suppression mechanisms, and engineering-based methods are some of the mitigation strategies that can reduce the financial and property losses of fire occurrences. Hakes et al. [27] have represented the typical building components and systems that can be considered as potential pathways to ignition in urban areas. They have discussed about a structure with the main components and systems including roof, valley, dormer, gutter, eave, gable, vent, siding, deck, fence, and mulch. When a structure is revealed to radiation, firebrands, and flame contact, some components on the structure would directly begin to flame or firstly smolder and then transition to the flaming ignition. Since wood shake and shingle roofs ignite easily with many crevices and a large surface area of the flammable material, they often cause an increased fire risk [28]. The debris utilized in gutter may be ignited by firebrands, especially at roof-gutter intersection. Thus, gutters are one of the potential pathways to ignition of a home. Eaves and vents are identified as possible sources of ignition at homes in urban areas. Besides, vents cause to open a way toward the burning brands and penetrate the interior of a structure [29]. Because of radiant heat exposure or direct flame contact, siding materials have been often considered to ignite on fires. Firebrand accumulation can ignite the nearby vegetation or other fuels (e.g., wood piles and mulch) without proper clearance around the foundation of a structure. It can cause radiant heat exposure and direct flame contact in the siding materials on exterior walls. Decks, fences, and mulches are the remainder materials of a structure that may cause the most potential sources of ignition [30-32].

森林在大多数方面保护着地球的生态平衡。当森林大火蔓延到大片区域时，将相应地通知大火。因此，有时无法控制和停止它。森林大火造成了巨大损失，对环境和大气造成了不可挽回的破坏，对生态也造成了不可挽回的破坏。森林火灾不仅造成人员伤亡，宝贵的自然环境和个人财产（包括数百所房屋和数千公顷森林）的悲剧性丧失，​​而且对生态健康和环境保护构成重大威胁。研究表明，森林大火在大气和生态环境中增加了30％的二氧化碳（CO2）和大量的烟雾[33]。此外，森林大火会造成可怕的后果和长期灾难性影响，例如全球变暖，对当地天气模式的不利影响以及动植物稀有物种的灭绝[17]。 Sahin [34]提出了森林火灾探测系统的示例基础设施，但没有任何基于视觉的智能技术。该系统包括各种元素，尤其是通信通道，中央分类器和移动生物传感器。通信信道由基于卫星的系统和几个接入点提供，以覆盖森林环境的所有位置。中央分类器用于通过访问点对移动生物传感器的读数进行分类，并且还可以通过使用决策支持系统来实现。移动生物传感器是该系统最重要的部分，可传输温度或辐射水平的任何变化，从而将动物的地理位置发送到访问点。

Forests protect the earth’s ecological balance, in the most aspects. When a forest fire has spread over a large area, the fire will be notified accordingly. Thus, it is not possible to control and stop it at times. Forest fires cause a big loss and an irrevocable damage to environment and atmosphere plus an irrevocable damage to the ecology. Forest fire not only causes the tragic loss of lives, valuable natural environments, and individual properties (including hundreds of houses and thousands of hectares of forest), but also it is a big danger to ecological health and protection of environment. Researches show that forest fires increase 30% of carbon dioxide (CO2) and huge amounts of smoke in atmosphere and ecology [33]. Furthermore, forest fires cause terrible consequences and long-term disastrous effects such as global warming, negative effect on local weather patterns, and extinction of rare species of the fauna and flora [17]. Sahin [34] has presented a sample infrastructure of the forest fire detection system, without any intelligent and vision-based technique. This system includes various elements, especially communication channels, a central classifier, and mobile biological sensors. Communication channels are provided by a satellite based system and several access points to cover all locations of the forest environment. The central classifier is used to categorize readings of the mobile biological sensors via access points and is also implemented by using a decision support system. Mobile biological sensors are the most important parts of this system, which transmit any changes of the temperature or radiation level and thereby send geographical location of the animals to access points.

由于地下煤矿含有煤尘，甲烷和其他有毒气体[35]，因此它们被认为是非常危险和危险的环境[36]。煤尘或甲烷爆炸的结果是导致采矿业约33.8％死亡的主要原因。最近的报道表明，盐范围地区煤矿的瓦斯积累已引起约38％的地下煤矿事故[37]。地下煤矿中存在各种有毒气体，包括甲烷气，一氧化碳（CO），CO2和硫化氢（H2S），这些气体会严重危害人体[38]。因此，针对地下矿山环境的有效监控系统可以在很大程度上确保矿工和矿山财产的安全。 Jo和Khan [39]提出了一种针对Hassan Kishore煤矿的预警安全系统，该系统使用物联网（IoT）而不是使用任何低成本，智能的，基于视觉的系统。该系统使用安装在各个位置的固定节点和移动节点来检测现象数据。此外，路由器节点专用于每个部分，以收集数据，然后通过网关节点将读数数据发送到基站。

Since underground coal mines contain coal dust, methane, and other toxic gases [35], they have been considered as a very hazardous and dangerous environment [36]. A consequence of coal dust or methane gas explosions is the main reason to cause approximately 33.8% of deaths in the mining sector. Recent reports represent that gas accumulation in the coal mines of salt-range region has caused approximately 38% of underground mine accidents [37]. There exist various toxic gases including methane gas, carbon monoxide (CO), CO2, and hydrogen sulfide (H2S) in underground coal mines that can critically lead to damage human body [38]. Consequently, an efficient monitoring system for underground mine environments can mostly guarantee the safety of miners and mine property. Jo and Khan [39] have presented an early-warning safety system for the Hassan Kishore coal mine that uses the Internet of Things (IoT) instead of using any low-cost, intelligent, vision based system. This system uses the stationary and mobile nodes that are installed in various locations to detect phenomena data. Moreover, a router node is dedicated for every section to gather and then transmit readings data toward a base station via gateway node.

3. 使用智能和基于视觉的方法的火灾探测系统

3. Fire detection systems using the intelligent and vision-based methods

可以组织火灾探测系统以在各种环境中使用。 由于每种环境都有一些特殊的规范，因此某些火灾探测可用于森林火灾，某些火灾探测可用于室外环境，而其他一些则可在所有环境中使用。 因此，本节将智能和基于视觉的火灾探测系统分为两组：森林火灾和所有环境。

Fire detection systems can be organized to use in various environments. Since every environment has some special specifications, some of the fire detections can be used in forest fires, some ones can be applied in outdoor environments, and some others can be utilized in all of the environments. Accordingly, this section describes the intelligent and vision-based fire detection systems into two groups: forest fires and all of the environments.

3.1. 森林火灾智能检测系统

3.1. Intelligent detection systems for forest fires

在图像分类和基于视觉的系统中，卷积神经网络（CNN）被认为是一种高性能技术[40]。它可用于提高火灾通知系统的检测准确性，最大程度地减少火灾，并减少社会和生态后果。张等。 [41]提出了一种基于深度学习的森林火灾检测系统。他们已经通过使用与深层CNN关联的完整图像和细粒补丁火分类器来训练该系统。该系统使用两种情况：（i）通过从头开始火灾补丁学习二进制分类器，以及（ii）学习全图像火灾分类器，然后应用细粒度补丁分类器。作者从在线资源中收集了火灾探测数据集[42]，这些数据已在最新的火灾探测文献中得到应用。该数据集包括从森林环境获得的25个视频。它包含21个正（激发）序列和4个负（非激发）序列。这些文献中的大多数都报告了图像级别的评估，而另一些文献还报告了补丁级别的检测准确性。作者使用以下方程式来检测火灾的发生：

Convolutional neural network (CNN) is considered as a high-performance technique in image classification and vision-based systems [40]. It can be used to improve detection accuracy of fire notification systems, minimize fire disasters, and reduce the social and ecological results. Zhang et al. [41] have presented a deep-learning based detection system for forest fires. They have trained this system by using a full image and fine grained patch fire classifier that are associated to a deep CNN. The system uses two scenarios: (i) learning the binary classifier by fire patches from scratch and (ii) learning a full-image fire classifier and then applying the fine-grained patch classifier. The authors have collected a fire detection dataset from the online resources [42] that have been applied in the latest fire detection literatures. This dataset includes 25 videos obtained from forest environments. It contains 21 positive (fire) sequences and 4 negative (non-fire) sequences. Most of these literatures report image-level evaluation while some of them also report patch-level detection accuracy. The authors have used the below equations to detect a fire occurrence:

$$accuracy=\genfrac{}{}{0ex}{}{}{POS+NEG}$$