Summary of defect detection algorithms based on deep learning

缺陷检测是工业上非常重要的一个应用，由于缺陷多种多样，传统的机器视觉算法很难做到对缺陷特征完整的建模和迁移，复用性不大，要求区分工况，这会浪费大量的人力成本。深度学习在特征提取和定位上取得了非常好的效果，越来越多的学者和工程人员开始将深度学习算法引入到缺陷检测领域中，下面将会介绍几种深度学习算法在缺陷检测领域中的应用。

• Defect detection is a very important application in industry. Due to the diversity of defects, it is difficult for traditional machine vision algorithm to complete modeling and migration of defect features, with little reusability and the requirement to distinguish working conditions, which will waste a large amount of labor costs.Deep learning has achieved very good results in feature extraction and localization. More and more scholars and engineers begin to introduce deep learning algorithm into the field of defect detection. The following will introduce the applications of several deep learning algorithms in the field of defect detection.

基于深度学习的缺陷检测算法汇总

1、A fast and robust convolutional neural network-based defect detection model in product quality control

论文链接：https://link.springer.com/article/10.1007/s00170-017-0882-0

检测对象：布匹缺陷。

主要思想：这是一篇比较早的文章了，主要通过对输入图像进行切片，然后把切片图像送入深度学习网络中做判断，较为简单。在推理时，通过滑窗检测方式进行逐位置识别。   

• Detection object: cloth defect.

  Main Idea: This is an early article, mainly through the input image slice, and then the slice image into the deep learning network to make judgment, relatively simple.In reasoning, the sliding window detection is used to identify the position by position.

优缺点：

• 由于使用分类方式，准确率较为高

•  由于滑窗遍历，速度慢

• The advantages and disadvantages:

  Due to the use of classification, the accuracy is relatively high

  Due to sliding window traversal, the speed is slow

2、Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network（基于深度卷积网络的接触网支架紧固件缺陷自动检测）

论文链接：https://ieeexplore.ieee.org/document/8126877

检测对象：紧固件是否缺失。

主要思想：检测接触网支架紧固件是否缺陷，采用方式为Object Detection。

• Detection object: Is the fastener missing?

  Main idea: Detect whether catenary bracket fasteners are defective, using Object Detection.

文章主要采用三阶段方式：

• This paper mainly adopts a three-stage approach:

第一阶段：定位主要结构件位置，采用SSD作为检测网络。

• Stage 1: Position the main structural parts and use SSD as the detection network.

第二阶段：对第一阶段检测出的各个结构图像中的紧固件做检测，也是采用Object Detection方式。

• The second stage: The Object Detection method is also used to detect the fasteners in each structure image detected in the first stage.

第三阶段：在第二阶段的基础上，对检测图像进行crop，将其送到分类网络进行分类，看是否缺少紧固件。

• The third stage: on the basis of the second stage, the detected images are cut off with crops and sent to the classification network for classification to see whether fasteners are missing.

思路也算比较简单，但不是端到端，实际应用起来可能比较困难。

• The idea is also relatively simple, but not end-to-end, practical application may be more difficult.

3、Automatic Fabric Defect Detection with a Multi-Scale Convolutional Denoising Autoencoder Network Model（基于多尺度卷积消噪自编码网络模型的织物疵点自动检测）

论文链接：https://pubmed.ncbi.nlm.nih.gov/29614813/

检测对象：纺织物和布匹的瑕疵点检测。

• Detection object: detection of flaw spots in textiles and cloth.

主要思想：织物疵点检测是纺织制造业质量控制中必不可少的环节。传统的织物检测通常采用人工视觉的方法进行，效率低，长期工业应用精度差。论文提出使用高斯金字塔结合语义分割的方式来重建缺陷，推理阶段通过结合多尺度结果，完成融合。这种方法在织物疵点检测中有几个突出的优点。首先，只需少量的无缺陷样本就可以进行训练。这对于收集大量有缺陷样品困难且不可行的情况尤其重要。其次，由于采用了多模态积分策略，与一般的检测方法相比，该方法具有更高的鲁棒性和准确性。第三，根据我们的结果，它可以处理多种类型的纺织面料，从简单到复杂。实验结果表明，该模型具有较强的鲁棒性和良好的整体性能。

• Main idea: Fabric defect detection is an essential link in quality control of textile manufacturing industry.Traditional fabric detection is usually carried out by artificial vision, which has low efficiency and poor precision in long-term industrial application.The paper proposes to reconstruct the defects by using Gauss gold tower combined with semantic segmentation, and to complete the fusion by combining the multi-scale results in the reasoning stage.This method has several outstanding advantages in fabric defect detection.First, training can be done with a small number of defect-free samples.This is especially important when collecting large Numbers of defective samples is difficult and not feasible.Secondly, due to the adoption of multi-mode integration strategy, this method has higher robustness and accuracy than the general detection methods.Thirdly, according to our results, it can handle many types of textile fabrics, from simple to complex.The experimental results show that the model has strong robustness and good overall performance.

模型结构：

 

检测结果：

• result:

 

4、An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces

论文链接：https://ieeexplore.ieee.org/document/8281622

检测对象：作者和上一篇文章相同，思想也非常相像，主要是针对均匀和非规则纹理表面上的缺陷。这里不再多叙述，有兴趣的话可以阅读原论文~

• Detection object: The author is the same as the previous article, and the idea is very similar, mainly aiming at the defects on the surface of uniform and irregular texture.No more description here, if you are interested in reading the original paper ~

5、Automatic Metallic Surface Defect Detection and Recognition with Convolutional Neural Networks

论文链接：https://www.researchgate.net/publication/327494004_Automatic_Metallic_Surface_Defect_Detection_and_Recognition_with_Convolutional_Neural_Networks

检测对象：金属表面缺陷检测

主要思想：本文讨论了用一种能准确定位和分类从实际工业环境中获取的输入图像中出现的缺陷的双重过程来自动检测金属缺陷。设计了一种新的级联自动编码器（CASAE）结构，用于缺陷的分割和定位。级联网络将输入的缺陷图像转化为基于语义分割的像素级预测掩模。利用压缩卷积神经网络（CNN）将分割结果的缺陷区域划分为特定的类。利用工业数据集可以成功地检测出各种条件下的金属缺陷。实验结果表明，该方法满足金属缺陷检测的稳健性和准确性要求。同时，它也可以扩展到其他检测应用中。

其实还是使用语义分割网络先定位像素级别的缺陷位置，然后通过分类网络对缺陷进行识别。

• Detection object: metal surface defect detection

  Main Idea: This paper discusses the automatic detection of metal defects by a dual process that accurately locates and classifies defects in input images obtained from real industrial environments.A new CASAE structure is designed for defect segmentation and location.The cascade network converts the defect image into a pixel level predictive mask based on semantic segmentation.Using compressed convolutional neural network (CNN), the defect regions of the segmentation results are divided into specific classes.Metal defects under various conditions can be successfully detected by using industrial data sets.The experimental results show that the method meets the requirements of robustness and accuracy of metal defect detection.It can also be extended to other detection applications.

  In fact, the semantic segmentation network is still used to locate the defect location at pixel level and then identify the defect through classification network.

 

实验结果：

• result:

 

6、Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types

论文链接：https://onlinelibrary.wiley.com/doi/abs/10.1111/mice.12334

检测对象：混凝土裂缝、中、高两级钢腐蚀、螺栓腐蚀和钢分层五种损伤类型

主要思想：通过Faster RCNN检测网络实现的，偏应用型的一篇文章，效果不是特别好，这里只提供一个检测思路，详细内容请看论文。

• Detection objects: concrete crack, medium and high grade steel corrosion, bolt corrosion and steel stratification five types of damage

  Main idea: An application-oriented article realized through the detecting network of Faster RCNN has not a particularly good effect, and only one detection idea is provided here. Please refer to the paper for detailed content.

 

7、A Surface Defect Detection Method Based on Positive Samples

论文链接：https://link.springer.com/chapter/10.1007/978-3-319-97310-4_54

检测对象：密集织物

主要思想：本文提出了一种新的基于正样本训练的缺陷检测框架。基本检测的概念是建立一个重建网络，它可以修复样本中存在的缺陷区域，然后对输入样本与恢复样本进行比较，以指示缺陷区域的准确。结合GAN和自动编码器进行缺陷图像重建，利用LBP进行图像局部对比度检测缺陷。在算法的训练过程中，只需要正样本，不需要缺陷样本和人工标注。

• Detection object: dense fabric

  Main idea: A new defect detection framework based on positive sample training is proposed.The basic detection concept is to establish a reconstruction network, which can repair the defect area in the sample, and then compare the input sample with the recovery sample to indicate the accuracy of the defect area.GAN and automatic encoder are combined to reconstruct the defect image and LBP is used to detect the defect.In the process of algorithm training, only positive samples are needed, and defect samples and manual annotation are not needed.

主要结构：

 

在训练阶段，x是从训练集中随机拍摄的随机照片。C（x∼| x）是一个人工缺陷模块。它的功能是自动生成损坏的、有缺陷的样本，x∼是它的输出。EN和DE构成一个自动编码器，EN是一个编码器，DE是一个解码器，整个自动编码器在G an模型中可以看作是一个生成器。G的任务是修复有缺陷的图片。D是一个鉴别器，D的输出是它的鉴别器是真正样本的概率。

在测试阶段，我们将测试图像x输入到自动编码器G中，得到恢复图像y，然后使用LBP算法提取x和y的特征，并比较x的每个像素的特征，其中x和y的特征差异较大，即缺陷。

通过GAN的方式来进行缺陷检测和重建一直是个热点方向，而且效果比较好，本文的思路算是比较新颖，值得借鉴。

• In the training phase, X is a random photograph taken at random from the training set.C (x ~ | x) is a manual defect module.Its function is to automatically generate corrupted, defective samples, and x ~ is its output.EN and DE constitute an automatic encoder, EN is an encoder and DE is a decoder. The whole automatic encoder can be regarded as a generator in the G An model.G's job is to fix the defective image.D is a discriminator, and the output of D is the probability that its discriminator is the actual sample.

  In the testing stage, we input the test image X into the automatic encoder G to obtain the recovered image Y, and then extract the features of X and Y using LBP algorithm, and compare the features of each pixel of X, among which the features of X and Y are quite different, namely the defect.

  GAN has always been a hot direction for defect detection and reconstruction, and the effect is relatively good. The idea of this paper is relatively novel and worthy of reference.

实验结果：

8、Segmentation-based deep-learning approach for surface-defect detection

论文链接：https://link.springer.com/article/10.1007/s10845-019-01476-x

检测对象：表面缺陷检测、裂纹检测（金属）

主要思想：本文主要采用了两个网络，一个是判别网络，一个是分割网络。分割网络主要完成缺陷的分割，而判别网络在此基础上对缺陷作进一步分类。

• Detection objects: surface defect detection, crack detection (metal)

  Main idea: This paper mainly USES two networks, one is the discriminant network, one is the segmentation network.The segmentation network mainly completes the segmentation of defects, and the discriminant network further classifies the defects on this basis.

 

算法很新颖，实验结果也还不错：

 

 

9、SDD-CNN: Small Data-Driven Convolution Neural Networks for Subtle Roller Defect Inspection（小数据驱动卷积神经网络在轧辊微小缺陷检测中的应用）

论文链接：https://www.mdpi.com/2076-3417/9/7/1364

检测对象：轧辊微小缺陷检查

主要思想：滚柱轴承是旋转机械中最关键、应用最广泛的部件之一。外观缺陷检测是轴承质量控制的关键。然而，在实际工业中，轴承缺陷往往是极其细微的，并且发生的概率很低。这就导致了正负样本数量的分布差异，使得依靠数据驱动的检测方法难以开发和部署。本文提出了一种用于轧辊微小缺陷检测的小数据驱动卷积神经网络（SDD-CNN）一种用于小数据预处理的集成方法。首先，应用标签膨胀（LD）来解决类分布不平衡的问题。其次，提出了一种半监督数据增广（SSDA）方法，以更有效和可控的方式扩展数据集。该方法通过训练一个粗糙的CNN模型来生成地面真值类激活，并指导图像的随机裁剪。第三，介绍了CNN模型的四种变体，即SqueezeNet v1, Inception v3,VGG-16, and ResNet-18，并将其用于轧辊表面缺陷的检测和分类。最后，进行了一系列丰富的实验和评估，表明SDD-CNN模型，特别是SDD Inception v3模型，在滚筒缺陷分类任务中执行得非常好，top-1精度达到99.56%。此外，与原始CNN模型相比，SDD-CNN模型的收敛时间和分类精度都有显著提高。

• Detection object: small defect inspection of roll

  Main idea: Roller bearing is one of the most critical and widely used parts in rotary machinery.Appearance defect detection is the key of bearing quality control.However, in the actual industry, bearing defects are often extremely small and the probability of occurrence is very low.This leads to the difference in the distribution of positive and negative samples, making it difficult to develop and deploy data-driven detection methods.In this paper, a small data driven convolutional neural network (SDD-CNN) integrated method for small data preprocessing is presented.First, label bloat (LD) is applied to solve the problem of unbalanced class distribution.Secondly, a semi-supervised data enhancement (SSDA) method is proposed to extend the data set in a more effective and controllable way.The method generates ground truth class activation by training a rough CNN model and directs the random cropping of images.Third, four variants of CNN model, namely SqueezeNet V1, Inception V3,VGG-16, and Resnet-18, were introduced and used for the detection and classification of roll surface defects.Finally, a rich series of experiments and assessments have been conducted to show that the SDD-CNN model, in particular the SDD Inception V3 model, performed very well in the drum defect classification task with a top-1 accuracy of 99.56%.In addition, compared with the original CNN model, the convergence time and classification accuracy of SDD-CNN model are significantly improved.

网络结构：

实验结果：

• result