《计算机视觉度量：从特征描述到深度学习》-- 传统差分模板(Variation Model)进行检测设计

军哥讲视觉

已于 2023-09-01 11:07:52 修改

阅读量502

点赞数 3

文章标签：计算机视觉人工智能

于 2023-09-01 10:50:03 首次发布

本文链接：https://blog.csdn.net/qq_17767255/article/details/132618136

版权

差分模板是以比对为核心理念的设计方法，通过设计检测产品的正常模板，尤其是刚性特征（比如印刷品的字符，工件的刚性轮廓，包括明显的特征区域），和产品线上生产的产品进行比对。检测出产品的缺陷。

适配的行业：印刷，PCB检测，结构件检测等行业。是目前行业检测的最核心的方法

原理是将待检测的图像与一张标准图像作比较，找出待检测图像与标准图像（ideal image）的明显差异（也就是不良）。标准图像可以采用几张OK品的图像训练（training）得到，也可以通过对一张OK品图像进行处理得到。训练后得到标准图像和一张variation图像（variation image），variation图像中包含了图像中每个像素点灰度值允许变化的范围。标准图像和variation图像用来创建一个variation model，如此，其他图像就可以与variation model作比较了。

下面是一个简单的halcon例子（print_check_single_chars.hdev）：

* This example program shows how to use HALCON's variation model operators
* to perform a typical print quality inspection.  The program detects incorrect
* prints on the clips of pens.  The program is similar to the program print_check.hdev.
* The main difference is that each character is checked separately.  This is not
* a particularly good strategy for this application because it does not detect
* errors that lie on the clip itself and errors because the V moves with respect
* to the other characters.  In some applications, however, precisely this
* invariance is useful.  This program can be used as a basis for such
* applications.  In the first step, variation models for each character are
* constructed from images of correct prints.  Since the position of the objects can
* vary, the images of the different characters must be transformed to a reference
* position (the position of the character in the variation model).  The position of
* the character in the model is given by selecting a part of the first image that
* to the character and a buffer around the character from the first image.  The
* operator affine_trans_image_size is used to perform this task efficiently.
* HALCON's shape-based matching is used to detect the pose of each
* character in the images in a single call to find_shape_models.  The found
* poses are used to transform the images of the different characters to their
* respective reference positions.  In the second part of the program, the prints
* of the correct clips and of several incorrect clips is checked and classified.
dev_update_off ()
read_image (Image, 'pen/pen-01')
get_image_size (Image, Width, Height)
dev_close_window ()
dev_open_window (0, 0, Width, Height, 'black', WindowHandle)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
dev_set_color ('red')
dev_display (Image)
* Note: the shape models will be constructed from ROIs that are computed
* automatically based on a simple image segmentation.
threshold (Image, Region, 100, 255)
fill_up (Region, RegionFillUp)
difference (RegionFillUp, Region, RegionDifference)
shape_trans (RegionDifference, RegionTrans, 'convex')
dilation_circle (RegionTrans, RegionDilation, 8.5)
reduce_domain (Image, RegionDilation, ImageReduced)
threshold (ImageReduced, Region, 0, 180)
connection (Region, ConnectedRegions)
sort_region (ConnectedRegions, SortedRegions, 'character', 'true', 'row')
dilation_circle (SortedRegions, RegionDilation, 1.5)
smallest_rectangle1 (RegionDilation, Row1, Column1, Row2, Column2)
count_obj (RegionDilation, Number)
Heights := Row2 - Row1 + 5
Widths := Column2 - Column1 + 5
gen_empty_obj (ShapeModels)
gen_empty_obj (VariationModelROIs)
ShapeModelIDs := []
VariationModelIDs := []
RowsRef := []
ColumnsRef := []
for I := 1 to Number by 1
    select_obj (RegionDilation, ObjectSelected, I)
    Height := Heights[I - 1]
    Width := Widths[I - 1]
    move_region (ObjectSelected, RegionMoved, -Row1[I - 1], -Column1[I - 1])
    crop_part (Image, ImagePart, Row1[I - 1], Column1[I - 1], Widths[I - 1], Heights[I - 1])
    reduce_domain (ImagePart, RegionMoved, ImageReduced)
    inspect_shape_model (ImageReduced, ModelImages, ModelRegions, 1, [15,15,10])
    gen_contours_skeleton_xld (ModelRegions, ModelContour, 1, 'filter')
    area_center (RegionMoved, Area, RowRef, ColumnRef)
    create_shape_model (ImageReduced, 5, rad(-10), rad(20), 'auto', 'none', 'use_polarity', 20, 10, ShapeModelID)
    create_variation_model (Width, Height, 'byte', 'standard', VariationModelID)
    concat_obj (ShapeModels, ModelContour, ShapeModels)
    concat_obj (VariationModelROIs, RegionMoved, VariationModelROIs)
    ShapeModelIDs := [ShapeModelIDs,ShapeModelID]
    VariationModelIDs := [VariationModelIDs,VariationModelID]
    RowsRef := [RowsRef,RowRef]
    ColumnsRef := [ColumnsRef,ColumnRef]
endfor
* Visualize the different shape models.
gen_empty_obj (Models)
for I := 1 to Number by 1
    select_obj (ShapeModels, ModelSelected, I)
    hom_mat2d_identity (HomMat2DIdentity)
    hom_mat2d_translate (HomMat2DIdentity, Row1[I - 1], Column1[I - 1], HomMat2D)
    affine_trans_contour_xld (ModelSelected, ModelTrans, HomMat2D)
    concat_obj (Models, ModelTrans, Models)
endfor
dev_display (Image)
dev_set_colored (6)
dev_set_draw ('margin')
dev_set_line_width (3)
dev_display (Models)
dev_set_color ('yellow')
disp_message (WindowHandle, 'Shape models', 'image', 12, 20, 'yellow', 'false')
disp_message (WindowHandle, '(moved to original pos.)', 'image', 45, 20, 'yellow', 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* Train the different variation models.
for J := 1 to 15 by 1
    read_image (Image, 'pen/pen-' + J$'02d')
    find_shape_models (Image, ShapeModelIDs, rad(-10), rad(20), 0.5, [1,1,1,1,1], 0.5, 'least_squares', 0, 0.9, Row, Column, Angle, Score, Model)
    for K := 0 to |Score| - 1 by 1
        vector_angle_to_rigid (Row[K], Column[K], Angle[K], RowsRef[Model[K]], ColumnsRef[Model[K]], 0, HomMat2D)
        affine_trans_image_size (Image, ImageTrans, HomMat2D, 'constant', Widths[Model[K]], Heights[Model[K]])
        select_obj (VariationModelROIs, ROI, Model[K] + 1)
        reduce_domain (ImageTrans, ROI, ImageTransReduced)
        train_variation_model (ImageTransReduced, VariationModelIDs[Model[K]])
    endfor
endfor
* Visualize the variation models' data.
get_image_size (Image, Width, Height)
gen_empty_obj (MeanImages)
gen_empty_obj (VarImages)
gen_empty_obj (ROIs)
for I := 1 to Number by 1
    get_variation_model (MeanImage, VarImage, VariationModelIDs[I - 1])
    prepare_variation_model (VariationModelIDs[I - 1], 20, 3)
    select_obj (VariationModelROIs, ROI, I)
    move_region (ROI, ROIMoved, Row1[I - 1], Column1[I - 1])
    reduce_domain (MeanImage, ROI, MeanImageReduced)
    reduce_domain (VarImage, ROI, VarImageReduced)
    concat_obj (MeanImages, MeanImageReduced, MeanImages)
    concat_obj (VarImages, VarImageReduced, VarImages)
    concat_obj (ROIs, ROIMoved, ROIs)
    * We can now free the training data to save some memory.
    clear_train_data_variation_model (VariationModelIDs[I - 1])
endfor
tile_images_offset (MeanImages, MeanImage, Row1, Column1, gen_tuple_const(Number,-1), gen_tuple_const(Number,-1), gen_tuple_const(Number,-1), gen_tuple_const(Number,-1), Width, Height)
tile_images_offset (VarImages, VarImage, Row1, Column1, gen_tuple_const(Number,-1), gen_tuple_const(Number,-1), gen_tuple_const(Number,-1), gen_tuple_const(Number,-1), Width, Height)
get_domain (Image, Domain)
dev_set_color ('black')
dev_set_draw ('fill')
dev_display (Domain)
dev_display (MeanImage)
dev_set_colored (6)
dev_set_draw ('margin')
dev_set_line_width (1)
dev_display (ROIs)
dev_set_color ('yellow')
disp_message (WindowHandle, 'Reference images', 'image', 20, 20, 'yellow', 'false')
disp_message (WindowHandle, '(moved to original pos.)', 'image', 45, 20, 'yellow', 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
dev_set_color ('black')
dev_set_draw ('fill')
dev_display (Domain)
dev_display (VarImage)
dev_set_colored (6)
dev_set_draw ('margin')
dev_set_line_width (1)
dev_display (ROIs)
dev_set_color ('yellow')
disp_message (WindowHandle, 'Variation images', 'image', 20, 20, 'yellow', 'false')
disp_message (WindowHandle, '(moved to original pos.)', 'image', 45, 20, 'yellow', 'false')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
* Find the printing errors.
for J := 1 to 30 by 1
    read_image (Image, 'pen/pen-' + J$'02d')
    gen_empty_obj (RegionsErrorTrans)
    find_shape_models (Image, ShapeModelIDs, rad(-10), rad(20), 0.5, [1,1,1,1,1], 0.5, 'least_squares', 0, 0.9, Row, Column, Angle, Score, Model)
    NumberFound := |Score|
    for K := 0 to NumberFound - 1 by 1
        vector_angle_to_rigid (Row[K], Column[K], Angle[K], RowsRef[Model[K]], ColumnsRef[Model[K]], 0, HomMat2D)
        affine_trans_image_size (Image, ImageTrans, HomMat2D, 'constant', Widths[Model[K]], Heights[Model[K]])
        select_obj (VariationModelROIs, VariationModelROI, Model[K] + 1)
        reduce_domain (ImageTrans, VariationModelROI, ImageTransReduced)
        compare_variation_model (ImageTransReduced, RegionDiff, VariationModelIDs[Model[K]])
        connection (RegionDiff, ConnectedRegions)
        select_shape (ConnectedRegions, RegionsError, 'area', 'and', 20, 1000000)
        count_obj (RegionsError, NumError)
        if (NumError > 0)
            vector_angle_to_rigid (RowsRef[Model[K]], ColumnsRef[Model[K]], 0, Row[K], Column[K], Angle[K], HomMat2D)
            affine_trans_region (RegionsError, RegionErrorTrans, HomMat2D, 'nearest_neighbor')
            concat_obj (RegionsErrorTrans, RegionErrorTrans, RegionsErrorTrans)
        endif
    endfor
    dev_clear_window ()
    dev_display (Image)
    dev_set_color ('red')
    dev_display (RegionsErrorTrans)
    count_obj (RegionsErrorTrans, NumError)
    if (NumError == 0 and NumberFound == Number)
        disp_message (WindowHandle, 'Clip OK', 'image', 20, 20, 'green', 'false')
    else
        disp_message (WindowHandle, 'Clip not OK', 'image', 20, 20, 'red', 'false')
    endif
    if (J < 30)
        disp_continue_message (WindowHandle, 'black', 'true')
        stop ()
    endif
endfor
for I := 1 to Number by 1
    clear_shape_model (ShapeModelIDs[I - 1])
    clear_variation_model (VariationModelIDs[I - 1])
endfor

Variation Model方法中常用的算子：

create_variation_model( : : Width, Height, Type, Mode : ModelID)

说明：创建一个ID为ModelID，宽为Width,高为Height,类型为Type的Variation Model，参数Mode决定了创建标准图像和相应的variation图像的方法。'standard'表示标准的训练方法，标准图像的位置是各训练图像位置的平均，'robust'表示鲁棒的训练方法，标准图像的位置是各训练图像的中值，此模式在训练图像中可能存在ERROR时使用，'direct'表示标准图像由单张图像经过处理得到，由此方法得到的标准图像只能应用prepare_direct_variation_model算子得到variation model。

train_variation_model(Images : : ModelID : )

说明：训练一个Variation Model。

get_variation_model( : Image, VarImage : ModelID : )

说明：返回variation model中的标准图像（Image）和variation image（VarImage），此算子主要用来检视创建的variation model是否OK。

prepare_variation_model( : : ModelID, AbsThreshold, VarThreshold : )

说明：设置variation model的绝对阈值和相对阈值。绝对阈值即待检测图像与标准图像的差值，相对阈值即待检测图像与variation model与VarThreshold乘绩的差值。

clear_train_data_variation_model( : : ModelID : )

说明：清除variation model的训练数据所占用的内存。

compare_variation_model(Image : Region : ModelID : )

说明：待检测图像与variation model进行比较，超过阈值的区域在Rgion参数中返回。同threshold一样，返回的区域被看做一个区域，可以使用connection算子进行连通性分析，然后根据区域的特征（如面积）对区域进行选择。

clear_variation_model( : : ModelID : )

说明：释放一个variation model的内存空间

read_variation_model( : : FileName : ModelID)

说明：读取一个差分模板

write_variation_model( : : FileName : ModelID)

说明：写入一个差分模板

Variation Model使用标准图像与待检测图像灰度值相比较，来判断产品是否OK，适用于印刷品检测及产品表面检测。从实际算法过程可以看出，此检测实际可分为两部分，对于图像中的大面积灰度一致区域，主要利用待检测图像与标准图像（ideal image)比较得出差异区域，对于图像中的边缘位置（edges）区域，主要利用待检测图像与Variation图像（variation image）比较得出差异区域。所以在实际应用中，应根据实际情况设置AbsThreshold和VarThreshold的值。