openpnp - Default Pipeline

openpnp - Default Pipeline

概述

元件的底部相机视觉识别有点问题，看看官方关于视觉管道的文档。
将可以添加的Pipeline命令熟悉一下。

笔记

Pipeline - 官方文档上的介绍

ImageCapture

ImageCapture: Waits for the camera to settle and captures an image.
等待相机稳定下来并捕捉图像

ImageWriteDebug

ImageWriteDebug: Writes the input image to a file on disk to help with debugging.
将图像写入磁盘，用来帮助调试图像识别

BlurGaussian

BlurGaussian: Performs minor blurring on the input image. This is used to reduce noise in the image.
对输入图像执行轻微模糊。这是用来减少图像中的噪声

MaskCircle

MaskCircle: Blacks out everything outside of a circle of a given diameter. On the development machine this circle represents a “safe” area in the image where nothing is visible except the nozzle.
把给定直径的圆圈外的一切都抹掉(不参与图像识别)

ConvertColor

ConvertColor: Convert from RGB color to HSV color, which is required for the next stage.
从RGB颜色转换为HSV颜色, 将结果作为输入给后续命令

MaskHsv

MaskHsv: Searches the image for any pixels that match a certain hue (the H in HSV) and turns them black. The purpose of this is to remove green and “greenish” pixels from the image. Green is the color of the nozzle holder. This is similar to the concept of “green screening”. See also the Nozzle Tip Background Calibration.
在图像中搜索任何匹配某种色调（HSV中的H）的像素，并将它们变为黑色。

ConvertColor

ConvertColor: Convert back from HSV to RGB. This is required by the next stage.
从HSV转换回RGB，将结果作为输入给后续命令

ConvertColor

ConvertColor: Convert from RGB to grayscale.
从RGB转换到灰度

Threshold

Threshold: Turns the image into a binary image - meaning that it has only two colors: white and black. Any gray pixels that are darker than the threshold value turn black and any lighter turn white.
将图像转换为二进制图像-这意味着它只有两种颜色：白色和黑色。任何比阈值更暗的灰度像素都变成黑色，任何更亮的像素都变成白色

FindContours

FindContours: Find connected contours in the image. Contours are a way to describe simple features in an image such as lines and curves.

在图像中找到连接的轮廓(e.g. 直线，曲线)

FilterContours

FilterContours: Removes any contours from the previous stage that are smaller than a specified value. This helps remove noise and features that don’t pertain to the main contour around the part.
从命令输入中，删除小于指定值的任何轮廓(用来消除消除噪音和特征，e.g. 不想参与图像识别的东西(吸嘴上方电机的4个安装螺丝))

SetColor

SetColor: Sets the entire image to black. This simply provides a blank canvas for the next stage to draw on.
将整个图像设置为黑色作为输出。这只是为下一个命令提供了一个空白画布

DrawContours

DrawContours: Draw all the of the remaining contours in white on the black background. At this point we hope that we’re simply drawing the shape of the part only.
在黑色背景上用白色绘制所有剩余的轮廓

MinAreaRect

MinAreaRect: This is where the magic happens! MinAreaRect creates a RotatedRect that fits around any non-black pixels in the image. Since we drew contours representing the part this now finds the bounds and rotation of the part.
将所有非黑色的像素创建出来一个旋转的矩形(这个矩形就代表了找到的元件外形)

ImageRecall

ImageRecall: Recalls the original input image so that we can show the user the results of all this work.
重新载入原始图像(用于在原始图像上显示找到的元件外形的前置操作)

DrawRotatedRects

DrawRotatedRects: Draw the RotatedRect in red overtop the recalled original image. If all went well we should now see the original input image with a red rectangle surrounding the part.
在当前图像上画找到的元件的矩形(旋转矩形)，默认是红色的框。

ImageWriteDebug

ImageWriteDebug: Writes the resulting image out to a file for help with debugging.
将结果图像保存为图片，用于调试。

Pipeline - 整理在CvPipeLineEditor中可以添加的命令的大致含义

官方文档中没介绍的opencv管道命令, 应该是不常用的。
不过大概看一下，万一视觉识别出了问题，那么也许能加上试一试。
最不济的效果，看到官方给出的默认视觉识别命令组合时，也能明白官方为啥用这些opencv命令做组合处理了。
如果暂时不知道具体咋用，也没关系，那就暂时不用。
只要是翻资料，总能挖到一些好东西。

ImageRead

ImageRead - Replace the working image with the image loaded from a given path.
将当前图像替换为从全路径文件中载入的图像(这可以用于没有设备情况下的调试, e.g. 人家设备上底部相机视觉识别出了问题，他在ImageCapture的下一个命令中用ImageWriteDebug将图像保存起来给能调试视觉识别参数的人，帮他调试视觉识别参数)

Add

Add - Adds two images together, scale either, or subtract second.
将2个图像叠加在一起显示成当前的图像

AffineWarp

AffineWarp - Extracts a rectangular (parallelogrammatic) region of interest from the image that can have any position, size, rotation, scale, shear or even be mirrored. A so-called Affine Transformation Warp. The coordinates are given in real length units rather than pixels and are relative to the camera center, Y pointing up. This allows the pipeline to be independent of camera model and resolution, lens, focus distance etc. To setup, pin the previous stage and read off length unit coordinates from the mouse position.

从图像中提取感兴趣的矩形（平行四边形）区域，该区域可以具有任何位置、大小、旋转、缩放、剪切甚至镜像。这就是所谓的仿射变换翘曲

AffineUnwarp

AffineUnwarp - Result model coordinates obtained from images gone through AfficeWarp are not usable as camera coordinates. This stage applies the proper reverse Affine Transformation to reconstruct camera coordinates. The stage currently supports Lists or single instances of Circles, RotatedRects and KeyPoints. For transformations with stretch and shear, some of the model properties are approximated.

AffineWarp的输出作为输入，重建相机坐标。

BlurGaussian

Performs gaussian blurring on the working image.
对工作图像执行高斯模糊

BlurMedian

Performs median blurring on the working image.
对工作图像执行中值模糊

ClosestModel

Filter RotatedRects that fit in the template’s size, with a tolerance and find the closest RotatedRect to the center of the screen.
过滤适合模板大小的RotatedRect，带一个公差，并找到离屏幕中心最近的RotatedRect。

ComposeResult

Retrieves the result of a user defined stage, or the working result in the pipeline, when this stage is processed.
在处理该阶段时，检索用户定义阶段的结果或管道中的工作结果。

ConvertColor

Converts the internal representation of an image from one color space to another. Note that the conversions (other than to/from gray) do not alter the colors in the image but rather change the underlying numerical represention of the colors such that the preceived colors in the new color space are the same as those in the old color space. See https://en.wikipedia.org/wiki/Color_space for a detailed explaination of color spaces.

将图像的颜色标准转换为另外一种(e.g. rgb_to_gray, 就可以将彩色图像转为灰度图)
如果将已经转为灰度图的当前图像再转成rgb, 是没反应的，因为灰度图中没有彩色图像信息。

ConvertModelToKeyPoints

Convert a variety of built in types to KeyPoints. Currently handles Points, Circles, TemplateMatches, and RotatedRects. The center point of each is stored, along with a score and diameter where appropriate. If the input model is a single value the result will be a single value. If the input is a List the result will be a List.

将各种内置类型转换为关键点。目前处理点，圆，模板匹配，和旋转矩形。每个的中心点都被存储，并在适当的地方存储分数和直径。如果输入模型是单个值，则结果将是单个值。如果输入是List，则结果将是List。

ConvertModelToPoints

Convert a variety of built in types to Points. Currently handles KeyPoints, Circles and RotatedRects. The center point of each is stored. If the input model is a single value the result will be a single value. If the input is a List the result will be a List.
将各种内置类型转换为点。目前处理关键点，圆圈和旋转矩形。每个点的中心点被存储。如果输入模型是单个值，则结果将是单个值。如果输入是List，则结果将是List。

CreateFootprintTemplateImage

Creates a template from the specified footprint and camera properties. The template is scaled to the camera’s units.

建立封装模板图像，根据指定的内存占用和相机属性创建模板。模板被缩放到相机的单位。

CreateModelTemplateImage

Create a cropped template image in portrait orientation based on a model.
基于模型创建纵向裁剪的模板图像

CreateShapeTemplateImage

Creates a template from the specified shape and camera properties. The shape is scaled from Millimeters to the camera’s units.
根据指定的形状和相机属性创建模板。形状从毫米缩放到相机的单位。

DetectCirclesHough

Finds circles in the working image. Diameter and spacing can be specified.
在工作图像中查找圆。直径和间距可以指定。

DetectCircularSymmetry

Finds circular symmetry in the working image. Diameter range and maximum search distance can be specified.
在工作图像中找到圆对称。可以指定直径范围和最大搜索距离。

DetectEdgesRobertsCross

直接一步到位，将元件的边沿用黑白图像显示处理。但是可能会有一些噪声图像(e.g. 28电机上的固定螺丝)

DetectFixedCirclesHough

在工作图像中查找圆。直径和间距由管道提供。

DetectRectlinearSymmetry

Finds the subject and angle with maximum rectlinear symmetry in the working image. This is achieved by first computing reclinear, i.e. horizontal and vertical cross-sections at various angles. The angle with the largest contrasts is then used. As a second step, the mid-points with the largest symmetry in the horizontal and vertical cross-sections is determined to detect the subject center.

在工作图像中找到具有最大直线对称性的主体和角度。这是通过首先计算斜线，即不同角度的水平和垂直截面来实现的。然后使用反差最大的角度。作为第二步，在水平和垂直截面中确定对称性最大的中点来检测主体中心。

DilateModel

Dilate or contract a model by a given number of pixels.
将模型放大或缩小给定的像素数。

DrawImageCenter

Draw a mark at the center of the image.
在图像的中心画一个标记

不看了

openpnp带的命令太多了，且大多数看不出啥效果(可能有上下文)，等遇到了再实验。
只要知道开启CvPipeLineEditor来调试由openpnp保存出来的图片文件就行。具体效果再去拿实际的图片和openpnp官方给的CvPipeLineEditor组合去调试学习

总结

有些命令执行后，没有效果(e.g. 高斯模糊). 不要怀疑没效果，只要图像下面的结果窗口没有报错信息，就是已经执行了。
只是有些处理的结果没有显示出来，或者是显示出来了，但是人眼分辨不出来。

有些命令是要和真实的摄像头联系起来的，如果只是开了单独的CvPipeLineEditor来在载入的图像上研究openpnp提供的opencv命令，那么有些命令执行不了，抛出异常。只能是先看一下命令大概啥意思。

对于完全没有官方注释的命令, 如果能看出是啥意思，也记录一下(e.g. DetectEdgesRobertsCross)。

有一些命令只依赖图像，但是运行抛异常(也有可能这些命令本身有bug)。只能以后在官方例子中看到才直到具体怎么用。

还有些命令，是依赖输入图像或数据的，如果没有这些数据, 也不报错，只是没效果(e.g. DrawKeyPoints, 如果输入的是原始图像，就没有KeyPoints数据，也就画不出任何东西)

DrawXX 是画东西，执行命令时，工作区的图像会有视觉上的变化
DetectXX 是检测特征点，并将特征点用线在二值图像上画出来。

备注

以后有体会再来补充, 这次先大概翻一下。

END