基于Hlcon和C#联合编程的九眼标定法

白纸道

已于 2023-03-14 14:11:01 修改

阅读量2k

点赞数 3

分类专栏：图像识别算法文章标签：计算机视觉算法 Powered by 金山文档

于 2023-03-13 17:10:22 首次发布

本文链接：https://blog.csdn.net/antonw163com/article/details/129498186

版权

图像识别算法专栏收录该内容

2 篇文章 1 订阅

订阅专栏

说明：本次内容基于九眼标定算法的理论进行算法设计，并通过halcon和C#联合编程的办法进行APP端的设计

一、主要内容

九眼标定算法理论说明

halcon代码说明

C#代码说明

二、九眼标定算法理论说明

本次算法通过视觉识别的9个特征点位置和机器人在空间中得到的对应的9个点的位置，通过最小二乘法得到齐次变换矩阵，再将齐次变换矩阵左乘对应的像素坐标点，这样即可得到在机器人坐标系下的像素点

以最小二乘法在二维图像配准的情况下说明(网上有很多关于最小二乘法的理论详解,本次内容主要是介绍代码端的实现)：

1.图像配准简介：

图像配准是指将两幅或多幅图像对齐的过程，使它们具有相同的几何形状、大小和位置。图像配准是图像处理和计算机视觉领域中的一个重要问题，它在很多应用中都有重要的作用，如医学影像分析、遥感图像分析、计算机辅助设计等。

图像配准可以通过两种方法实现：基于特征的配准和基于区域的配准。

基于特征的配准是通过识别两幅图像中的关键特征点（如角点、边缘、斑点等），然后将这些特征点匹配起来，从而确定两幅图像的变换关系，进而实现图像的配准。常见的基于特征的配准算法包括SIFT、SURF、ORB等。

基于区域的配准是通过匹配两幅图像中的像素区域来实现图像的配准。这种方法通常需要定义一种相似性度量来评估两幅图像之间的相似度，然后通过优化这个相似性度量来确定图像的变换关系。基于区域的配准算法包括互信息（MI）、归一化互相关（NCC）等。

无论是基于特征的配准还是基于区域的配准，都有其适用的场合和限制，需要根据具体情况选择合适的算法来实现图像配准。

2.最小二乘法的一般情况(矩阵表达)

假设有模型变量 $\text{[math]}$ ,则可用线性函数表示如下：

对n个样本来说，可以用如下线性方程组来表示:

将式(2)转化为矩阵形式，如下：

则（3）式可以表达为:

则根据最小二乘法计算参数 $\text{[math]}$ ，有如下公式

则该参数的最优解为：

3.最小二乘法在图像配准中的应用

简要流程

选择特征点：从基准图像和待配准图像中选择一些具有代表性的特征点。

特征点匹配：对选择的特征点进行匹配，建立两幅图像之间的对应关系。

计算变换矩阵：根据对应关系，利用最小二乘法求解一个变换矩阵，将待配准图像映射到基准图像坐标系中。

图像配准：将待配准图像按照变换矩阵映射到基准图像坐标系中，完成图像配准。

算法说明

我们记待配准图像中像素特征点为( $\text{[math]}$ ),变换后基准图像像素点坐标为( $\text{[math]}$ ),则根据坐标变换的原理，有如下公式:

我们假设识别特征点数为3个，根据(6)式子我们可以得到如下式子：

我们记伪逆函数为Pinv(Matrix m)并根据式(5)(6)可以得到如下式子：

根据(8)式得到参数矩阵

最后我们将待配准图像中所有像素点乘以该参数矩阵，就可以完成图像配准

二、halcon代码

啥也不说了，直接上代码(简洁好用)！！！

dev_clear_window()
dev_open_window (0, 0, 512, 512, 'black', WindowHandle)
px:=[]
py:=[]
Qx :=[0,28,56,0,28,56,0,28,56]
Qy :=[0,0,0,28,28,28,56,56,56]
read_image (Image, 'C:/Users/Pan/Desktop/8.png')
threshold (Image, Region, 0, 40)
dilation_circle (Region, RegionDilation, 5)
erosion_circle (RegionDilation, RegionErosion, 4)
closing_circle (RegionErosion, RegionClosing, 5)
connection (RegionClosing, ConnectedRegions)
select_shape (ConnectedRegions, SelectedRegions, ['area','row','column','circularity'], 'and', [86111.1,283.33,226.85,0.86204], [113519,1935.19,2791.67,0.94722])
area_center (SelectedRegions, Area, Row, Column)
px:=Row
py:=Column
vector_to_hom_mat2d (px,py,Qx,Qy,HomMat2D)
affine_trans_point_2d (HomMat2D,px,py, Qx, Qy)

二、C#代码

界面图

代码

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using HalconDotNet;

namespace HandEye_Calibration
{
    public partial class Form1 : Form
    {
        HTuple hv_AcqHandle = new HTuple(), hv_Width = new HTuple();
        HTuple hv_Height = new HTuple();
        HObject ho_Image = null;
        HObject ho_Image2 = null;
        HTuple hv_WindowHandle = new HTuple();
        HTuple hv_WindowHandle1 = new HTuple();
        // Local control variables 
        HObject ho_Image1 = null;

        //
        HObject ho_Region, ho_RegionDilation;
        HObject ho_RegionErosion, ho_RegionClosing, ho_ConnectedRegions;
        HObject ho_SelectedRegions;

        // Local control variables 

        HTuple hv_px = new HTuple();
        HTuple hv_py = new HTuple(), hv_Qx = new HTuple(), hv_Qy = new HTuple();
        HTuple hv_Area = new HTuple(), hv_Row = new HTuple(), hv_Column = new HTuple();
        HTuple hv_HomMat2D = new HTuple();
        //

        // Local control variables 
        double[] m_px = new double[9];
        double[] m_py0 = new double[3];
        double[] m_py1 = new double[3];
        double[] m_py2 = new double[3];
        double[] m_py_min1 = new double[3];
        double[] m_py_min2 = new double[3];
        double[] m_py_min3 = new double[3];

        public Form1()
        {
            InitializeComponent();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            HOperatorSet.GenEmptyObj(out ho_Image);
            //Image Acquisition 01: Code generated by Image Acquisition 01
            hv_AcqHandle.Dispose();
            HOperatorSet.OpenFramegrabber("DirectShow", 1, 1, 0, 0, 0, 0, "default", 8, "rgb",
                -1, "false", "default", "[0] ", 0, -1, out hv_AcqHandle);
            HOperatorSet.GrabImageStart(hv_AcqHandle, -1);

        }

        private void timer1_Tick(object sender, EventArgs e)
        {
            ho_Image.Dispose();
            HOperatorSet.GrabImageAsync(out ho_Image, hv_AcqHandle, -1);
            HOperatorSet.GetImageSize(ho_Image, out hv_Width, out hv_Height);
            if (HDevWindowStack.IsOpen())
            {
                HOperatorSet.SetPart(HDevWindowStack.GetActive(), 0, 0, hv_Height, hv_Width);
            }
            if (HDevWindowStack.IsOpen())
            {
                HOperatorSet.DispObj(ho_Image, hv_WindowHandle);
            }            
        }

        private void button4_Click(object sender, EventArgs e)
        {
            HOperatorSet.CloseFramegrabber(hv_AcqHandle);
            ho_Image.Dispose();
        }

        private void button5_Click(object sender, EventArgs e)
        {
            HOperatorSet.CloseWindow(hv_WindowHandle1);
            HOperatorSet.OpenWindow(0, 0, hWindowControl2.Width, hWindowControl2.Height, hWindowControl2.HalconWindow, "visible", "", out hv_WindowHandle1);
            HDevWindowStack.Push(hv_WindowHandle1);
            if (HDevWindowStack.IsOpen())
            {
                //显示图片
                HOperatorSet.DispObj(ho_Image, hv_WindowHandle1);
            }
            ho_Image2 = ho_Image;
            HOperatorSet.WriteImage(ho_Image2, "png", 0, "C:/Users/Pan/Desktop/8.png");
        }

        private void button3_Click(object sender, EventArgs e)
        {
            HOperatorSet.GenEmptyObj(out ho_Image2);
            HOperatorSet.GenEmptyObj(out ho_Region);
            HOperatorSet.GenEmptyObj(out ho_RegionDilation);
            HOperatorSet.GenEmptyObj(out ho_RegionErosion);
            HOperatorSet.GenEmptyObj(out ho_RegionClosing);
            HOperatorSet.GenEmptyObj(out ho_ConnectedRegions);
            HOperatorSet.GenEmptyObj(out ho_SelectedRegions);

            hv_px.Dispose();
            hv_px = new HTuple();
            hv_py.Dispose();
            hv_py = new HTuple();
            hv_Qx.Dispose();
            hv_Qx = new HTuple();
            if (text_x1.Text == "")
            {
                MessageBox.Show("请按顺序输入机器人坐标!");
                return;
            }
            hv_Qx = QX();
            hv_Qy.Dispose();
            hv_Qy = QY();

            HOperatorSet.ReadImage(out ho_Image2, "C:/Users/Pan/Desktop/8.png");
            ho_Region.Dispose();
            HOperatorSet.Threshold(ho_Image2, out ho_Region, 0, 40);

            ho_RegionDilation.Dispose();
            HOperatorSet.DilationCircle(ho_Region, out ho_RegionDilation, 5);
            ho_RegionErosion.Dispose();
            HOperatorSet.ErosionCircle(ho_RegionDilation, out ho_RegionErosion, 4);
            ho_RegionClosing.Dispose();
            HOperatorSet.ClosingCircle(ho_RegionErosion, out ho_RegionClosing, 5);
            ho_ConnectedRegions.Dispose();
            HOperatorSet.Connection(ho_RegionClosing, out ho_ConnectedRegions);
            ho_SelectedRegions.Dispose();
            HOperatorSet.SelectShape(ho_ConnectedRegions, out ho_SelectedRegions, (((new HTuple("area")).TupleConcat(
                "row")).TupleConcat("column")).TupleConcat("circularity"), "and", (((new HTuple(86111.1)).TupleConcat(
                283.33)).TupleConcat(226.85)).TupleConcat(0.86204), (((new HTuple(113519)).TupleConcat(
                1935.19)).TupleConcat(2791.67)).TupleConcat(0.94722));
            hv_Area.Dispose(); hv_Row.Dispose(); hv_Column.Dispose();
            HOperatorSet.AreaCenter(ho_SelectedRegions, out hv_Area, out hv_Row, out hv_Column);
            hv_px.Dispose();
            hv_px = new HTuple(hv_Row);
            hv_py.Dispose();
            hv_py = new HTuple(hv_Column);
            hv_HomMat2D.Dispose();
           

            //求解手眼标定矩阵
            try
            {
                //-------------------------------排序，并按顺序输出数据------------------------------//
                m_px = hv_px;
                m_py0[0] = hv_py[0];
                m_py0[1] = hv_py[1];
                m_py0[2] = hv_py[2];

                m_py1[0] = hv_py[3];
                m_py1[1] = hv_py[4];
                m_py1[2] = hv_py[5];

                m_py2[0] = hv_py[6];
                m_py2[1] = hv_py[7];
                m_py2[2] = hv_py[8];

                m_py_min1 = Min(m_py0);
                m_py_min2 = Min(m_py1);
                m_py_min3 = Min(m_py2);

                for (int i = 0; i < 3; i++)
                {
                    m_py_min2[i] = m_py_min2[i] + 3;
                    m_py_min3[i] = m_py_min3[i] + 6;
                }

                hv_px[0] = m_px[Convert.ToInt32(m_py_min1[0])];
                hv_px[1] = m_px[Convert.ToInt32(m_py_min1[1])];
                hv_px[2] = m_px[Convert.ToInt32(m_py_min1[2])];
                hv_px[3] = m_px[Convert.ToInt32(m_py_min2[0])];
                hv_px[4] = m_px[Convert.ToInt32(m_py_min2[1])];
                hv_px[5] = m_px[Convert.ToInt32(m_py_min2[2])];
                hv_px[6] = m_px[Convert.ToInt32(m_py_min3[0])];
                hv_px[7] = m_px[Convert.ToInt32(m_py_min3[1])];
                hv_px[8] = m_px[Convert.ToInt32(m_py_min3[2])];

                hv_py[0] = m_py0[Convert.ToInt32(m_py_min1[0])];
                hv_py[1] = m_py0[Convert.ToInt32(m_py_min1[1])];
                hv_py[2] = m_py0[Convert.ToInt32(m_py_min1[2])];
                hv_py[3] = m_py1[Convert.ToInt32(m_py_min2[0]) - 3];
                hv_py[4] = m_py1[Convert.ToInt32(m_py_min2[1]) - 3];
                hv_py[5] = m_py1[Convert.ToInt32(m_py_min2[2]) - 3];
                hv_py[6] = m_py2[Convert.ToInt32(m_py_min3[0]) - 6];
                hv_py[7] = m_py2[Convert.ToInt32(m_py_min3[1]) - 6];
                hv_py[8] = m_py2[Convert.ToInt32(m_py_min3[2]) - 6];


                //-----------------------------------------------------------------------------//
                HOperatorSet.VectorToHomMat2d(hv_px, hv_py, hv_Qx, hv_Qy, out hv_HomMat2D);
                hv_Qx.Dispose();
                hv_Qy.Dispose();
                HOperatorSet.AffineTransPoint2d(hv_HomMat2D, hv_px, hv_py, out hv_Qx, out hv_Qy);
            }
            catch
            {
                MessageBox.Show("标定错误，请移动机器人重新拍照!");
                return;
            }


            //显示图片
            
            HOperatorSet.SetColor(HDevWindowStack.GetActive(), "green");
            HOperatorSet.SetLineWidth(HDevWindowStack.GetActive(), 2);
            HOperatorSet.SetDraw(HDevWindowStack.GetActive(), "margin");
            HOperatorSet.DispObj(ho_SelectedRegions, hv_WindowHandle1);
            Get_New_Center();
            ho_Image2.Dispose();

            ho_Region.Dispose();
            ho_RegionDilation.Dispose();
            ho_RegionErosion.Dispose();
            ho_RegionClosing.Dispose();
            ho_ConnectedRegions.Dispose();
            ho_SelectedRegions.Dispose();

            hv_px.Dispose();
            hv_py.Dispose();
            hv_Qx.Dispose();
            hv_Qy.Dispose();
            hv_Area.Dispose();
            hv_Row.Dispose();
            hv_Column.Dispose();
            hv_HomMat2D.Dispose();

            Get_Cricle_Center();
        }
        public void Get_Cricle_Center()
        {
            text_CX1.Text = Math.Round((double)hv_px[0], 4).ToString();
            text_CX2.Text = Math.Round((double)hv_px[1], 4).ToString();
            text_CX3.Text = Math.Round((double)hv_px[2], 4).ToString();
            text_CX4.Text = Math.Round((double)hv_px[3], 4).ToString();
            text_CX5.Text = Math.Round((double)hv_px[4], 4).ToString();
            text_CX6.Text = Math.Round((double)hv_px[5], 4).ToString();
            text_CX7.Text = Math.Round((double)hv_px[6], 4).ToString();
            text_CX8.Text = Math.Round((double)hv_px[7], 4).ToString();
            text_CX9.Text = Math.Round((double)hv_px[8], 4).ToString();

            text_CY1.Text = Math.Round((double)hv_py[0], 4).ToString();
            text_CY2.Text = Math.Round((double)hv_py[1], 4).ToString();
            text_CY3.Text = Math.Round((double)hv_py[2], 4).ToString();
            text_CY4.Text = Math.Round((double)hv_py[3], 4).ToString();
            text_CY5.Text = Math.Round((double)hv_py[4], 4).ToString();
            text_CY6.Text = Math.Round((double)hv_py[5], 4).ToString();
            text_CY7.Text = Math.Round((double)hv_py[6], 4).ToString();
            text_CY8.Text = Math.Round((double)hv_py[7], 4).ToString();
            text_CY9.Text = Math.Round((double)hv_py[8], 4).ToString();
        }

        public HTuple QX()
        {
            HTuple qx = new HTuple();
            qx[0] = Convert.ToDouble(text_x1.Text);
            qx[1] = Convert.ToDouble(text_x2.Text);
            qx[2] = Convert.ToDouble(text_x3.Text);
            qx[3] = Convert.ToDouble(text_x4.Text);
            qx[4] = Convert.ToDouble(text_x5.Text);
            qx[5] = Convert.ToDouble(text_x6.Text);
            qx[6] = Convert.ToDouble(text_x7.Text);
            qx[7] = Convert.ToDouble(text_x8.Text);
            qx[8] = Convert.ToDouble(text_x9.Text);
            return qx;
        }

        public HTuple QY()
        {
            HTuple qy = new HTuple();
            qy[0] = Convert.ToDouble(text_y1.Text);
            qy[1] = Convert.ToDouble(text_y2.Text);
            qy[2] = Convert.ToDouble(text_y3.Text);
            qy[3] = Convert.ToDouble(text_y4.Text);
            qy[4] = Convert.ToDouble(text_y5.Text);
            qy[5] = Convert.ToDouble(text_y6.Text);
            qy[6] = Convert.ToDouble(text_y7.Text);
            qy[7] = Convert.ToDouble(text_y8.Text);
            qy[8] = Convert.ToDouble(text_y9.Text);
            return qy;
        }

        public void Get_New_Center()
        {
            text_NewX1.Text = Math.Round((double)hv_Qx[0], 4).ToString();
            text_NewX2.Text = Math.Round((double)hv_Qx[1], 4).ToString();
            text_NewX3.Text = Math.Round((double)hv_Qx[2], 4).ToString();
            text_NewX4.Text = Math.Round((double)hv_Qx[3], 4).ToString();
            text_NewX5.Text = Math.Round((double)hv_Qx[4], 4).ToString();
            text_NewX6.Text = Math.Round((double)hv_Qx[5], 4).ToString();
            text_NewX7.Text = Math.Round((double)hv_Qx[6], 4).ToString();
            text_NewX8.Text = Math.Round((double)hv_Qx[7], 4).ToString();
            text_NewX9.Text = Math.Round((double)hv_Qx[8], 4).ToString();

            text_NewY1.Text = Math.Round((double)hv_Qy[0], 4).ToString();
            text_NewY2.Text = Math.Round((double)hv_Qy[1], 4).ToString();
            text_NewY3.Text = Math.Round((double)hv_Qy[2], 4).ToString();
            text_NewY4.Text = Math.Round((double)hv_Qy[3], 4).ToString();
            text_NewY5.Text = Math.Round((double)hv_Qy[4], 4).ToString();
            text_NewY6.Text = Math.Round((double)hv_Qy[5], 4).ToString();
            text_NewY7.Text = Math.Round((double)hv_Qy[6], 4).ToString();
            text_NewY8.Text = Math.Round((double)hv_Qy[7], 4).ToString();
            text_NewY9.Text = Math.Round((double)hv_Qy[8], 4).ToString();
        }

        private void button2_Click(object sender, EventArgs e)
        {
            timer1.Enabled = true;
            //在显示图像之前需要打开一个窗口并设置其属性
            HOperatorSet.OpenWindow(0, 0, hWindowControl1.Width, hWindowControl1.Height, hWindowControl1.HalconWindow, "visible", "", out hv_WindowHandle);
            HDevWindowStack.Push(hv_WindowHandle);           
        }


        /// <summary>
        /// 按从小到大对数组元素进行排序，并输出各元素所对应的原来的序号
        /// </summary>
        /// <param name="aa">输入所需要排序的数组</param>
        /// <returns></returns>
        public double[] Min(double[] aa)
        {
            double[] item = new double[3];
            int n = aa.Length;

            Tuple<double, double>[] indexedArr = new Tuple<double, double>[n];
            for (int i = 0; i < n; i++)
            {
                indexedArr[i] = new Tuple<double, double>(aa[i], i);
            }

            for (int i = 0; i < n; i++)
            {
                for (int j = i + 1; j < n; j++)
                {
                    if (indexedArr[i].Item1 > indexedArr[j].Item1)
                    {
                        Tuple<double, double> temp = indexedArr[i];
                        indexedArr[i] = indexedArr[j];
                        indexedArr[j] = temp;
                    }
                }
            }
            item[0] = indexedArr[0].Item2;
            item[1] = indexedArr[1].Item2;
            item[2] = indexedArr[2].Item2;

            return item;
        }

    }
}