【opencv源码剖析】图像分割 floodFill与graphsegmentation

一、前言

最近在使用floodFill这个算法时，突然想起selective search里的基础图像分割算法 - 基于图的graphsementation。

两者是比较简单的算法，存在相似之处，即都使用了相邻像素之间像素强度关系。

同时又存在不同点，floodFill关注点是像素层面上，生成一个区域；

而graphsementation由像素逐步构建出图块，生成多个区域。

二、graphsementation源码分析

基于图的分割算法graphsementation没有在opencv主模块版本中实现，其代码位于opencv_contrib/modules/ximgproc模块

中的graphsegmentation.cpp。算法主体步骤如下：

void GraphSegmentationImpl::processImage(InputArray src, OutputArray dst) {

                Mat img = src.getMat();

                dst.create(img.rows, img.cols, CV_32SC1);
                Mat output = dst.getMat();
                output.setTo(0);

                // Filter graph
                Mat img_filtered;
                filter(img, img_filtered);

                // Build graph
                Edge *edges;
                int nb_edges;

                buildGraph(&edges, nb_edges, img_filtered);

                // Segment graph
                PointSet *es;

                segmentGraph(edges, nb_edges, img_filtered, &es);

                // Remove small areas
                filterSmallAreas(edges, nb_edges, es);

                // Map to final output
                finalMapping(es, output);

                delete [] edges;
                delete es;

            }

 1.高斯滤波降低噪声影响

void GraphSegmentationImpl::filter(const Mat &img, Mat &img_filtered) {

                Mat img_converted;

                // Switch to float
                img.convertTo(img_converted, CV_32F);

                // Apply gaussian filter
                GaussianBlur(img_converted, img_filtered, Size(0, 0), sigma, sigma);
            }

2.基于相邻像素强度差来构建图bulidGraph，计算权重(即相邻像素强度差)

void GraphSegmentationImpl::buildGraph(Edge **edges, int &nb_edges, const Mat &img_filtered) {
   //1.相邻像素连线用Edge表示，每个像素均计算4个方向(四连通，上下左右)。
                *edges = new Edge[img_filtered.rows * img_filtered.cols * 4];

                nb_edges = 0;

                int nb_channels = img_filtered.channels();
   //2.遍历所有像素，计算Edge的权重weight值。
                for (int i = 0; i < (int)img_filtered.rows; i++) {
                    const float* p = img_filtered.ptr<float>(i);

                    for (int j = 0; j < (int)img_filtered.cols; j++) {

                        //Take the right, left, top and down pixel
                        for (int delta = -1; delta <= 1; delta += 2) {
                            for (int delta_j = 0, delta_i = 1; delta_j <= 1; delta_j++ || delta_i--) {

                                int i2 = i + delta * delta_i;
                                int j2 = j + delta * delta_j;

                                if (i2 >= 0 && i2 < img_filtered.rows && j2 >= 0 && j2 < img_filtered.cols) {
                                    const float* p2 = img_filtered.ptr<float>(i2);

                                    float tmp_total = 0;
      //3.权重简单地等于像素差值的平方再开根号，彩色3通道则全部累加
                                    for ( int channel = 0; channel < nb_channels; channel++) {
                                        tmp_total += pow(p[j * nb_channels + channel] - p2[j2 * nb_channels + channel], 2);
                                    }

                                    float diff = 0;
                                    diff = sqrt(tmp_total);
      //4.from、to存储像素坐标索引
                                    (*edges)[nb_edges].weight = diff;
                                    (*edges)[nb_edges].from = i * img_filtered.cols +  j;
                                    (*edges)[nb_edges].to = i2 * img_filtered.cols + j2;

                                    nb_edges++;
                                }
                            }
                        }
                    }
                }
            }

 3.对构建的图，按权重进行分割

void GraphSegmentationImpl::segmentGraph(Edge *edges, const int &nb_edges, const Mat &img_filtered, PointSet **es) {

                int total_points = ( int)(img_filtered.rows * img_filtered.cols);
   //1.按权重大小对edge排序
                // Sort edges
                std::sort(edges, edges + nb_edges);
   //2.构建点集块，初始化时每个像素为独立一个点集块，后面逐步合并
                // Create a set with all point (by default mapped to themselfs)
                *es = new PointSet(img_filtered.cols * img_filtered.rows);
   //3.判断是否合并的阈值，每个块(初始化时即像素)对应一个阈值
                // Thresholds
                float* thresholds = new float[total_points];

                for (int i = 0; i < total_points; i++)
                    thresholds[i] = k;

                for ( int i = 0; i < nb_edges; i++) {
    //4.获取块的索引，即基点的坐标索引
                    int p_a = (*es)->getBasePoint(edges[i].from);
                    int p_b = (*es)->getBasePoint(edges[i].to);
    //5.根据阈值，判断是否合并这两个块
                    if (p_a != p_b) {
                        if (edges[i].weight <= thresholds[p_a] && edges[i].weight <= thresholds[p_b]) {
                            (*es)->joinPoints(p_a, p_b);
                            p_a = (*es)->getBasePoint(p_a);
                            thresholds[p_a] = edges[i].weight + k / (*es)->size(p_a);//块的权重更新

                            edges[i].weight = 0;
                        }
                    }
                }

                delete [] thresholds;
            } 

 4.合并过小的区域块filterSmallAreas

 5.根据最终的点集块生成分割结果finalMapping

 

三、floodFill源码分析

opencv的floodFill主要以入栈方式和行扫描进行。

当loDiff、upDiff均为0时，由icvFloodFill_CnIR函数完成计算，这里会按分割结果直接对原图设置newVal，不生成mask。

其他情况均是走icvFloodFillGrad_CnIR函数，主要步骤如下：

1.从seed坐标所在的那行开始填充mask

L = R = seed.x;//L、R为每行的最左及最右x坐标
    if( mask[L] )
        return;

    mask[L] = newMaskVal;//填充mask的值，默认为1，可通过flags设置
    _Tp val0 = img[L];//seed点的像素值
 
    if( fixedRange )//以seed点为基准
    {
        while( !mask[R + 1] && diff( img + (R+1), &val0 ))
            mask[++R] = newMaskVal;

        while( !mask[L - 1] && diff( img + (L-1), &val0 ))
            mask[--L] = newMaskVal;
    }
    else//以相邻点为基准
    {
        while( !mask[R + 1] && diff( img + (R+1), img + R ))
            mask[++R] = newMaskVal;

        while( !mask[L - 1] && diff( img + (L-1), img + L ))
            mask[--L] = newMaskVal;
    }

 2.循环扫描上下行

XMax = R;
    XMin = L;
 //seed点所在行入栈
    ICV_PUSH( seed.y, L, R, R + 1, R, UP );

    while( head != tail )
    {
        int k, YC, PL, PR, dir;
        ICV_POP( YC, L, R, PL, PR, dir );//出栈
 //扫描上下行
        int data[][3] =
        {
            {-dir, L - _8_connectivity, R + _8_connectivity},
            {dir, L - _8_connectivity, PL - 1},
            {dir, PR + 1, R + _8_connectivity}
        };

        unsigned length = (unsigned)(R-L);
......
     if( fillImage )//根据mask填充原图
         for( i = L; i <= R; i++ )
             img[i] = newVal;