从头开始opencv（三）——core:How to scan images, lookup tables and time measurement

---

参考资料

Goal

Our test case

  Let us consider a simple color reduction method. By using the unsigned char C and C++ type for matrix item storing, a channel of pixel may have up to 256 different values. For a three channel image this can allow the formation of way too many colors (16 million to be exact). Working with so many color shades may give a heavy blow to our algorithm performance. However, sometimes it is enough to work with a lot less of them to get the same final result.
  可以通过减少颜色的使用来提高性能（有时候颜色减少并不会降低图像的质量）
  

  In this cases it's common that we make a color space reduction. This means that we divide the color space current value with a new input value to end up with fewer colors. For instance every value between zero and nine takes the new value zero, every value between ten and nineteen the value ten and so on.
  （废话）这意味着我们使用的颜色数量会减少。
  

  When you divide an uchar (unsigned char - aka values between zero and 255) value with an int value the result will be also char. These values may only be char values. Therefore, any fraction will be rounded down.
  当将uchar类型的数据除以int类型的数据时，我们得到的依然只是uchar类型的数据，同时，分数会被四舍五入。
  

  A simple color space reduction algorithm would consist of just passing through every pixel of an image matrix and applying this formula. It's worth noting that we do a divide and a multiplication operation. These operations are bloody expensive for a system. If possible it's worth avoiding them by using cheaper operations such as a few subtractions, addition or in best case a simple assignment. Furthermore, note that we only have a limited number of input values for the upper operation. In case of the uchar system this is 256 to be exact.
  最简单的用来减少颜色数量的算法就是通过扫描每一个像素，对像素应用公式来计算。在这种公式下，我们只需要使用除法和乘法运算，但对于系统而言，这种开销过于昂贵了。所以我们需要做的就是通过使用减法、加法或者更简单的赋值操作来尽量使得我们消耗的资源更少一点。同样值得注意的是，我们所能够使用的input的数量也是极为有限的。
  

  Therefore, for larger images it would be wise to calculate all possible values beforehand and during the assignment just make the assignment, by using a lookup table. Lookup tables are simple arrays (having one or more dimensions) that for a given input value variation holds the final output value. Its strength lies that we do not need to make the calculation, we just need to read the result.
  对于大图像来说，我们常用的做法就是先把所有可能的值进行预计算，通过查找表的方式来减少资源的开销。
  

  Our test case program (and the sample presented here) will do the following: read in a console line argument image (that may be either color or gray scale - console line argument too) and apply the reduction with the given console line argument integer value. In OpenCV, at the moment there are three major ways of going through an image pixel by pixel. To make things a little more interesting will make the scanning for each image using all of these methods, and print out how long it took.
  流程就是：读取控制台参数、使用多种方法对像素进行处理，打印处理的时间。
  

  直接上代码：

#include <opencv2/core.hpp>
#include <opencv2/core/utility.hpp>
#include "opencv2/imgcodecs.hpp"
#include <opencv2/highgui.hpp>
#include <iostream>
#include <sstream>

using namespace std;
using namespace cv;

static void help()
{
    cout
        << "\n--------------------------------------------------------------------------" << endl
        << "This program shows how to scan image objects in OpenCV (cv::Mat). As use case"
        << " we take an input image and divide the native color palette (255) with the " << endl
        << "input. Shows C operator[] method, iterators and at function for on-the-fly item address calculation." << endl
        << "Usage:" << endl
        << "./how_to_scan_images <imageNameToUse> <divideWith> [G]" << endl
        << "if you add a G parameter the image is processed in gray scale" << endl
        << "--------------------------------------------------------------------------" << endl
        << endl;
}

Mat& ScanImageAndReduceC(Mat& I, const uchar* table);
Mat& ScanImageAndReduceIterator(Mat& I, const uchar* table);
Mat& ScanImageAndReduceRandomAccess(Mat& I, const uchar* table);

int main(int argc, char* argv[])
{
    help();
    if (argc < 3)
    {
        cout << "Not enough parameters" << endl;
        return -1;
    }

    Mat I, J;
    if (argc == 4 && !strcmp(argv[3], "G"))
        I = imread(argv[1], IMREAD_GRAYSCALE);
    else
        I = imread(argv[1], IMREAD_COLOR);

    if (I.empty())
    {
        cout << "The image" << argv[1] << " could not be loaded." << endl;
        return -1;
    }

    //! [dividewith]
    int divideWith = 0; // convert our input string to number - C++ style
    stringstream s;
    s << argv[2];
    s >> divideWith;
    if (!s || !divideWith)
    {
        cout << "Invalid number entered for dividing. " << endl;
        return -1;
    }

    uchar table[256];
    for (int i = 0; i < 256; ++i)
        table[i] = (uchar)(divideWith * (i / divideWith));
    //! [dividewith]

    const int times = 100;
    double t;

    t = (double)getTickCount();

    for (int i = 0; i < times; ++i)
    {
        cv::Mat clone_i = I.clone();
        J = ScanImageAndReduceC(clone_i, table);
    }

    t = 1000 * ((double)getTickCount() - t) / getTickFrequency();
    t /= times;

    cout << "Time of reducing with the C operator [] (averaged for "
        << times << " runs): " << t << " milliseconds." << endl;

    t = (double)getTickCount();

    for (int i = 0; i < times; ++i)
    {
        cv::Mat clone_i = I.clone();
        J = ScanImageAndReduceIterator(clone_i, table);
    }

    t = 1000 * ((double)getTickCount() - t) / getTickFrequency();
    t /= times;

    cout << "Time of reducing with the iterator (averaged for "
        << times << " runs): " << t << " milliseconds." << endl;

    t = (double)getTickCount();

    for (int i = 0; i < times; ++i)
    {
        cv::Mat clone_i = I.clone();
        ScanImageAndReduceRandomAccess(clone_i, table);
    }

    t = 1000 * ((double)getTickCount() - t) / getTickFrequency();
    t /= times;

    cout << "Time of reducing with the on-the-fly address generation - at function (averaged for "
        << times << " runs): " << t << " milliseconds." << endl;

    //! [table-init]
    Mat lookUpTable(1, 256, CV_8U);
    uchar* p = lookUpTable.ptr();
    for (int i = 0; i < 256; ++i)
        p[i] = table[i];
    //! [table-init]

    t = (double)getTickCount();

    for (int i = 0; i < times; ++i)
        //! [table-use]
        LUT(I, lookUpTable, J);
    //! [table-use]

    t = 1000 * ((double)getTickCount() - t) / getTickFrequency();
    t /= times;

    cout << "Time of reducing with the LUT function (averaged for "
        << times << " runs): " << t << " milliseconds." << endl;
    return 0;
}

//! [scan-c]
Mat& ScanImageAndReduceC(Mat& I, const uchar* const table)
{
    // accept only char type matrices
    CV_Assert(I.depth() == CV_8U);

    int channels = I.channels();

    int nRows = I.rows;
    int nCols = I.cols * channels;

    if (I.isContinuous())
    {
        nCols *= nRows;
        nRows = 1;
    }

    int i, j;
    uchar* p;
    for (i = 0; i < nRows; ++i)
    {
        p = I.ptr<uchar>(i);
        for (j = 0; j < nCols; ++j)
        {
            p[j] = table[p[j]];
        }
    }
    return I;
}
//! [scan-c]

//! [scan-iterator]
Mat& ScanImageAndReduceIterator(Mat& I, const uchar* const table)
{
    // accept only char type matrices
    CV_Assert(I.depth() == CV_8U);

    const int channels = I.channels();
    switch (channels)
    {
    case 1:
    {
        MatIterator_<uchar> it, end;
        for (it = I.begin<uchar>(), end = I.end<uchar>(); it != end; ++it)
            *it = table[*it];
        break;
    }
    case 3:
    {
        MatIterator_<Vec3b> it, end;
        for (it = I.begin<Vec3b>(), end = I.end<Vec3b>(); it != end; ++it)
        {
            (*it)[0] = table[(*it)[0]];
            (*it)[1] = table[(*it)[1]];
            (*it)[2] = table[(*it)[2]];
        }
    }
    }

    return I;
}
//! [scan-iterator]

//! [scan-random]
Mat& ScanImageAndReduceRandomAccess(Mat& I, const uchar* const table)
{
    // accept only char type matrices
    CV_Assert(I.depth() == CV_8U);

    const int channels = I.channels();
    switch (channels)
    {
    case 1:
    {
        for (int i = 0; i < I.rows; ++i)
            for (int j = 0; j < I.cols; ++j)
                I.at<uchar>(i, j) = table[I.at<uchar>(i, j)];
        break;
    }
    case 3:
    {
        Mat_<Vec3b> _I = I;

        for (int i = 0; i < I.rows; ++i)
            for (int j = 0; j < I.cols; ++j)
            {
                _I(i, j)[0] = table[_I(i, j)[0]];
                _I(i, j)[1] = table[_I(i, j)[1]];
                _I(i, j)[2] = table[_I(i, j)[2]];
            }
        I = _I;
        break;
    }
    }

    return I;
}
//! [scan-random]

代码详解

获取运行时间

    const int times = 100;
    double t;

    t = (double)getTickCount();

    for (int i = 0; i < times; ++i)
    {
        cv::Mat clone_i = I.clone();
        J = ScanImageAndReduceC(clone_i, table);
    }

    t = 1000 * ((double)getTickCount() - t) / getTickFrequency();
    t /= times;

    cout << "Time of reducing with the C operator [] (averaged for "
        << times << " runs): " << t << " milliseconds." << endl;

  geTrickCount()和getTickFrequency()都被定义在core.cpp下。
  getTickCount()：用于返回从操作系统启动到当前所经的计时周期数。
  getTickFrequency()：用于返回CPU的频率。get Tick Frequency。这里的单位是秒，也就是一秒内重复的次数。
公式如下：
$$\frac{总次数}{一秒内重复的次数}=时间(s)$$
$$1000\times \frac{总次数}{一秒内重复的次数}=时间(ms)$$

【注意事项】：在C版本中，cvGetTickFrequency()函数以ms计算频率，但是C++版本中getTickFrequency()以s计算频率。

方法一：Scan Image And Reduce C（最高效的方法）

Mat& ScanImageAndReduceC(Mat& I, const uchar* const table)
{
    // accept only char type matrices
    CV_Assert(I.depth() == CV_8U);

    int channels = I.channels();

    int nRows = I.rows;
    int nCols = I.cols * channels;

    if (I.isContinuous())
    {
        nCols *= nRows;
        nRows = 1;
    }

    int i, j;
    uchar* p;
    for (i = 0; i < nRows; ++i)
    {
        p = I.ptr<uchar>(i);
        for (j = 0; j < nCols; ++j)
        {
            p[j] = table[p[j]];
        }
    }
    return I;
}

思路

  Here we basically just acquire a pointer to the start of each row and go through it until it ends. In the special case that the matrix is stored in a continuous manner we only need to request the pointer a single time and go all the way to the end. We need to look out for color images: we have three channels so we need to pass through three times more items in each row.
  通过指针来进行扫描。
  

CV_Assert()

  CV_Assert()函数与C++标准库中的assert()函数功能基本相同。
  CV_Assert()作用：CV_Assert()若括号中的表达式值为false，则返回一个错误信息。

#include <assert.h> void assert(int expression);

  assert()的作用是计算表达式expression，如果其值为假（即为0），那么它先向stderr打印出一条出错信息，然后通过调用abort来终止程序运行。
  使用assert()的缺点是：频繁的调用会极大的影响程序的性能，增加额外的开销。

isContinuous()函数

  cv::Mat::isContinuous()函数，返回bool值，判断存储是否连续。
  这里连续的意思是行与行之间的储存是否衔接。也就是说$1\times 1$和$1\times N$的数据肯定是连续的，而$m\times n$的数据就是需要判断连续性的。如果数据存储是连续的，可以将图像看成是一个一维数组，否则只能用二维数组的方式进行访问。
  一般用Mat::create()创建的矩阵都是连续的，但是也不绝对，依然需要判断才可以进行连续性操作。而从图像中截取的图像通常都是不连续的，如果想要将其转换为连续存储，可以用clone拷贝一份。

ptr指针

    cv::Mat image = cv::Mat(400, 600, CV_8UC1); //宽400，长600
    uchar * data00 = image.ptr<uchar>(0);
    uchar * data10 = image.ptr<uchar>(1);
    uchar * data01 = image.ptr<uchar>(0)[1];

解释：

1. 定义了一个Mat变量image
2. data00是指向image第一行第一个元素的指针。
3. data10是指向image第二行第一个元素的指针。
4. data01是指向image第一行第二个元素的指针。

方法二：Scan Image And Reduce Iterator

Mat& ScanImageAndReduceIterator(Mat& I, const uchar* const table)
{
    // accept only char type matrices
    CV_Assert(I.depth() == CV_8U);

    const int channels = I.channels();
    switch (channels)
    {
    case 1:
    {
        MatIterator_<uchar> it, end;
        for (it = I.begin<uchar>(), end = I.end<uchar>(); it != end; ++it)
            *it = table[*it];
        break;
    }
    case 3:
    {
        MatIterator_<Vec3b> it, end;
        for (it = I.begin<Vec3b>(), end = I.end<Vec3b>(); it != end; ++it)
        {
            (*it)[0] = table[(*it)[0]];
            (*it)[1] = table[(*it)[1]];
            (*it)[2] = table[(*it)[2]];
        }
    }
    }

    return I;
}

思路

  In case of the efficient way making sure that you pass through the right amount of uchar fields and to skip the gaps that may occur between the rows was your responsibility. The iterator method is considered a safer way as it takes over these tasks from the user. All you need to do is ask the begin and the end of the image matrix and then just increase the begin iterator until you reach the end. To acquire the value pointed by the iterator use the * operator (add it before it).
  迭代器方法被认为是一种更为安全的方法，我们需要做的就是询问图像矩阵的开始和结尾，然后增加迭代器，直到到达结尾为止。
  

  In case of color images we have three uchar items per column. This may be considered a short vector of uchar items, that has been baptized in OpenCV with the Vec3b name. To access the n-th sub column we use simple operator[] access. It's important to remember that OpenCV iterators go through the columns and automatically skip to the next row. Therefore in case of color images if you use a simple uchar iterator you'll be able to access only the blue channel values.
  彩色图像，每一列有三个uchar类型的值。所以我们需要用一个short vector来代表每列的值。同时，OpenCV的迭代器会自动地进入下一行，所以如果在三通道的图像中使用uchar类型的迭代器，所能读取的只能是B类型的数据。
  

迭代器

  迭代器可以保证对所有容器的基本遍历方式，都是一样的，实现算法时若需要遍历，则使用迭代器，则可以不用关注容器的具体类型，实现数据结构和算法的分离。
  迭代器是一种抽象的设计概念，在设计模式一书中的定义为：提供一种方法，使之能够依序地访问某个聚合物（容器）中所含的各个元素，而又不暴露该聚合物中的内部实现细节，也就是说迭代器可以访问容器内部的实现，而不需要管容器中是怎么实现的。
  在C++的STL中广泛使用了迭代器的概念。STL将容器与算法分开实现，容器负责数据的存储，算法代表着用系统的方法描述解决问题的策略机制。

方法三：Scan Image And Reduce Random Access

Mat& ScanImageAndReduceRandomAccess(Mat& I, const uchar* const table)
{
    // accept only char type matrices
    CV_Assert(I.depth() == CV_8U);

    const int channels = I.channels();
    switch (channels)
    {
    case 1:
    {
        for (int i = 0; i < I.rows; ++i)
            for (int j = 0; j < I.cols; ++j)
                I.at<uchar>(i, j) = table[I.at<uchar>(i, j)];
        break;
    }
    case 3:
    {
        Mat_<Vec3b> _I = I;

        for (int i = 0; i < I.rows; ++i)
            for (int j = 0; j < I.cols; ++j)
            {
                _I(i, j)[0] = table[_I(i, j)[0]];
                _I(i, j)[1] = table[_I(i, j)[1]];
                _I(i, j)[2] = table[_I(i, j)[2]];
            }
        I = _I;
        break;
    }
    }

    return I;
}

思路

  The final method isn't recommended for scanning. It was made to acquire or modify somehow random elements in the image. Its basic usage is to specify the row and column number of the item you want to access. During our earlier scanning methods you could already observe that is important through what type we are looking at the image. It's no different here as you need to manually specify what type to use at the automatic lookup. You can observe this in case of the gray scale images for the following source code (the usage of the + cv::at() function):
  不建议使用这种方法来进行扫描。它可以获取或者修改图像中的随机元素。
  

The Core Function

  This is a bonus method of achieving lookup table modification in an image. In image processing it's quite common that you want to modify all of a given image values to some other value. OpenCV provides a function for modifying image values, without the need to write the scanning logic of the image. We use the cv::LUT() function of the core module. First we build a Mat type of the lookup table:
  如果是对图片进行量化，我们可以不需要遍历每一个元素，可以直接使用LUT函数。

    Mat lookUpTable(1, 256, CV_8U);
    uchar* p = lookUpTable.ptr();
    for( int i = 0; i < 256; ++i)
        p[i] = table[i];
    LUT(I, lookUpTable, J);

Performance Difference

  We can conclude a couple of things. If possible, use the already made functions of OpenCV (instead of reinventing these). The fastest method turns out to be the LUT function. This is because the OpenCV library is multi-thread enabled via Intel Threaded Building Blocks. However, if you need to write a simple image scan prefer the pointer method. The iterator is a safer bet, however quite slower. Using the on-the-fly reference access method for full image scan is the most costly in debug mode. In the release mode it may beat the iterator approach or not, however it surely sacrifices for this the safety trait of iterators.
  总结，如果可以的话，请使用OpenCV已有的功能而不是自己去重新写。最快的方法是LUT功能，这是因为OpenCV库通过Intel Threaded Building Blocks启用了多线程。但是，如果您需要编写简单的图像扫描，请使用指针方法，迭代器是一个比较安全的选择，但是比较慢。