opencv 4.0.0教程系列01.1-core模块-mat_the_basic_image_container

Mat - The Basic Image Container {#tutorial_mat_the_basic_image_container}

@next_tutorial{tutorial_how_to_scan_images}
添加链接描述

Goal 目标

We have multiple ways to acquire digital images from the real world: digital cameras, scanners, computed tomography, and magnetic resonance imaging to name a few. In every case what we (humans) see are images. However, when transforming this to our digital devices what we record are numerical values for each of the points of the image.
我们有多种方法可以从现实世界中获取数字图像：数码相机，扫描仪，计算机断层扫描和磁共振成像等等。 在每种情况下，我们（人类）看到的都是图像。 然而，当将其转换为我们的数字设备时，我们记录的是图像的每个点的数值。

For example in the above image you can see that the mirror of the car is nothing more than a matrix containing all the intensity values of the pixel points. How we get and store the pixels values may vary according to our needs, but in the end all images inside a computer world may be reduced to numerical matrices and other information describing the matrix itself. OpenCV is a computer vision library whose main focus is to process and manipulate this information. Therefore, the first thing you need to be familiar with is how OpenCV stores and handles images.
例如，在上图中，您可以看到汽车的镜子只不过是一个包含像素点所有强度值的矩阵。我们如何获取和存储像素值可能根据我们的需要而变化，但最终计算机世界中的所有图像可以简化为数字矩阵和描述矩阵本身的其他信息。OpenCV是一个计算机视觉库，其主要重点是处理和操作这些信息。因此，您需要熟悉的第一件事是OpenCV如何存储和处理图像。

Mat

图像矩阵
OpenCV has been around since 2001. In those days the library was built around a C interface and to store the image in the memory they used a C structure called IplImage. This is the one you’ll see in most of the older tutorials and educational materials. The problem with this is that it brings to the table all the minuses of the C language. The biggest issue is the manual memory management. It builds on the assumption that the user is responsible for taking care of memory allocation and
deallocation. While this is not a problem with smaller programs, once your code base grows it will be more of a struggle to handle all this rather than focusing on solving your development goal.
OpenCV自2001年开始出现。当时，这个库是围绕C接口构建的，并将图像存储在内存中，它们使用了一个名为IplImage的C结构。这是您在大多数旧教程和教学材料中看到的内容。这样做的问题是它引入了C语言的所有缺点。最大的问题是手动内存管理。它建立在用户负责内存分配和释放的假设之上。虽然这对于较小的程序来说不是问题，但是一旦你的代码库增长，你将会耗费更多的精力去处理所有这些内存管理问题，而不是专注于解决你的开发目标。

Luckily C++ came around and introduced the concept of classes making easier for the user through automatic memory management (more or less). The good news is that C++ is fully compatible with C so no compatibility issues can arise from making the change. Therefore, OpenCV 2.0 introduced a new C++ interface which offered a new way of doing things which means you do not need to fiddle with memory management, making your code concise (less to write, to achieve more). The main downside of the C++ interface is that many embedded development systems at the moment support only C. Therefore, unless
you are targeting embedded platforms, there’s no point to using the old methods (unless you’re a masochist programmer and you’re asking for trouble).
幸运的是，C++出现了，并引入了类的概念，通过自动内存管理(或多或少)使用户更容易使用类。好消息是C++与C完全兼容，因此进行更改不会产生兼容性问题。因此，OpenCV2.0引入了一个新的C++接口，它提供了一种新的管理方式，这意味着您不需要摆弄内存管理，从而使代码更简洁(编写更少，实现更多)。C++接口的主要缺点是目前许多嵌入式开发系统只支持C语言，因此，除非您的目标是嵌入式平台，使用旧的方法是没有意义的(除非您是一个受虐狂程序员，并且您在自找麻烦)。

The first thing you need to know about Mat is that you no longer need to manually allocate its memory and release it as soon as you do not need it. While doing this is still a possibility, most of the OpenCV functions will allocate its output data automatically. As a nice bonus if you pass on an already existing Mat object, which has already allocated the required space for the matrix, this will be reused. In other words we use at all times only as much memory as we need to perform the task.Mat is basically a class with two data parts: the matrix header (containing information such as the size of the matrix, the method used for storing, at which address is the matrix stored, and so on) and a pointer to the matrix containing the pixel values (taking any dimensionality depending on the method chosen for storing) . The matrix header size is constant, however the size of the matrix itself may vary from image to image and usually is larger by orders of magnitude.
关于Mat, 您需要了解的第一件事是，您不再需要手动分配其内存并在不需要时立即释放它。虽然这样做仍有可能，但大多数OpenCV函数都会自动分配其输出数据。如果您传递一个已经存在的Mat对象(已经为矩阵分配了所需的空间)，那么这将被重用，这是一个很好的额外好处。换句话说，我们在任何时候都只使用我们执行任务所需的内存。Mat基本上是一个包含两个数据部分的类：矩阵头(包含信息矩阵的大小等,用于存储的方法,在解决矩阵存储,等等)和一个指向包含像素值的矩阵的指针(采取任何维数取决于选择的方法来存储)。矩阵头大小是恒定的，但是矩阵本身的大小可能因图像而异，并且通常要大几个数量级。

OpenCV is an image processing library. It contains a large collection of image processing functions. To solve a computational challenge, most of the time you will end up using multiple functions of the library. Because of this, passing images to functions is a common practice. We should not forget that we are talking about image processing algorithms, which tend to be quite computational heavy. The last thing we want to do is further decrease the speed of your program by making unnecessary copies of potentially large images.
OpenCV是一个图像处理库。它包含大量图像处理功能。为了解决计算挑战，大多数时候您最终会使用库的多个功能。因此，将图像传递给函数是一种常见的做法。我们要注意我们正在讨论的是图像处理算法，这些算法往往计算量很大。我们最不愿意做的就是通过制作“大”图像的不必要副本来进一步降低程序的速度。

To tackle this issue OpenCV uses a reference counting system. The idea is that each Mat object has its own header, however the matrix may be shared between two instance of them by having their matrix pointers point to the same address. Moreover, the copy operators will only copy the headers and the pointer to the large matrix, not the data itself.
为解决此问题，OpenCV使用引用计数系统。这个想法是每个Mat对象都有自己的矩阵头，但是矩阵可以通过使它们的矩阵指针指向同一个地址实现它们的两个实例之间（数据）共享。此外，复制操作符只会复制标题和指向矩阵的指针，而不是数据本身。

@code{.cpp}
Mat A, C; // 创建矩阵头
A = imread(argv[1], IMREAD_COLOR); // 分配矩阵

Mat B(A); // 调用拷贝构造函数，浅拷贝

C = A; // 赋值运算符，浅拷贝
@endcode

All the above objects, in the end, point to the same single data matrix. Their headers are different, however, and making a modification using any of them will affect all the other ones as well. In practice the different objects just provide different access method to the same underlying data. Nevertheless, their header parts are different. The real interesting part is that you can create headers which refer to only a subsection of the full data. For example, to create a region of interest (ROI) in an image you just create a new header with the new boundaries:
最后，所有上述对象都指向同一个数据矩阵。它们的矩阵头不同，但是修改它们中的任何一个对象的数据也会影响所有其他对象的数据。实际上，不同的对象只是为相同的底层数据提供不同的访问方法。然而，他们的矩阵头部分是不同的。真正有趣的部分是您可以创建仅涉及部分数据的矩阵头。 例如，要在图像中创建感兴趣区域（ROI），您只需使用新边界创建新矩阵头：
@code{.cpp}
Mat D (A, Rect(10, 10, 100, 100) ); // 使用一个矩形
Mat E = A(Range::all(), Range(1,3)); // 使用行和列边界
@endcode
Now you may ask if the matrix itself may belong to multiple Mat objects who takes responsibility for cleaning it up when it’s no longer needed. The short answer is: the last object that used it.This is handled by using a reference counting mechanism. Whenever somebody copies a header of a Mat object, a counter is increased for the matrix. Whenever a header is cleaned this ounter is decreased. When the counter reaches zero the matrix too is freed.
现在你可能会问矩阵本身是否属于多个Mat对象，它们在不再需要时负责清理它。简短的回答是：使用它的最后一个对象负责它的释放。这是通过使用引用计数机制来处理的。每当有人复制矩阵头时，矩阵的计数器就会增加1。每当清理一个矩阵头时，这个计数器就会减少1。当计数器达到0时，矩阵也被释放。

Sometimes you will want to copy the matrix itself too, so OpenCV provides the @ref cv::Mat::clone() and @ref cv::Mat::copyTo() functions.
有时您也希望复制矩阵本身***（也就是包括矩阵数据）***，因此OpenCV提供了@ref cv:: Mat::clone（）和@ref cv::Mat::copyTo（）函数。
@code{.cpp}
Mat F = A.clone(); // 深拷贝
Mat G;
A.copyTo(G); // 深拷贝
@endcode
Now modifying F or G will not affect the matrix pointed by the Mat header. What you need to remember from all this is that:

• Output image allocation for OpenCV functions is automatic (unless specified otherwise).
  OpenCV函数的输出图像分配是自动的（除非另有说明）。
• You do not need to think about memory management with OpenCVs C++ interface.
  您在使用OpenCVs C++接口时不需要考虑进行内存管理。
• The assignment operator and the copy constructor only copies the header.
  赋值运算符和复制构造函数仅复制矩阵头。
• The underlying matrix of an image may be copied using the @ref cv::Mat::clone() and @ref cv::Mat::copyTo() functions.
  用@ref cv::Mat::clone（）和@ref cv::Mat::copyTo（）函数复制图像数据，实现深拷贝。
  Storing methods

---

存储方式

This is about how you store the pixel values. You can select the color space and the data type used.The color space refers to how we combine color components in order to code a given color. The simplest one is the gray scale where the colors at our disposal are black and white. The combination of these allows us to create many shades of gray.
这部分是关于如何存储像素值。您可以选择颜色空间和使用的数据类型。颜色空间是指我们如何组合颜色组件以编码给定颜色。最简单的是灰度，我们处理的颜色是黑色和白色。这些组合使我们可以创建许多灰色阴影*（什么意思？没明白）*。
For colorful ways we have a lot more methods to choose from. Each of them breaks it down to three or four basic components and we can use the combination of these to create the others. The most popular one is RGB, mainly because this is also how our eye builds up colors. Its base colors are
red, green and blue. To code the transparency of a color sometimes a fourth element: alpha (A) is added.

对于彩色方式，我们有更多方法供选择。它们都将其分解为三个或四个基本分量，我们可以使用这些组合来创建其他方式。最受欢迎的是RGB，主要是因为它与是我们的眼睛如何建立颜色类似。它的基色是红色，绿色和蓝色。要编码颜色的透明度，有时会加第四个元素：添加alpha（A）。
There are, however, many other color systems each with their own advantages:
然而，还有许多其他颜色系统各有其优点:

• RGB is the most common as our eyes use something similar, however keep in mind that OpenCV standard display
  system composes colors using the BGR color space (a switch of the red and blue channel).
  RGB颜色空间是最常见的，因为我们的眼睛使用类似的方式，但请记住，OpenCV标准显示系统使用BGR颜色空间（红色和蓝色通道的开关）组成颜色。
  RGB和BGR的区别参考我另一篇博客：
  https://blog.csdn.net/weixin_42142612/article/details/80804039
• The HSV and HLS decompose colors into their hue, saturation and value/luminance components, which is a more natural way for us to describe colors. You might, for example, dismiss the last component, making your algorithm less sensible to the light conditions of the input image.
  HSV颜色空间和HLS颜色空间将颜色分解为色调，饱和度和值/亮度分量，这是我们描述颜色的更自然的方式。例如，您可能会忽略最后一个分量，使您的算法对输入图像的光照条件不太敏感。
• YCrCb is used by the popular JPEG image format.
  YCrCb颜色空间广泛应用在JPEG图像格式。
• CIE L*a*b* is a perceptually uniform color space, which comes handy if you need to measure the distance of a given color to another color.
  CIE L*a*b*是感知上均匀的颜色空间，如果您需要测量给定颜色与另一种颜色之间的距离，用它会很方便。
  Each of the building components has their own valid domains. This leads to the data type used. How we store a component defines the control we have over its domain. The smallest data type possible is char, which means one byte or 8 bits. This may be unsigned (so can store values from 0 to 255) or signed (values from -127 to +127). Although in case of three components this already gives 16 million possible colors to represent (like in case of RGB) we may acquire an even finer control by using the float (4 byte = 32 bit) or double (8 byte = 64 bit) data types for each component. Nevertheless, remember that increasing the size of a component also increases the size of the wholepicture in the memory.
  每个颜色空间都有自己的有效域。有效域决定了使用的数据类型。我们如何存储分量决定了我们对其有效域域的控制。可能的最小数据类型是char，表示一个字节或8位。这可能是无符号的（因此可以存储0到255之间的值）或有符号（值从-127到+127）。虽然在三个分量的情况下，这已经提供了1600万种可能的颜色来表示（如在RGB的情况下），我们可以通过使用浮点（4字节= 32位）或双（8字节= 64位）数据来获得更精细的控制每个分量的类型。 不过，请记住，增加分量的大小也会增加内存中整个图片的大小。

Creating a Mat object explicitly

显式创建一个Mat类对象
In the @ref tutorial_load_save_image tutorial you have already learned how to write a matrix to an image file by using the @ref cv::imwrite() function. However, for debugging purposes it’s much more convenient to see the actual values. You can do this using the << operator of Mat. Be aware that this only works for two dimensional matrices.
在@ref tutorial_load_save_image教程中，您已经学习了如何使用@ref cv::imwrite()函数将矩阵写入图像文件。 但是，出于调试目的，查看实际值会更方便。您可以使用* Mat的\ <\ <运算符来完成此操作。请注意，这仅适用于二维矩阵。
查看图像像素值工具ImageWatch,参考我另一篇博客:(VS2013/VS2015/VS2017均可使用)
https://blog.csdn.net/weixin_42142612/article/details/80771538

Although Mat works really well as an image container, it is also a general matrix class.Therefore, it is possible to create and manipulate multidimensional matrices. You can create a Mat object in multiple ways:
尽管Mat作为图像容器非常有效，但它也是一般的矩阵类。因此，可以创建和操作多维矩阵。您可以通过多种方式创建Mat对象：

下面剩余部分就不翻译了，学习一下mat_the_basic_image_container.cpp文件就知道其表达的含义了，纯翻译没有代码翻译出来很奇怪。

• @ref cv::Mat::Mat Constructor

  @snippet mat_the_basic_image_container.cpp constructor

  For two dimensional and multichannel images we first define their size: row and column count wise. Then we need to specify the data type to use for storing the elements and the number of channels per matrix point. To do this we have multiple definitions constructed according to the following convention:

  @code
  CV_[The number of bits per item][Signed or Unsigned][Type Prefix]C[The channel number]

  @endcode
  For instance, CV_8UC3 means we use unsigned char types that are 8 bit long and each pixel has three of these to form the three channels. This are predefined for up to four channel numbers. The @ref cv::Scalar is four element short vector. Specify this and you can initialize all matrix points with a custom value. If you need more you can create the type with the upper macro, setting the channel number in parenthesis as you can see below.

• Use C/C++ arrays and initialize via constructor

  @snippet mat_the_basic_image_container.cpp init

  The upper example shows how to create a matrix with more than two dimensions. Specify its dimension, then pass a pointer containing the size for each dimension and the rest remains the same.

• @ref cv::Mat::create function:

  @snippet mat_the_basic_image_container.cpp create

  You cannot initialize the matrix values with this construction. It will only reallocate its matrix data memory if the new size will not fit into the old one.

• MATLAB style initializer: @ref cv::Mat::zeros , @ref cv::Mat::ones , @ref cv::Mat::eye . Specify size and data type to use:

  Matlab风格的初始化：cv::Mat::zeros，cv::Mat::ones, cv::Mat::eye, 可以指定size和type
  @snippet mat_the_basic_image_container.cpp matlab

• For small matrices you may use comma separated initializers or initializer lists (C++11 support is required in the last case):

  @snippet mat_the_basic_image_container.cpp comma

  @snippet mat_the_basic_image_container.cpp list

• Create a new header for an existing Mat object and @ref cv::Mat::clone or @ref cv::Mat::copyTo it.

  @snippet mat_the_basic_image_container.cpp clone

  @note
  You can fill out a matrix with random values using the @ref cv::randu() function. You need to give the lower and upper value for the random values:
  @snippet mat_the_basic_image_container.cpp random

Output formatting

In the above examples you could see the default formatting option. OpenCV, however, allows you to
format your matrix output:

• Default
  @snippet mat_the_basic_image_container.cpp out-default

• Python
  @snippet mat_the_basic_image_container.cpp out-python

• Comma separated values (CSV)
  @snippet mat_the_basic_image_container.cpp out-csv

• Numpy
  @snippet mat_the_basic_image_container.cpp out-numpy

• C
  @snippet mat_the_basic_image_container.cpp out-c

Output of other common items

OpenCV offers support for output of other common OpenCV data structures too via the << operator:

• 2D Point
  @snippet mat_the_basic_image_container.cpp out-point2
• 3D Point
  @snippet mat_the_basic_image_container.cpp out-point3
• std::vector via cv::Mat
  @snippet mat_the_basic_image_container.cpp out-vector
• std::vector of points
  @snippet mat_the_basic_image_container.cpp out-vector-points

Most of the samples here have been included in a small console application. You can download it from
here
or in the core section of the cpp samples.

You can also find a quick video demonstration of this on
YouTube.

@youtube{1tibU7vGWpk}

学习mat_the_basic_image_container.cpp

• cv::Mat M(…)
  Mat M(2,2, CV_8UC3, Scalar(0,0,255));
  创建一个2*2,uchar类型，3通道的图像，每个像素的初始值为B/G/R = 0/0/255
  

• M.create
  M.create(4,4, CV_8UC(2));
  备注：此语句是在上一条语句基础上执行，下面的也是类似，不重复说明了。

  执行create函数，新size为44，与原size22不同，因此重新分配空间，并不是全部重新分配，而是在原来的size2*2基础上重新分配。也就说数据区域的地址是不变的。比如原来的data地址为0x0000000000469dc0，执行create函数之后data的地址依然为0x0000000000497e80。
  
  此处通道数由3变成2之后，初始值由B/G/R = 0/0/255变成了255/255，这个变化的机制是怎么样的呢？
  这个可能就需要看create函数的实现了，猜想是最后一个通道r通道的值保留了作为初始值。

• init

  int sz[3] = {2,2,2};  
  Mat L(3,sz, CV_8UC(1), Scalar::all(0));
  

  展示了如何创建一个二维以上的矩阵。指定它的维数，然后传递一个指针，指针指向的内容包含每个维数的大小。比较疑惑这种创建方式什么情况会用到？用ImageWatch无法查看它的内部值，操作符<<打印输出也无法使用。

• matlab样式初始化

  Mat E = Mat::eye(4, 4, CV_64F);
  Mat O = Mat::ones(2, 2, CV_32F);
  Mat Z = Mat::zeros(3,3, CV_8UC1);  
  

其中ones和zeros在制作图像掩码时可以用到，eyes我还没用过。

• comma
  Mat C = (Mat_<double>(3,3) << 0, -1, 0, -1, 5, -1, 0, -1, 0);
  
  Mat_是模板类，指定数据类型和size就可以进行初始化了。
• list（c++11)
  C = (Mat_<double>({0, -1, 0, -1, 5, -1, 0, -1, 0})).reshape(3);
  
  使用c++11中的list容器完成初始化。
• clone
  Mat RowClone = C.row(1).clone();
  
• randu

  Mat R = Mat(3, 2, CV_8UC3);
  randu(R, Scalar::all(0), Scalar::all(255));
  

randu函数声明:
Non-template variant of the function fills the matrix dst with uniformly-distributed random numbers from the specified range:
函数功能：用限定范围的服从正态分布的随机数填充矩阵
@param dst output array of random numbers; the array must be pre-allocated.
@param low inclusive lower boundary of the generated random numbers.
@param high exclusive upper boundary of the generated random numbers.
@sa RNG, randn, theRNG
CV_EXPORTS_W void randu(InputOutputArray dst, InputArray low, InputArray high);

• output

  out << "R (default) = " << endl <<        R           << endl << endl;
  cout << "R (python)  = " << endl << format(R, Formatter::FMT_PYTHON) << endl << endl;
  cout << "R (numpy)   = " << endl << format(R, Formatter::FMT_NUMPY ) << endl << endl;
  cout << "R (csv)     = " << endl << format(R, Formatter::FMT_CSV   ) << endl << endl;
  cout << "R (c)       = " << endl << format(R, Formatter::FMT_C     ) << endl << endl;  
  

format函数支持的六种类型：
enum FormatType {
FMT_DEFAULT = 0,
FMT_MATLAB = 1,
FMT_CSV = 2,
FMT_PYTHON = 3,
FMT_NUMPY = 4,
FMT_C = 5
};

• point和vector

  Point3f P3f(2, 6, 7);
  vector<float> v;
  v.push_back( (float)CV_PI);   v.push_back(2);    v.push_back(3.01f);
  vector<Point2f> vPoints(20);
  for (size_t i = 0; i < vPoints.size(); ++i)
      vPoints[i] = Point2f((float)(i * 5), (float)(i % 7));