opencv mat图片的CV_8UC3转格式显示到UE的texture2d

背景

项目需要使用相机采图实时显示到界面，使用的相机居然只能输出3通道的图，然而UE4的texture2D也是坑，居然不支持3通道图的初始化和数据拷贝，使用下面的方式可以实现转换，但是效率太低，导致游戏的整体帧率大幅度下降。网上查了好久都没有解决方案，最后自己研究了下opencv的接口，获取到意外收获，完美解决问题。虽然很简单，但是网上既然没有，还是想写了分享出来，希望遇到此问题的人能早出苦海。

void ConvertMatToTexture(cv::Mat& InFrame, UTexture2D* OutTexture)
{
	if(OutTexture == nullptr)
	{
		return;
	}

	TArray<FColor> ColorData;
	ColorData.Init(FColor(0, 0, 0, 255), InFrame.cols * InFrame.rows);

	for (int Row = 0; Row < InFrame.rows; Row++)
	{
		for (int Col = 0; Col < InFrame.cols; Col++)
		{
            const int Index = Col + (Row * InFrame.cols);

			ColorData[Index] = FColor(InFrame.data[Index * 3 + 2],
				InFrame.data[Index * 3 + 1],
				InFrame.data[Index * 3 + 0],
				255);
		}
	}

	void* TextureData = OutTexture->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_WRITE);
	FMemory::Memcpy(TextureData, ColorData.GetData(), ColorData.Num() * sizeof(FColor));
	// 解锁Texture
	OutTexture->PlatformData->Mips[0].BulkData.Unlock();

	// 让Texture根据我们写入的像素数据刷新一次
	OutTexture->UpdateResource();
}

解决

就是下面这个接口，接口里面有这么一段
If conversion adds the alpha channel, its value will set to the maximum of corresponding channel
range: 255 for CV_8U, 65535 for CV_16U, 1 for CV_32F.
这不是说可以添加alpha，而且默认值还是255，正式我需要的嘛。

/** @brief Converts an image from one color space to another.

The function converts an input image from one color space to another. In case of a transformation
to-from RGB color space, the order of the channels should be specified explicitly (RGB or BGR). Note
that the default color format in OpenCV is often referred to as RGB but it is actually BGR (the
bytes are reversed). So the first byte in a standard (24-bit) color image will be an 8-bit Blue
component, the second byte will be Green, and the third byte will be Red. The fourth, fifth, and
sixth bytes would then be the second pixel (Blue, then Green, then Red), and so on.

The conventional ranges for R, G, and B channel values are:
-   0 to 255 for CV_8U images
-   0 to 65535 for CV_16U images
-   0 to 1 for CV_32F images

In case of linear transformations, the range does not matter. But in case of a non-linear
transformation, an input RGB image should be normalized to the proper value range to get the correct
results, for example, for RGB \f$\rightarrow\f$ L\*u\*v\* transformation. For example, if you have a
32-bit floating-point image directly converted from an 8-bit image without any scaling, then it will
have the 0..255 value range instead of 0..1 assumed by the function. So, before calling #cvtColor ,
you need first to scale the image down:
@code
    img *= 1./255;
    cvtColor(img, img, COLOR_BGR2Luv);
@endcode
If you use #cvtColor with 8-bit images, the conversion will have some information lost. For many
applications, this will not be noticeable but it is recommended to use 32-bit images in applications
that need the full range of colors or that convert an image before an operation and then convert
back.

If conversion adds the alpha channel, its value will set to the maximum of corresponding channel
range: 255 for CV_8U, 65535 for CV_16U, 1 for CV_32F.

@param src input image: 8-bit unsigned, 16-bit unsigned ( CV_16UC... ), or single-precision
floating-point.
@param dst output image of the same size and depth as src.
@param code color space conversion code (see #ColorConversionCodes).
@param dstCn number of channels in the destination image; if the parameter is 0, the number of the
channels is derived automatically from src and code.

@see @ref imgproc_color_conversions
 */
CV_EXPORTS_W void cvtColor( InputArray src, OutputArray dst, int code, int dstCn = 0 );

开始实操,就下面这一句,source和dest还能相同，不用建临时的变量，简直完美。

 cv::cvtColor(Frame, Frame, CV_BGR2BGRA);

除此之外，相机居然还改不了输出的图片分辨率，没办法还得自己来，我不需要那么大分辨率，所以把图片分辨率该小就可以提升一点性能。

/** @brief Resizes an image.

The function resize resizes the image src down to or up to the specified size. Note that the
initial dst type or size are not taken into account. Instead, the size and type are derived from
the `src`,`dsize`,`fx`, and `fy`. If you want to resize src so that it fits the pre-created dst,
you may call the function as follows:
@code
    // explicitly specify dsize=dst.size(); fx and fy will be computed from that.
    resize(src, dst, dst.size(), 0, 0, interpolation);
@endcode
If you want to decimate the image by factor of 2 in each direction, you can call the function this
way:
@code
    // specify fx and fy and let the function compute the destination image size.
    resize(src, dst, Size(), 0.5, 0.5, interpolation);
@endcode
To shrink an image, it will generally look best with #INTER_AREA interpolation, whereas to
enlarge an image, it will generally look best with c#INTER_CUBIC (slow) or #INTER_LINEAR
(faster but still looks OK).

@param src input image.
@param dst output image; it has the size dsize (when it is non-zero) or the size computed from
src.size(), fx, and fy; the type of dst is the same as of src.
@param dsize output image size; if it equals zero, it is computed as:
 \f[\texttt{dsize = Size(round(fx*src.cols), round(fy*src.rows))}\f]
 Either dsize or both fx and fy must be non-zero.
@param fx scale factor along the horizontal axis; when it equals 0, it is computed as
\f[\texttt{(double)dsize.width/src.cols}\f]
@param fy scale factor along the vertical axis; when it equals 0, it is computed as
\f[\texttt{(double)dsize.height/src.rows}\f]
@param interpolation interpolation method, see #InterpolationFlags

@sa  warpAffine, warpPerspective, remap
 */
CV_EXPORTS_W void resize( InputArray src, OutputArray dst,
                          Size dsize, double fx = 0, double fy = 0,
                          int interpolation = INTER_LINEAR );

同样的，这个接口的源目也是可以相同的

 cv::resize(Frame, Frame, cv::Size(640, 480));

这样修改之后，就可以用内存块拷贝一次将所有数据给贴图了。这里的FrameTexture可以在FrameTexture大小变化的时候才创建一次，其他时候只需要修改数据块就可以了，同样可以优化程序。

		FTexture2DMipMap& Mip = FrameTexture->PlatformData->Mips[0];
        void* TexData = Mip.BulkData.Lock(LOCK_READ_WRITE);
        FMemory::Memcpy(TexData, Frame.data, FrameList[CurCount].u->size);
        Mip.BulkData.Unlock();
        FrameTexture->UpdateResource();

完整示例

		cv::resize(Frame, Frame, cv::Size(640, 480));//改大小在前，这样后面的操作都会变少
		cv::cvtColor(Frame, Frame, CV_BGR2BGRA);
		FTexture2DMipMap& Mip = FrameTexture->PlatformData->Mips[0];
        void* TexData = Mip.BulkData.Lock(LOCK_READ_WRITE);
        FMemory::Memcpy(TexData, Frame.data, FrameList[CurCount].u->size);
        Mip.BulkData.Unlock();
        FrameTexture->UpdateResource();