Part1前言
在用UE5进行项目开发的时候,进程需要进行安防视频的播放,如rtsp的视频流。本文针对这一需求,开发了基于UE5的InVideo插件,使用了安防视频的播放。
![d8add40d4ef500f246d68c2171ed7dcd.jpeg](https://i-blog.csdnimg.cn/blog_migrate/fe5532b83bd7baa1888d749e93d54e96.jpeg)
Part2实现思路
视频流的获取基于opencv库实现,之后将获取的数据转成Texture2D对象,再通过Image组件进行更新播放。另外,为了提高渲染的性能,我将所有的操作都移植到子系统进行,从而避免game线程卡主,影响整体渲染性能。
1引用opencv库
我们在InOpenCV.Build.cs文件中,设置相关依赖库的加载。方法如下:
// Copyright Epic Games, Inc. All Rights Reserved.
using System;
using System.IO;
using UnrealBuildTool;
public class InOpenCV : ModuleRules
{
public InOpenCV(ReadOnlyTargetRules Target) : base(Target)
{
Type = ModuleType.External;
string PlatformDir = Target.Platform.ToString();
string IncPath = Path.Combine(ModuleDirectory, "include");
string BinaryPath = Path.GetFullPath(Path.Combine(ModuleDirectory, "../../../Binaries/ThirdParty", PlatformDir));
if (Target.Platform == UnrealTargetPlatform.Win64)
{
PublicSystemIncludePaths.Add(IncPath);
string LibPath = Path.Combine(ModuleDirectory, "lib", PlatformDir);
string LibName = "opencv_world460";
string LibFfmpeg = "opencv_videoio_ffmpeg460_64";
PublicAdditionalLibraries.Add(Path.Combine(LibPath, LibName + ".lib"));
string DLLName = LibName + ".dll";
string DLLFfmpeg = LibFfmpeg + ".dll";
PublicDelayLoadDLLs.Add(DLLName);
PublicDelayLoadDLLs.Add(DLLFfmpeg);
RuntimeDependencies.Add(Path.Combine(BinaryPath, DLLName));
RuntimeDependencies.Add(Path.Combine(BinaryPath, DLLFfmpeg));
PublicDefinitions.Add("OPENCV_PLATFORM_PATH=Binaries/ThirdParty/" + PlatformDir);
PublicDefinitions.Add("OPENCV_DLL_NAME=" + DLLName);
PublicDefinitions.Add("OPENCV_DLL_FFMPEG=" + DLLFfmpeg);
}
else // unsupported platform
{
}
}
}
2使用Opencv库
由于虚幻的check宏定义和opencv的check冲突,所以直接使用编译是无法通过的。我们需要是使用头文件的时候,增加PreOpenCVHeaders和PostOpenCVHeaders两个文件,一前一后。
#include "PreOpenCVHeaders.h"
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/videoio.hpp>
#include "PostOpenCVHeaders.h"
3子线程
为了提高渲染效率,我们采用FRunnable方式使用子线程进行相关操作。
创建子线程
m_Thread = FRunnableThread::Create(this, TEXT("Video Thread"));
线程同步
由于UTexture2D只能在GameThread进行创建,但是我们使用是在子线程进行,所以这里需要进行线程同步。我们采用事件的方式进行。示例如下:
FEvent* SyncEvent = FGenericPlatformProcess::GetSynchEventFromPool(false);
AsyncTask(ENamedThreads::GameThread, [this, SyncEvent]()
{
VideoTexture = UTexture2D::CreateTransient(m_VideoSize.X, m_VideoSize.Y);
if (VideoTexture)
{
VideoTexture->UpdateResource();
}
m_Texture2DResource = (FTexture2DResource*)VideoTexture->GetResource();
UE_LOG(LogTemp, Log, TEXT("UInVideoWidget UpdateTexture CreateTransient"));
SyncEvent->Trigger();
});
SyncEvent->Wait();
FGenericPlatformProcess::ReturnSynchEventToPool(SyncEvent);
原子对象
在控制线程退出时,我们通过原子变量进行控制。示例如下
TAtomic<bool> m_Stopping = false;
while (false == m_Stopping)
{
}
4更新纹理
再获取到数据之后,我们直接将数据更新到渲染线程的纹理中,不再进行gpu进行,示例代码如下 另外,UE5也提供了UpdateTextureRegions函数,但是经过测试这个函数只能在GameThread进行使用,我们改造了一下,使得其可以适应在子线程中运行。
for (int y = 0; y < m_VideoSize.Y; y++)
{
for (int x = 0; x < m_VideoSize.X; x++)
{
int i = x + (y * m_VideoSize.X);
Data[i].B = m_WrapOpenCv->m_Frame.data[i * 3 + 0];
Data[i].G = m_WrapOpenCv->m_Frame.data[i * 3 + 1];
Data[i].R = m_WrapOpenCv->m_Frame.data[i * 3 + 2];
}
}
UpdateTextureRegions(VideoTexture,(int32)0, (uint32)1, m_VideoUpdateTextureRegion, (uint32)(4 * m_VideoSize.X), (uint32)4, (uint8*)Data.GetData(), false);
void UInVideoWidget::UpdateTextureRegions(UTexture2D* Texture, int32 MipIndex, uint32 NumRegions, FUpdateTextureRegion2D* Regions, uint32 SrcPitch, uint32 SrcBpp, uint8* SrcData, bool bFreeData)
{
if (m_Texture2DResource)
{
struct FUpdateTextureRegionsData
{
FTexture2DResource* Texture2DResource;
int32 MipIndex;
uint32 NumRegions;
FUpdateTextureRegion2D* Regions;
uint32 SrcPitch;
uint32 SrcBpp;
uint8* SrcData;
};
FUpdateTextureRegionsData* RegionData = new FUpdateTextureRegionsData;
RegionData->Texture2DResource = m_Texture2DResource;
RegionData->MipIndex = MipIndex;
RegionData->NumRegions = NumRegions;
RegionData->Regions = Regions;
RegionData->SrcPitch = SrcPitch;
RegionData->SrcBpp = SrcBpp;
RegionData->SrcData = SrcData;
ENQUEUE_RENDER_COMMAND(UpdateTextureRegionsData)([RegionData, bFreeData](FRHICommandListImmediate& RHICmdList) {
for (uint32 RegionIndex = 0; RegionIndex < RegionData->NumRegions; ++RegionIndex)
{
int32 CurrentFirstMip = RegionData->Texture2DResource->GetCurrentFirstMip();
if (RegionData->MipIndex >= CurrentFirstMip)
{
RHIUpdateTexture2D(
RegionData->Texture2DResource->GetTexture2DRHI(),
RegionData->MipIndex - CurrentFirstMip,
RegionData->Regions[RegionIndex],
RegionData->SrcPitch,
RegionData->SrcData
+ RegionData->Regions[RegionIndex].SrcY * RegionData->SrcPitch
+ RegionData->Regions[RegionIndex].SrcX * RegionData->SrcBpp
);
}
}
if (bFreeData)
{
FMemory::Free(RegionData->Regions);
FMemory::Free(RegionData->SrcData);
}
delete RegionData;
});
}
}
5更新视频
直接调用ImageVideo更新纹理即可实现视频的播放
ImageVideo->SetBrushFromTexture(VideoTexture);
6插件使用
新建一个蓝图widget继承自InVideoWgidget,并新建一个Image对象,重新命名为ImageVideo。即可进行调用,可以参考videoUmg组件。
打开视频方法
![0bc48da1bee728af74c00c57891bb214.png](https://i-blog.csdnimg.cn/blog_migrate/f0877b8d8ba3c6153820a666a1c2d458.png)
关闭视频方法
![0213970844f01bed6e35411045aa1063.png](https://i-blog.csdnimg.cn/blog_migrate/89bd5ff05e7e2c438e2fa1cf623e75b4.png)
Part3总结
本文主要介绍了一种基于UE5的安防视频播放思路。最后划重点,项目开源地址如下:https://github.com/inveta/InVideo
Part4关于IN VETA
IN VETA是一支由建模、美术、UE5组成的年轻团队。
![f196be7a19c87748bd907aaa9cc92f2b.jpeg](https://i-blog.csdnimg.cn/blog_migrate/a3e82431f27aefe19acb211d48c6525e.jpeg)
我们致力于三维数字孪生技术分享与研发。
欢迎与各界朋友一起探讨技术与商务合作,一起推动数字孪生的发展。