公司让我研究gstreamer,过程中踩坑无数，现将经历写下来
什么是gstreamer
官网：https://gstreamer.freedesktop.org/
gstreamer是一个跨平台的播放器，底层用C语言写的，上层用C#,python,java封装过，由于公司有用C#播放视频的场景，所以就决定尝试gstreamer
踩坑开始
了解了半天这个库，发现需要下载安装，然后才能使用C#调用！！！
下载gstreamer
到官网，找到下载页：https://gstreamer.freedesktop.org/data/pkg/windows/
一看，天呐，这么多版本，那就下载最新的windows版本吧
点V1.18.1版本2020-10-29 16:36 更新的，点进去，如下：
有mingw/msvc/uwp版本，选哪个呢？uwp是手机上用的肯定不用选。
后来发现这两个编译版本不一样，我都下载了比较的！！！！
mingw:核心的C库前面带有lib前缀
msvc:核心的C库前面不带lib前缀
如下图比较截图：
如果你下载错了，会发现C#调用动不动找不到函数，找不到入口点
安装gstreamer
我这里选择mingw版本安装
安装之后，会自动将安装路径加入到环境变量GSTREAMER_1_0_ROOT_MINGW_X86_64
如果选择安装的是msvc版本,会自动将安装路径加入到环境变量GSTREAMER_1_0_ROOT_MSVC_X86_64
将安装路径下的bin目录设置到环境变量Path里去
这一步是非常关键的，决定着C#能否调用gstreamer成功，如果没有把bin目录设置到环境变量，则C#会报找不到ligstreamer-1.0.0.dll或者找不到gstreamer-1.0.0.dll
上一步安装时，会自动设置一个环境变量，我们可以引用这个环境变量，后面加个bin的目录就可以了：
我这里是mingw版本，在Path环境变量后面加入：%GSTREAMER_1_0_ROOT_MINGW_X86_64%\bin
如果安装的是msvc版本，在Path环境变量后面加入：%GSTREAMER_1_0_ROOT_MSVC_X86_64%\bin
设置完之后，安装目录的bin目录下有个gst-play-1.0.exe,这个可以用来播放视频。
有个测试视频播放的网址集合：https://blog.csdn.net/qq_17497931/article/details/80824328
随便测试一个，按Ctrl+R,打开命令终端（注意不要cd到安装的bin目录），输入
gst-play-1.0.exe https://stream7.iqilu.com/10339/upload_transcode/202002/18/20200218114723HDu3hhxqIT.mp4
如果能播放，则说明将bin目录加入到Path中成功了：
C#调用Gstreamer：gstreamer-sharp
为了确保C#能调用到gstreamer,要确保如下事情：
1.安装路径的bin目录成功加入到环境变量Path中，这样gstreamer-sharp就能找到libgstreamer-1.0.0.dll
2.如果安装的gstreamer是64位的，就必须将C#的项目编译设置为x64,如下所示：
gstreamer-sharp在github上只是镜像，真正的仓库在gitlab上：
github网址：https://github.com/gstreamer-sharp/gstreamer-sharp
真正的官网：https://gitlab.freedesktop.org/gstreamer/gstreamer-sharp
github上有一些例子，但是有的可能还是不能跑。但是例子具有参考价值：
https://github.com/vvvv/VL.GStreamer
https://github.com/ttustonic/GStreamerSharpSamples
https://github.com/vladkol/gstreamer-netcore
首先新建一个winform的项目，在nuget搜索gstsharp:
最新版本是V1.18.0,我安装的是V1.16.0.
最新版本有BUG.问题见这里：
https://gitlab.freedesktop.org/gstreamer/gstreamer-sharp/-/issues/46
而且，最新的版本引用的dll都是gstreamer-1.0.0.dll是针对msvc版本的，我们安装的是mingw版本。可以看看V1.18.0 gstreamer-sharp的github源码：
https://hub.fastgit.org/GStreamer/gstreamer-sharp/blob/master/sources/generated/Gst/Application.cs
// This file was generated by the Gtk# code generator.
// Any changes made will be lost if regenerated.
namespace Gst {
using System;
using System.Runtime.InteropServices;
#region Autogenerated code
public partial class Application {
[DllImport("gstreamer-1.0-0.dll", CallingConvention = CallingConvention.Cdecl)]
static extern void gst_version(out uint major, out uint minor, out uint micro, out uint nano);
public static void Version(out uint major, out uint minor, out uint micro, out uint nano) {
gst_version(out major, out minor, out micro, out nano);
}
[DllImport("gstreamer-1.0-0.dll", CallingConvention = CallingConvention.Cdecl)]
static extern IntPtr gst_version_string();
public static string VersionString() {
IntPtr raw_ret = gst_version_string();
string ret = GLib.Marshaller.PtrToStringGFree(raw_ret);
return ret;
}
#endregion
}
}
还有一种办法：
下载V1.18版本的代码，将所有DllImport的地方加上lib然后编译也是可以运行的。别问我怎么知道的。我不会告诉你我改了3000多个地方，最后可以运行！！！！
参考winform其他例子，写代码
传入Winform空间的句柄，这样写，最后可以播放视频，但是第一次总是弹出系统自带的视频窗口，无法嵌入到窗体中，所有的地方都尝试过了，无法解决！！！
失败例子代码如下：
public partial class Form1 : Form
{
Gst.Element playbin;
GLib.MainLoop mainLoop;
public Form1()
{
InitializeComponent();
Gst.Application.Init();
mainLoop = new GLib.MainLoop();
System.Threading.ThreadPool.QueueUserWorkItem(x => mainLoop.Run());
this.Load += Form1_Load;
}
private void Form1_Load(object sender, EventArgs e)
{
playbin = ElementFactory.Make("playbin", "playbin");
playbin["uri"] = "http://stream4.iqilu.com/ksd/video/2020/02/17/97e3c56e283a10546f22204963086f65.mp4";
var overlay= ((Gst.Bin)playbin).GetByInterface(Gst.Video.VideoOverlayAdapter.GType);
Gst.Video.VideoOverlayAdapter adapter = new Gst.Video.VideoOverlayAdapter(overlay.Handle);
adapter.WindowHandle = this.panel1.Handle;
adapter.HandleEvents(true);
playbin.SetState( State.Playing);
}
private void btnPlay_Click(object sender, EventArgs e)
{
playbin.SetState( State.Playing);
}
private void btnPause_Click(object sender, EventArgs e)
{
playbin.SetState(State.Paused);
}
private void btnStop_Click(object sender, EventArgs e)
{
playbin.SetState(State.Ready);
}
}
其中这一句adapter.WindowHandle = this.panel1.Handle;就是传入控件的句柄，参考其他人写的例子。会导致第一次播放弹出自带的D3DD窗口。如下所示：
弹出的是自带的视频播放窗口！！！
从其他例子中总结出来的播放方法
这个播放的思路就是一只拉取视频的数据，转换为图片，一直刷新pixturebox的Image来达到播放视频的目的（注意不能刷新panel的backgroundImage，播放视频会闪烁，我不会告诉你我试过的）
代码如下：要引用VL.Core
public partial class Form1 : Form
{
Gst.Element playbin;
GLib.MainLoop mainLoop;
Gst.App.AppSink videosink;
public Form1()
{
InitializeComponent();
Gst.Application.Init();
mainLoop = new GLib.MainLoop();
System.Threading.ThreadPool.QueueUserWorkItem(x => mainLoop.Run());
GLib.Timeout.Add(1000, () =>
{
return true;
});
System.Threading.ThreadPool.QueueUserWorkItem(x =>
{
while (true)
{
if (playbin == null) continue ;
if (videosink == null) continue;
var sample = videosink.TryPullSample(500);
if(sample!=null)
{
if (sample == null) continue ;
GstreamerTest.Unnormal.Image image = new GstreamerTest.Unnormal.Image(sample);
this.Invoke(new Action(() =>
{
this.pictureBox1.Image?.Dispose();
this.pictureBox1.Image = image.FromImage(true);
}));
image.Dispose();
sample.Dispose();
}
System.Threading.Thread.Sleep(10);
}
});
this.Load += Form1_Load;
}
private void Form1_Load(object sender, EventArgs e)
{
playbin = ElementFactory.Make("playbin", "playbin");
playbin["uri"] = "http://stream4.iqilu.com/ksd/video/2020/02/17/97e3c56e283a10546f22204963086f65.mp4";
videosink = new Gst.App.AppSink("videosink");
videosink.Sync = true;
videosink.Qos = false;
videosink.Drop = false;
videosink.Caps = Caps.FromString($"video/x-raw, format=RGBA");
videosink.MaxBuffers = 1;
videosink.EmitSignals = true;
playbin["video-sink"] = videosink;
playbin.SetState(State.Playing);
}
private void btnPlay_Click(object sender, EventArgs e)
{
playbin.SetState(State.Playing);
}
private void btnPause_Click(object sender, EventArgs e)
{
playbin.SetState(State.Paused);
}
private void btnStop_Click(object sender, EventArgs e)
{
playbin.SetState(State.Ready);
}
}
最后播放的画面如下：
本实例的代码：https://github.com/lishuangquan1987/gstreamer-sharp-test
最终的稳定使用C#播放Gstreamer的方法
这种方法会因为图片大小转换处理耗时长而导致一定的丢帧，CPU占比会有10%左右，不过不影响使用。如果谁有C#高效率的图片大小转换的方法，请告知我一下
为什么要做图片大小转换
视频输出的图片大概是固定大小的，而我们的播放器的播放画面是可以变化大小的，所以要把图片转换为播放器画面的大小。
最新的例子是在github:https://github.com/lishuangquan1987/gstreamer-sharp-test
还是贴代码：
public partial class GStreamerPlayerControl : UserControl
{
private Element playbin;
private Gst.App.AppSink videosink;
/// <summary>
/// 播放总时长
/// </summary>
TimeSpan totalTime;
/// <summary>
/// 播放当前位置
/// </summary>
TimeSpan currentPosition;
/// <summary>
/// 播放位置变化触发的事件
/// </summary>
public event Action<TimeSpan, TimeSpan> PositionChanged;
/// <summary>
/// 播放状态变化触发的事件
/// </summary>
public event Action<State> StatusChanged;
/// <summary>
/// 播放总帧数（会随着播放变化）
/// </summary>
public long TotalFrame { get; set; }
/// <summary>
/// 因图片转换慢而丢弃的帧数（会随着播放变化）
/// </summary>
public long IgnoreFrame { get; set; }
private ConcurrentStack<Bitmap> cacheImageList = new ConcurrentStack<Bitmap>();
private State state;
GLib.MainLoop mainLoop;
public GStreamerPlayerControl()
{
InitializeComponent();
Type type = this.pictureBox.GetType();
System.Reflection.PropertyInfo pi = type.GetProperty("DoubleBuffered",
System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic);
pi.SetValue(this.pictureBox, true, null);
this.Load += GStreamerPlayerControl_Load;
Init();
//this.SizeChanged += (sender, e) =>
//{
// this.pictureBox.Height = this.Height - 100;
//};
}
private void GStreamerPlayerControl_Load(object sender, EventArgs e)
{
ThreadPool.QueueUserWorkItem(x =>
{
while (!IsDisposed)
{
Thread.Sleep(10);
if (playbin == null)
continue;
if (videosink == null)
continue;
if (!this.pictureBox.IsHandleCreated) continue;
var sample = videosink.TryPullSample(10);
if (sample == null)
{
continue;
}
TotalFrame++;
var image = ConvertSampleToImage(sample);
if (this.cacheImageList.Count > 0)
{
IgnoreFrame += this.cacheImageList.Count;
this.cacheImageList.Clear();
}
this.cacheImageList.Push(image);
sample.Dispose();
}
});
ThreadPool.QueueUserWorkItem(x =>
{
while (!IsDisposed)
{
Thread.Sleep(500);
if (playbin == null)
continue;
if (videosink == null)
continue;
if (playbin.QueryDuration(Gst.Format.Time, out long duration) &&
playbin.QueryPosition(Gst.Format.Time, out long pos))
{
var tsPos = new TimeSpan(pos / 100).RoundSeconds(0);
var tsDur = new TimeSpan(duration / 100).RoundSeconds(0);
if (tsPos != currentPosition || tsDur != totalTime)
{
currentPosition = tsPos;
totalTime = tsDur;
PositionChanged?.Invoke(currentPosition, totalTime);
}
}
var ret = playbin.GetState(out var playStatus, out var pending, Gst.Constants.SECOND * 5L);
if (ret == StateChangeReturn.Success && playStatus != state)
{
state = playStatus;
StatusChanged?.Invoke(state);
}
}
});
//处理图片线程
ThreadPool.QueueUserWorkItem(x =>
{
while (!IsDisposed)
{
Thread.Sleep(10);
if (cacheImageList.TryPop(out Bitmap bitmap))
{
try
{
var imageToShow = bitmap.ToSize(this.pictureBox.Width, this.pictureBox.Height);
this.Invoke(new Action(() =>
{
this.pictureBox.Image?.Dispose();
this.pictureBox.Image = imageToShow;
}));
}
catch (Exception ee)
{ }
}
}
});
}
private void Init()
{
Gst.Application.Init();
playbin = ElementFactory.Make("playbin", "playbin");
if (!string.IsNullOrEmpty(Url))
{
playbin["uri"] = Url;
}
videosink = new Gst.App.AppSink("videosink");
videosink.Sync = true;
videosink.Qos = false;
videosink.Drop = false;
videosink.Caps = Caps.FromString($"video/x-raw, format={videoFormat.ToString().ToUpperInvariant()}");
videosink.MaxBuffers = 1;
videosink.EmitSignals = true;
playbin["video-sink"] = videosink;
mainLoop = new GLib.MainLoop();
ThreadPool.QueueUserWorkItem(x => mainLoop.Run());//must be run to collect memory
}
private string url;
/// <summary>
/// 视频播放地址
/// </summary>
public string Url
{
get { return url; }
set
{
url = value;
if (playbin != null)
{
playbin["uri"] = value;
}
}
}
private Gst.Video.VideoFormat videoFormat = Gst.Video.VideoFormat.Bgra;
/// <summary>
/// 视频格式
/// </summary>
public Gst.Video.VideoFormat VideoFormat
{
get { return videoFormat; }
set
{
videoFormat = value;
videosink.Caps = Caps.FromString($"video/x-raw, format={videoFormat.ToString().ToUpperInvariant()}");
}
}
public void Play()
{
playbin.SetState(State.Playing);
}
public void Play(string url)
{
Url = url;
TotalFrame = 0;
IgnoreFrame = 0;
Play();
}
public void Pause()
{
playbin.SetState(State.Paused);
}
public void Stop()
{
playbin.SetState(State.Ready);
}
private Bitmap ConvertSampleToImage(Sample sample, Gst.Video.VideoFormat videoFormat = Gst.Video.VideoFormat.Gbra)
{
using (var caps = sample.Caps)
using (var structure = caps.GetStructure(0))
{
int width, height;
structure.GetInt("width", out width);
structure.GetInt("height", out height);
var formatStr = structure.GetString("format");
var format = ConvertVideoFormatStrToPixelFormat(formatStr);
var img = new System.Drawing.Bitmap(width, height, format);
var imageData = img.LockBits(new System.Drawing.Rectangle(0, 0, img.Width, img.Height), System.Drawing.Imaging.ImageLockMode.ReadWrite, img.PixelFormat);
var pointer = imageData.Scan0;
var length = imageData.Stride * imageData.Height;
sample.Buffer.Map(out MapInfo mapInfo, MapFlags.Read);
var data = mapInfo.Data;
System.Runtime.InteropServices.Marshal.Copy(data, 0, pointer, length);
img.UnlockBits(imageData);
data = null;
sample.Buffer.Unmap(mapInfo);//这一句一定要加，否则内存飙升
return img;
}
}
private System.Drawing.Imaging.PixelFormat ConvertVideoFormatStrToPixelFormat(string format)
{
Gst.Video.VideoFormat videoFormat;
if (!Enum.TryParse(format, true, out videoFormat))
return System.Drawing.Imaging.PixelFormat.Undefined;
if (videoFormat == Gst.Video.VideoFormat.Encoded ||
videoFormat == Gst.Video.VideoFormat.I420 ||
videoFormat == Gst.Video.VideoFormat.Yv12 ||
videoFormat == Gst.Video.VideoFormat.Yuy2 ||
videoFormat == Gst.Video.VideoFormat.Uyvy ||
videoFormat == Gst.Video.VideoFormat.Ayuv ||
videoFormat == Gst.Video.VideoFormat.Rgbx ||
videoFormat == Gst.Video.VideoFormat.Bgrx)
{
return System.Drawing.Imaging.PixelFormat.Format32bppRgb;
}
else if (videoFormat == Gst.Video.VideoFormat.Xrgb ||
videoFormat == Gst.Video.VideoFormat.Xbgr ||
videoFormat == Gst.Video.VideoFormat.Rgba ||
videoFormat == Gst.Video.VideoFormat.Bgra)
{
return System.Drawing.Imaging.PixelFormat.Format32bppArgb;
}
else if (videoFormat == Gst.Video.VideoFormat.Argb ||
videoFormat == Gst.Video.VideoFormat.Abgr ||
videoFormat == Gst.Video.VideoFormat.Rgb)
{
return System.Drawing.Imaging.PixelFormat.Format24bppRgb;
}
else
{
return System.Drawing.Imaging.PixelFormat.Undefined;
}
}
}
1040
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
