Gstreamer信号、属性、消息、事件、问询、状态的梳理
1. 信号signal
信号属于gobject的通用机制,一般用于特定交互。
一般来讲信号是属于元件的,用于元件和外部(应用或者其它元件)之间,基于预定事件的交互,比如元件创建了新的cap,或者gstbin上添加了新的元件,gstbin也会发出element-added信号。
元件内部定义了本元件相关的信号。如果外部需要关心和了解这一信号,可以连接该信号和处理函数。这样,元件内部发生信号时,该函数将被调用执行。
1.1 元件:信号的创建和发送
信号一般在class_init函数进行创建,通过g_signal_new 创建信号.参考api说明.
guint g_signal_new (const gchar *signal_name,
GType itype,//G_TYPE_FROM_CLASS (klass)信号所有者
GSignalFlags signal_flags,//调用顺序
guint class_offset,//默认回调函数在class中的偏移量,没有就设0.
GSignalAccumulator accumulator,//信号的累加器函数,默NULL
gpointer accu_data,//累加器函数参数,默NULL
GSignalCMarshaller c_marshaller,//回调函数的额外参数类型,默NULL
GType return_type,//返回类型,G_TYPE_NONE表示不返回
guint n_params,//额外参数个数
...);
该行为设涉及多个回调函数,首先是用户连接该信号时用户自己设定的回调函数,其次是per-object handler默认回调函数,这些函数的调用顺序由GSignalFlags指定。
accumulator大意时收集各个回调返回值,当返回false的时候信号不再继续执行.
示例代码:gstreamer-1.18.4/plugins/elements/gstfakesink.c
//接口描述:https://docs.gtk.org/gobject/func.signal_new.html
gst_fake_sink_signals[SIGNAL_PREROLL_HANDOFF] =
g_signal_new ("preroll-handoff", G_TYPE_FROM_CLASS (klass),
G_SIGNAL_RUN_LAST, G_STRUCT_OFFSET (GstFakeSinkClass, preroll_handoff),
NULL, NULL, NULL, G_TYPE_NONE, 2,GST_TYPE_BUFFER | G_SIGNAL_TYPE_STATIC_SCOPE, GST_TYPE_PAD);
通过g_signal_emit 发送信号:
//接口描述:https://docs.gtk.org/gobject/func.signal_emit.html
g_signal_emit (sink, gst_fake_sink_signals[SIGNAL_PREROLL_HANDOFF], 0, buffer, bsink->sinkpad);
//或者直接调用名字发送信号
g_signal_emit_by_name(filter,"pad-added");
通过通过g_signal_emitv 发送信号:
gst-plugins-good-1.18.4/gst/rtpmanager/gstrtpsession.c
g_signal_emitv (args, gst_rtp_session_signals[SIGNAL_REQUEST_PT_MAP], 0,
&ret);
其余详见:https://docs.gtk.org/gobject/
1.2 应用程序:信号的连接和处理
信号连接比较简单,直接传入对象,信号名,回调函数即可:
g_signal_connect (sink, "pad-added", G_CALLBACK (pad_added_handler), &data);
1.3 为回调函数增加参数:
信号创建时,GSignalCMarshaller 是执行一个函数类型匹配. 比如回调函数额外有一个int参数,那么就填写g_cclosure_marshal_VOID__INT,对应void (*callback) (gpointer instance, gint arg1, gpointer user_data).
然后n_params写1,再加一个int参数.
//为回调函数新增一个int参数:
g_signal_new("sgn_args",G_TYPE_FROM_CLASS(klass),G_SIGNAL_RUN_FIRST,0,NULL,NULL,
g_cclosure_marshal_VOID__INT,//指回调函数为void XXX(int arg)
G_TYPE_NONE,
1,G_TYPE_INT);//指回调函数增加一个参数,类型为int
//发送时携带int参数传给回调函数,这里为filter->datacount :
g_signal_emit(filter,
g_myfilter_group_signals[SIGNAL_ACTION_ARGS],
0,filter->datacount ) ;
//应用回调函数参数顺序为 instance, arg1, user_data,注意新增的int在中间
void filter_sgn_args_handler(gpointer instance, gint datacount, gpointer user_data)
{
g_print("filter_sgn_args_handler in cnt:%d.\n",datacount);
}
特别要注意, 额外的参数都是放在instance和user_data中间的,不是放在最后面. 如果发现新增的参数传参值和实际拿到的不一样,就留意是不是参数顺序写错了.
但是如果写了g_cclosure_marshal_VOID__INT实际用的不一样运行会报错(比如实际用了两个int参数),默认定义的marshal都只有一个,如果要自己新增的话要重新增加代码和模板,看起来比较麻烦.
不过,简单的是, gst源码中很多地方直接写了NULL. 实测写成NULL直接传两个int没有问题,因此默认写NULL就好了,这里参考资料较少,没有继续深究.
//设置marshal类型为NULL,传入两个int参数
g_signal_new("sgn_args",G_TYPE_FROM_CLASS(klass),G_SIGNAL_RUN_FIRST,0,NULL,NULL,
NULL,
G_TYPE_NONE,
2,G_TYPE_INT,G_TYPE_INT);//传入两个int参数
//发出信号,携带两个int,这里是filter->datacount和99
g_signal_emit(filter, g_myfilter_group_signals[SIGNAL_ACTION_ARGS],
0,filter->datacount,99 ) ;
//注意回调函数的两个int参数还是在中间:
void filter_sgn_args_handler(gpointer instance, gint arg1,gint arg2, gpointer user_data)
{
g_print("filter_sgn_args_handler in cnt:%d %d.\n",arg1, arg2);
}
运行输出:
filter_sgn_args_handler in cnt:200 99.
1.4 用工具查看信号:
信号可以通过gst-inspect-1.0 工具来查看元件的信号,如:
gst-inspect-1.0 ../bin/filter_test my_filter
Factory Details:
Rank none (0)
Long-name MyFilter
Klass FIXME:Generic
Description FIXME:Generic Template Element
Author yuanguochao <<user@hostname.org>>
Plugin Details:
……
Element Signals:
"dataCnt300" : void user_function (GstElement* object,
gpointer user_data);
"sgn-args" : void user_function (GstElement* object,
gint arg0,
gint arg1,
gpointer user_data);
以上也可以看出最后一个信号"sgn-args"的回调函数 : void user_function的参数详情, 是两个int在中间.所以写应用程序不清楚参数的时候,可以直接参考这里的信息就好了.
2. 属性property
属性也是gobject的一种机制,gst中常用于行为和参数控制。
2.1 元件:属性的创建
和信号一样,属性的创建一般也是在class_init函数进行,用g_object_class_install_property初始化,可选地实现_get_property()和_set_property()函数。注意属性的枚举定义,该枚举的ID才是判断属性的关键参数。
在g_object_class_install_property初始化中,该ID会与属性的name作为绑定,由应用通过name来访问属性。
/* properties */
enum {
PROP_0,
PROP_SILENT
/* FILL ME */
};
static void gst_my_filter_set_property (GObject *object,
guint prop_id,
const GValue *value,
GParamSpec *pspec);
static void gst_my_filter_get_property (GObject *object,
guint prop_id,
GValue *value,
GParamSpec *pspec);
static void gst_my_filter_class_init (GstMyFilterClass *klass)
{
GObjectClass *object_class = G_OBJECT_CLASS (klass);
/* define virtual function pointers */
object_class->set_property = gst_my_filter_set_property;
object_class->get_property = gst_my_filter_get_property;
/* define properties */
g_object_class_install_property (object_class, PROP_SILENT,
g_param_spec_boolean ("silent", "Silent",
"Whether to be very verbose or not",
FALSE, G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
}
static void gst_my_filter_set_property (GObject *object,
guint prop_id,
const GValue *value,
GParamSpec *pspec)
{
GstMyFilter *filter = GST_MY_FILTER (object);
switch (prop_id) {
case PROP_SILENT:
filter->silent = g_value_get_boolean (value);
g_print ("Silent argument was changed to %s\n",
filter->silent ? "true" : "false");
break;
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
static void
gst_my_filter_get_property (GObject *object,
guint prop_id,
GValue *value,
GParamSpec *pspec)
{
GstMyFilter *filter = GST_MY_FILTER (object);
switch (prop_id) {
case PROP_SILENT:
g_value_set_boolean (value, filter->silent);
break;
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
2.2 应用程序:属性的访问
应用通过通过g_object_set或者g_object_get读写属性,
也可以通过g_object_set_property和g_object_get_property读写属性。但是使用_property需要通过GValue来转换一次。见https://docs.gtk.org/gobject/class.Object.html。
void g_object_get_property ( GObject* object, const gchar* property_name, GValue* value);
void g_object_set_property ( GObject* object, const gchar* property_name, const GValue* value);
GValue val = {0};
g_value_init(&val,G_TYPE_BOOLEAN);
g_value_set_boolean(&val,1);
gboolean myval=0;
//改写属性
g_object_set(my_filter,"silent",0,NULL);//可以直接使用0来设定,不用初始化为GValue。
g_object_set_property(G_OBJECT (filter),"silent",&val);//必须使用GValue*,不然运行报错
//读取属性
g_object_get_property(filter,"silent",&val);//必须使用GValue*,
g_print("silent:%d\n", g_value_get_boolean(&val));
g_object_get(G_OBJECT (filter),"booltest",&myval,NULL);//使用gboolean
g_print("booltest:%d\n", mytest);
2.3 用工具查看属性
属性可以通过gst-inspect-1.0 工具来查看元件的信号,可以看到属性的类型,读写权限,取值范围等。如:
# gst-inspect-1.0 filesrc
Factory Details:
Rank primary (256)
Long-name File Source
Klass Source/File
......
Element Properties:
blocksize : Size in bytes to read per buffer (-1 = default)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 4096
do-timestamp : Apply current stream time to buffers
flags: readable, writable
Boolean. Default: false
location : Location of the file to read
flags: readable, writable, changeable only in NULL or READY state
3. 消息message
消息是GstMessage,所以是gstreamer所有的形式,和glib无关。消息总是基于总线bus的,gstbin会拦截所在bus上子节点的所有消息,然后根据对应的不同消息,采取一些既定的行为,比如EOS,SEGMENT等。
3.1 元件:消息的创建和发送
消息是针对总线bus的。gst内部是通过gst_element_post_message来在bus上发送消息。对于消息本身创建,不同的元件可能有不同的封装函数。
gst_element_post_message (GstElement * element, GstMessage * message)//向element所在的bus发送message:
一个gstbin没有总线和时钟,所以创建顶层的bin需要用pipeline代替。gstbin会拦截每个子元件的msg,并实现默认消息行为。比如所有的sink pad都发送了EOS,那么gstbin会向上通知EOS消息,应用可以监听bus上的信息来得到这个消息。
3.2 应用程序:接收和处理消息
消息是从bus中获取,因此首先要通过pipeline拿到bus。之后应用可以通过gst_bus_add_watch 注册消息回调来处理,也可以使用gst_bus_poll 轮询消息。
bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
watch_id = gst_bus_add_watch (bus, bus_call, loop);
static gboolean bus_call (GstBus *bus, GstMessage *msg, gpointer data)
{
GMainLoop *loop = data;
switch (GST_MESSAGE_TYPE (msg)) {
case GST_MESSAGE_EOS:
//do sth.
break;
case GST_MESSAGE_ERROR:
break;
default:
break;
}
return TRUE;
}
bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
msg = gst_bus_poll (bus, GST_MESSAGE_ERROR, 0);
if (msg) {
GError *err = NULL;
gst_message_parse_error (msg, &err, NULL);
g_print ("ERROR: %s\n", err->message);
g_error_free (err);
gst_message_unref (msg);
}
4. 事件event
事件是gstreamer针对media定义的一些概念。事件和缓冲区一样,在pipeline往上下游传输。event是绑定到pad的,通过 gst_pad_set_event_function来设置处理函数。一般元件之间会用event相互传递事件。应用程序用的event较少,比如seek。
4.1 元件:处理事件
元件在初始化时,会通过gst_pad_set_event_function设定对应的处理函数。
static void gst_my_filter_init (GstMyFilter * filter)
{
[..]
gst_pad_set_event_function (filter->sinkpad,
gst_my_filter_sink_event);
[..]
}
static gboolean gst_my_filter_sink_event (GstPad *pad, GstObject *parent, GstEvent *event)
{
GstMyFilter *filter = GST_MY_FILTER (parent);
switch (GST_EVENT_TYPE (event)) {
case GST_EVENT_CAPS:
/* we should handle the format here */
break;
case GST_EVENT_EOS:
/* end-of-stream, we should close down all stream leftovers here */
gst_my_filter_stop_processing (filter);
break;
default:
break;
}
return gst_pad_event_default (pad, parent, event);
}
4.2 应用程序:创建和发送事件
每一个事件都有定义对应的创建函数gst_event_new_xxx,然后通过gst_element_send_event往元件发送。
seek event:
static void seek_to_time (GstElement *element, guint64 time_ns)
{
GstEvent *event;
event = gst_event_new_seek (1.0, GST_FORMAT_TIME,
GST_SEEK_FLAG_NONE,
GST_SEEK_METHOD_SET, time_ns,
GST_SEEK_TYPE_NONE, G_GUINT64_CONSTANT (0));
gst_element_send_event (element, event);
}
eos event:
if (ret == GST_FLOW_NOT_LINKED || ret < GST_FLOW_EOS) {
event = gst_event_new_eos ();
/* for fatal errors we post an error message, post the error
* first so the app knows about the error first. */
GST_ELEMENT_FLOW_ERROR (midiparse, ret);
gst_pad_push_event (midiparse->srcpad, event);
}
gstbin默认会向管道上的所有元件转发事件,如果所有元件都返回true,就返回true,否则就返回false。
5. 问询query
GstQuery是向一个element或者pad查询信息,比如当前位置,总时间等。和事件Event一样,它也是绑定到pad的。通过gst_pad_set_query_function来设置某一pad的处理函数。
GstBin实现了默认的行为,GST_QUERY_DURATION和GST_QUERY_POSITION会转发给所有sink并返回最大值,其它的问询,会返回第一个成功应答的sink。
问询类型定义在/gstreamer-1.18.4/gst/gstquery.h中:
typedef enum {
GST_QUERY_UNKNOWN = GST_QUERY_MAKE_TYPE (0, 0),
GST_QUERY_POSITION = GST_QUERY_MAKE_TYPE (10, _FLAG(BOTH)),
GST_QUERY_DURATION = GST_QUERY_MAKE_TYPE (20, _FLAG(BOTH)),
GST_QUERY_LATENCY = GST_QUERY_MAKE_TYPE (30, _FLAG(BOTH)),
GST_QUERY_JITTER = GST_QUERY_MAKE_TYPE (40, _FLAG(BOTH)),
GST_QUERY_RATE = GST_QUERY_MAKE_TYPE (50, _FLAG(BOTH)),
GST_QUERY_SEEKING = GST_QUERY_MAKE_TYPE (60, _FLAG(BOTH)),
GST_QUERY_SEGMENT = GST_QUERY_MAKE_TYPE (70, _FLAG(BOTH)),
GST_QUERY_CONVERT = GST_QUERY_MAKE_TYPE (80, _FLAG(BOTH)),
GST_QUERY_FORMATS = GST_QUERY_MAKE_TYPE (90, _FLAG(BOTH)),
GST_QUERY_BUFFERING = GST_QUERY_MAKE_TYPE (110, _FLAG(BOTH)),
GST_QUERY_CUSTOM = GST_QUERY_MAKE_TYPE (120, _FLAG(BOTH)),
GST_QUERY_URI = GST_QUERY_MAKE_TYPE (130, _FLAG(BOTH)),
GST_QUERY_ALLOCATION = GST_QUERY_MAKE_TYPE (140, _FLAG(DOWNSTREAM) | _FLAG(SERIALIZED)),
GST_QUERY_SCHEDULING = GST_QUERY_MAKE_TYPE (150, _FLAG(UPSTREAM)),
GST_QUERY_ACCEPT_CAPS = GST_QUERY_MAKE_TYPE (160, _FLAG(BOTH)),
GST_QUERY_CAPS = GST_QUERY_MAKE_TYPE (170, _FLAG(BOTH)),
GST_QUERY_DRAIN = GST_QUERY_MAKE_TYPE (180, _FLAG(DOWNSTREAM) | _FLAG(SERIALIZED)),
GST_QUERY_CONTEXT = GST_QUERY_MAKE_TYPE (190, _FLAG(BOTH)),
GST_QUERY_BITRATE = GST_QUERY_MAKE_TYPE (200, _FLAG(DOWNSTREAM)),
} GstQueryType;
5.1 元件:处理问询
同Event一样,一般在class_init函数设定号问询的处理函数:
static gboolean gst_ogg_pad_src_query (GstPad * pad, GstObject * parent, GstQuery * query)
{
gboolean res = TRUE;
GstOggDemux *ogg;
ogg = GST_OGG_DEMUX (parent);
switch (GST_QUERY_TYPE (query)) {
case GST_QUERY_POSITION:
{
GstFormat format;
GstOggPad *ogg_pad = GST_OGG_PAD (pad);
gst_query_parse_position (query, &format, NULL);
/* can only get position in time */
if (format != GST_FORMAT_TIME)
goto wrong_format;
gst_query_set_position (query, format, ogg_pad->position);
break;
...
}
static void gst_ogg_pad_init (GstOggPad * pad)
{
gst_pad_set_event_function (GST_PAD (pad), GST_DEBUG_FUNCPTR (gst_ogg_pad_event));
gst_pad_set_query_function (GST_PAD (pad), GST_DEBUG_FUNCPTR (gst_ogg_pad_src_query));
gst_pad_use_fixed_caps (GST_PAD (pad));
也可以直接通过gstbasesrc_class来赋值:
GstBaseSrcClass *gstbasesrc_class;
gobject_class = (GObjectClass *) klass;
gstelement_class = (GstElementClass *) klass;
gstbasesrc_class = GST_BASE_SRC_CLASS (klass);
... ...
gstbasesrc_class->start = GST_DEBUG_FUNCPTR (rsn_dvdsrc_start);
gstbasesrc_class->stop = GST_DEBUG_FUNCPTR (rsn_dvdsrc_stop);
gstbasesrc_class->unlock = GST_DEBUG_FUNCPTR (rsn_dvdsrc_unlock);
gstbasesrc_class->unlock_stop = GST_DEBUG_FUNCPTR (rsn_dvdsrc_unlock_stop);
gstbasesrc_class->event = GST_DEBUG_FUNCPTR (rsn_dvdsrc_src_event);
gstbasesrc_class->query = GST_DEBUG_FUNCPTR (rsn_dvdsrc_src_query);
gstbasesrc_class->is_seekable = GST_DEBUG_FUNCPTR (rsn_dvdsrc_is_seekable);
5.2 应用程序:创建和发送问询
GstQuery 定义了不同的Query类型,因此有不同的创建函数和对应的处理函数,见gstreamer-1.18.4/gst/gstquery.h
/* seeking query */
GST_API GstQuery* gst_query_new_seeking (GstFormat format) G_GNUC_MALLOC;
GST_API void gst_query_set_seeking (GstQuery *query, GstFormat format,
gboolean seekable,
gint64 segment_start,
gint64 segment_end);
GST_API void gst_query_parse_seeking (GstQuery *query, GstFormat *format,
gboolean *seekable,
gint64 *segment_start,
gint64 *segment_end);
/* segment query */
GST_API GstQuery* gst_query_new_segment (GstFormat format) G_GNUC_MALLOC;
GST_API void gst_query_set_segment (GstQuery *query, gdouble rate, GstFormat format,
gint64 start_value, gint64 stop_value);
GST_API void gst_query_parse_segment (GstQuery *query, gdouble *rate, GstFormat *format,
gint64 *start_value, gint64 *stop_value);
通过gst_element_query 向gstbin发送问询,根据结果进行下一步处理:
/* We just moved to PLAYING. Check if seeking is possible */
GstQuery *query;
gint64 start, end;
query = gst_query_new_seeking (GST_FORMAT_TIME);
if (gst_element_query (data->playbin, query)) {
gst_query_parse_seeking (query, NULL, &data->seek_enabled, &start, &end);
if (data->seek_enabled) {
g_print ("Seeking is ENABLED from %" GST_TIME_FORMAT " to %" GST_TIME_FORMAT "\n",
GST_TIME_ARGS (start), GST_TIME_ARGS (end));
} else {
g_print ("Seeking is DISABLED for this stream.\n");
}
}
else {
g_printerr ("Seeking query failed.");
}
gst_query_unref (query);
6. 状态
状态是针对一条pipeline上的元件的,每一个元件都有一个状态机,包含“NULL”, “READY”, “PAUSED” 和“PLAYING”。不同的基础类型都有对应的处理函数,因此继承时不需要特别关心,重写start() 和stop()即可 。muxer和demuxer没有基础类,所以需要实现自己的状态转换函数。
6.1 元件:创建状态处理函数
一般在class_init函数中完成函数的实现,并赋值给change_state:
static GstStateChangeReturn gst_ffmpegmux_change_state (GstElement * element, GstStateChange transition)
{
GstStateChangeReturn ret;
GstFFMpegMux *ffmpegmux = (GstFFMpegMux *) (element);
switch (transition) {
case GST_STATE_CHANGE_NULL_TO_READY:
break;
case GST_STATE_CHANGE_READY_TO_PAUSED:
gst_collect_pads_start (ffmpegmux->collect);
break;
case GST_STATE_CHANGE_PAUSED_TO_PLAYING:
break;
case GST_STATE_CHANGE_PAUSED_TO_READY:
gst_collect_pads_stop (ffmpegmux->collect);
break;
default:
break;
}
ret = GST_ELEMENT_CLASS (parent_class)->change_state (element, transition);
switch (transition) {
case GST_STATE_CHANGE_PLAYING_TO_PAUSED:
break;
case GST_STATE_CHANGE_PAUSED_TO_READY:
gst_tag_setter_reset_tags (GST_TAG_SETTER (ffmpegmux));
if (ffmpegmux->opened) {
ffmpegmux->opened = FALSE;
gst_ffmpegdata_close (ffmpegmux->context->pb);
}
break;
case GST_STATE_CHANGE_READY_TO_NULL:
break;
default:
break;
}
return ret;
}
static void gst_ffmpegmux_class_init (GstFFMpegMuxClass * klass)
{
GObjectClass *gobject_class;
GstElementClass *gstelement_class;
gobject_class = (GObjectClass *) klass;
gstelement_class = (GstElementClass *) klass;
...
gstelement_class->change_state = gst_ffmpegmux_change_state;
}
6.2 应用程序:设定状态
gst_element_set_state函数用于状态设定,该函数最终会执行上述的状态处理函数。一般是直接设定整条pipeline上所有元件的状态。
gst_element_set_state(pipeline, GST_STATE_PLAYING);
...
gst_element_set_state(pipeline, GST_STATE_NULL);
7. 总结
信号signal和属性property都是glib原有的机制,两者基本是基于元件的。
信号signal一般用于元件内部发出特定事件的通知,比如内部新增了成员等等。
属性property一般用于向外界提供内部特定变量的访问,比如全局变量配置,字符串设定等,通过对应的set和get函数来读写。
这两者都是不固定的,每个元件都可以定义的信号和属性。
事件event和问询query是gstreamer机制。他们一般属于对外界特定控制访问的处理接口。事件在一定程度上会和消息关联,比如eos事件可能会促使gstbin向整条bus向上发出EOS消息。
事件和问询的类型是相对固定的,在相关头文件都有对应每个事件和问询的创建函数。
消息message是对于bus总线的,是固定的,在GstMessageType有明确的定义。
状态是整条pipeline的统一控制接口,也是固定的,会对管道上所有的节点进行操作。