简介
在多媒体应用中,我们通常需要查询媒体文件的总时间、当前播放位置,以及跳转到指定的时间点。GStreamer提供了相应的接口来实现此功能,在本文中,我们将通过示例了解如何查询时间信息,以及如何进行跳转到指定位置。
GStreamer查询机制
GStreamer提供了GstQuery的查询机制,用于查询Element或Pad的相应信息。例如:查询当前的播放速率,产生的延迟,是否支持跳转等。可查看GstQuery文档了解所支持的类型。
要查询所需的信息,首先需要构造一个查询的类型,然后使用Element或Pad的查询接口获取数据,最终再解析相应结果。 下面的例子介绍了如何使用GstQuery查询Pipeline的总时间:
GstQuery *query = gst_query_new_duration (GST_FORMAT_TIME); gboolean res = gst_element_query (pipeline, query); if (res) { gint64 duration; gst_query_parse_duration (query, NULL, &duration); g_print ("duration = %"GST_TIME_FORMAT, GST_TIME_ARGS (duration)); } else { g_print ("duration query failed..."); } gst_query_unref (query);
示例代码
在本示例中,我们通过查询Pipeline是否支持跳转(seeking),如果支持跳转(有些媒体不支持跳转,例如实时视频),我们会在播放10秒后跳转到其他位置。
在以前的示例中,我们在Pipeline开始执行后,只等待ERROR和EOS消息,然后退出。本例中,我们会在消息等在中设置等待超时时间,超时后,我们会去查询当前播放的时间,用于显示,这与播放器的进度条类似。
![](https://i-blog.csdnimg.cn/blog_migrate/8f900a89c6347c561fdf2122f13be562.gif)
![](https://i-blog.csdnimg.cn/blog_migrate/961ddebeb323a10fe0623af514929fc1.gif)
#include <gst/gst.h> /* Structure to contain all our information, so we can pass it around */ typedef struct _CustomData { GstElement *playbin; /* Our one and only element */ gboolean playing; /* Are we in the PLAYING state? */ gboolean terminate; /* Should we terminate execution? */ gboolean seek_enabled; /* Is seeking enabled for this media? */ gboolean seek_done; /* Have we performed the seek already? */ gint64 duration; /* How long does this media last, in nanoseconds */ } CustomData; /* Forward definition of the message processing function */ static void handle_message (CustomData *data, GstMessage *msg); int main(int argc, char *argv[]) { CustomData data; GstBus *bus; GstMessage *msg; GstStateChangeReturn ret; data.playing = FALSE; data.terminate = FALSE; data.seek_enabled = FALSE; data.seek_done = FALSE; data.duration = GST_CLOCK_TIME_NONE; /* Initialize GStreamer */ gst_init (&argc, &argv); /* Create the elements */ data.playbin = gst_element_factory_make ("playbin", "playbin"); if (!data.playbin) { g_printerr ("Not all elements could be created.\n"); return -1; } /* Set the URI to play */ g_object_set (data.playbin, "uri", "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm", NULL); /* Start playing */ ret = gst_element_set_state (data.playbin, GST_STATE_PLAYING); if (ret == GST_STATE_CHANGE_FAILURE) { g_printerr ("Unable to set the pipeline to the playing state.\n"); gst_object_unref (data.playbin); return -1; } /* Listen to the bus */ bus = gst_element_get_bus (data.playbin); do { msg = gst_bus_timed_pop_filtered (bus, 100 * GST_MSECOND, GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS | GST_MESSAGE_DURATION_CHANGED); /* Parse message */ if (msg != NULL) { handle_message (&data, msg); } else { /* We got no message, this means the timeout expired */ if (data.playing) { gint64 current = -1; /* Query the current position of the stream */ if (!gst_element_query_position (data.playbin, GST_FORMAT_TIME, ¤t)) { g_printerr ("Could not query current position.\n"); } /* If we didn't know it yet, query the stream duration */ if (!GST_CLOCK_TIME_IS_VALID (data.duration)) { if (!gst_element_query_duration (data.playbin, GST_FORMAT_TIME, &data.duration)) { g_printerr ("Could not query current duration.\n"); } } /* Print current position and total duration */ g_print ("Position %" GST_TIME_FORMAT " / %" GST_TIME_FORMAT "\r", GST_TIME_ARGS (current), GST_TIME_ARGS (data.duration)); /* If seeking is enabled, we have not done it yet, and the time is right, seek */ if (data.seek_enabled && !data.seek_done && current > 10 * GST_SECOND) { g_print ("\nReached 10s, performing seek...\n"); gst_element_seek_simple (data.playbin, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, 30 * GST_SECOND); data.seek_done = TRUE; } } } } while (!data.terminate); /* Free resources */ gst_object_unref (bus); gst_element_set_state (data.playbin, GST_STATE_NULL); gst_object_unref (data.playbin); return 0; } static void handle_message (CustomData *data, GstMessage *msg) { GError *err; gchar *debug_info; switch (GST_MESSAGE_TYPE (msg)) { case GST_MESSAGE_ERROR: gst_message_parse_error (msg, &err, &debug_info); g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message); g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none"); g_clear_error (&err); g_free (debug_info); data->terminate = TRUE; break; case GST_MESSAGE_EOS: g_print ("End-Of-Stream reached.\n"); data->terminate = TRUE; break; case GST_MESSAGE_DURATION_CHANGED: /* The duration has changed, mark the current one as invalid */ data->duration = GST_CLOCK_TIME_NONE; break; case GST_MESSAGE_STATE_CHANGED: { GstState old_state, new_state, pending_state; gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state); if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->playbin)) { g_print ("Pipeline state changed from %s to %s:\n", gst_element_state_get_name (old_state), gst_element_state_get_name (new_state)); /* Remember whether we are in the PLAYING state or not */ data->playing = (new_state == GST_STATE_PLAYING); if (data->playing) { /* We just moved to PLAYING. Check if seeking is possible */ GstQuery *query; gint64 start, end; query = gst_query_new_seeking (GST_FORMAT_TIME); if (gst_element_query (data->playbin, query)) { gst_query_parse_seeking (query, NULL, &data->seek_enabled, &start, &end); if (data->seek_enabled) { g_print ("Seeking is ENABLED from %" GST_TIME_FORMAT " to %" GST_TIME_FORMAT "\n", GST_TIME_ARGS (start), GST_TIME_ARGS (end)); } else { g_print ("Seeking is DISABLED for this stream.\n"); } } else { g_printerr ("Seeking query failed."); } gst_query_unref (query); } } } break; default: /* We should not reach here */ g_printerr ("Unexpected message received.\n"); break; } gst_message_unref (msg); }
将源码保存为basic-tutorial-5.c,执行下列命令可得到编译结果:
gcc basic-tutorial-5.c -o basic-tutorial-5 `pkg-config --cflags --libs gstreamer-1.0`
源码分析
示例前部分内容与其他示例类似,构造Pipeline并使其进入PLAYING状态。之后开始监听Bus上的消息。
msg = gst_bus_timed_pop_filtered (bus, 100 * GST_MSECOND, GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS | GST_MESSAGE_DURATION_CHANGED);
与以前的示例相比,我们在gst_bus_timed_pop_filtered ()中加入了超时时间(100毫秒),这使得此函数如果在100毫秒内没有收到任何消息就会返回超时(msg == NULL),我们会在超时中去更新当前时间,如果返回相应消息(msg != NULL),我们在handle_message中处理相应消息。
GStreamer内部有统一的时间类型(GstClockTime),时间计算方式为:GstClockTime = 数值 x 时间单位。GStreamer提供了3种时间单位(宏定义):GST_SECOND(秒),GST_MSECOND(毫秒),GST_NSECOND(纳秒)。例如:
10秒: 10 * GST_SECOND
100毫秒:100 * GST_MSECOND
100纳秒:100 * GST_NSECOND
刷新播放时间
/* We got no message, this means the timeout expired */ if (data.playing) { /* Query the current position of the stream */ if (!gst_element_query_position (data.pipeline, GST_FORMAT_TIME, ¤t)) { g_printerr ("Could not query current position.\n"); } /* If we didn't know it yet, query the stream duration */ if (!GST_CLOCK_TIME_IS_VALID (data.duration)) { if (!gst_element_query_duration (data.pipeline, GST_FORMAT_TIME, &data.duration)) { g_printerr ("Could not query current duration.\n"); } }
我们首先判断Pipeline的状态,仅在PLAYING状态时才更新当前时间,在非PLAYING状态时查询可能失败。这部分逻辑每秒大概会执行10次,频率足够用于界面的刷新。这里我们只将查询到的时间输出到终端。
GstElement封装了相应的接口分别用于查询当前时间(gst_element_query_position)和总时间(gst_element_query_duration )。
/* Print current position and total duration */ g_print ("Position %" GST_TIME_FORMAT " / %" GST_TIME_FORMAT "\r", GST_TIME_ARGS (current), GST_TIME_ARGS (data.duration));
这里使用GST_TIME_FORMAT 和GST_TIME_ARGS 帮助我们方便地将GstClockTime的值转换为: ”时:分:秒“格式的字符串输出。
/* If seeking is enabled, we have not done it yet, and the time is right, seek */ if (data.seek_enabled && !data.seek_done && current > 10 * GST_SECOND) { g_print ("\nReached 10s, performing seek...\n"); gst_element_seek_simple (data.pipeline, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT, 30 * GST_SECOND); data.seek_done = TRUE; }
我们同时在超时处理中判断是否需要进行seek操作(在播放到10s时,自动跳转到30s),这里我们直接在Pipeline对象上使用gst_element_seek_simple()来执行跳转操作。
gst_element_seek_simple所需要的参数为:
- element : 需要执行seek操作的Element,这里是Pipeline。
- format:执行seek的类型,这里使用GST_FORMAT_TIME表示我们基于时间的方式进行跳转。其他支持的类型可以查看 GstFormat。
- seek_flags :通过标识指定seek的行为。 常用的标识如下,其他支持的flag详见GstSeekFlags。
-
- GST_SEEK_FLAG_FLUSH:在执行seek前,清除Pipeline中所有buffer中缓存的数据。这可能导致Pipeline在填充的新数据被显示之前出现短暂的等待,但能提高应用更快的响应速度。如果不指定这个标志,Pipeline中的所有缓存数据会依次输出,然后才会播放跳转的位置,会导致一定的延迟。
- GST_SEEK_FLAG_KEY_UNIT:对于大多数的视频,如果跳转的位置不是关键帧,需要依次解码该帧所依赖的帧(I帧及P帧)后,才能解码此非关键帧。使用这个标识后,seek会自动从最近的I帧开始播放。这个标识降低了seek的精度,提高了seek的效率。
- GST_SEEK_FLAG_ACCURATE:一些媒体文件没有提供足够的索引信息,在这种文件中执行seek操作会非常耗时,针对这类文件,GStreamer通过内部计算得到需要跳转的位置,大部分的计算结果都是正确的。如果seek的位置不能达到所需精度时,可以增加此标识。但需要注意的是,使用此标识可能会导致seek耗费更多时间来寻找精确的位置。
- seek_pos :需要跳转的位置,前面指定了seek的类型为时间,所以这里是30秒。
消息处理
我们在handle_message接口中处理所有Pipeline上的消息,ERROR和EOS与以前示例处理方式相同,此例中新增了以下内容:
case GST_MESSAGE_DURATION_CHANGED: /* The duration has changed, mark the current one as invalid */ data->duration = GST_CLOCK_TIME_NONE; break;
在文件的总时间发生变化时,我们会收到此消息,这里简单的将总长度标记为非法值,在下次更新时间时进行查询。
case GST_MESSAGE_STATE_CHANGED: { GstState old_state, new_state, pending_state; gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state); if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->pipeline)) { g_print ("Pipeline state changed from %s to %s:\n", gst_element_state_get_name (old_state), gst_element_state_get_name (new_state)); /* Remember whether we are in the PLAYING state or not */ data->playing = (new_state == GST_STATE_PLAYING);
跳转和时间查询操作仅在PUASED和PLAYING状态时才能得到正确的结果,因为所有的Element只能在这2个状态才能接收处理seek和query的指令。这里会保存播放的状态便于后续使用,并且在进入PLAYING状态时查询当前所播放的文件/流是否支持跳转操作:
if (data->playing) { /* We just moved to PLAYING. Check if seeking is possible */ GstQuery *query; gint64 start, end; query = gst_query_new_seeking (GST_FORMAT_TIME); if (gst_element_query (data->pipeline, query)) { gst_query_parse_seeking (query, NULL, &data->seek_enabled, &start, &end); if (data->seek_enabled) { g_print ("Seeking is ENABLED from %" GST_TIME_FORMAT " to %" GST_TIME_FORMAT "\n", GST_TIME_ARGS (start), GST_TIME_ARGS (end)); } else { g_print ("Seeking is DISABLED for this stream.\n"); } } else { g_printerr ("Seeking query failed."); } gst_query_unref (query); }
这里的查询步骤与文章开始介绍的方式相同:
- 首先,通过gst_query_new_seeking()构造一个跳转的查询对象,使用GST_FORMAT_TIME作为参数,表明我们需要知道当前的文件是否支持通过时间进行跳转。我们同样可以使用GST_FORMAT_BYTES作为参数,用于查询是否可以根据文件的偏移量来就行跳转,但这种使用方式不常见。
- 接着,将查询对象传入gst_element_query()查询,并得到结果。
- 最后,通过gst_query_parse_seeking()解析是否支持跳转及所支持的范围。
一定需要记住在使用完后释放查询对象。
总结
在本教程中,我们学习了:
- 如何通过GstQuery查询Pipeline上的信息。
- 如何通过gst_element_query_position()和gst_element_query_duration() 查询当前时间和总时间。
- 如何通过gst_element_seek_simple()操作跳转到任意位置。
- 在哪些状态可以执行查询和跳转操作。
后续我们将介绍如何获取媒体文件中的元数据(Metadata)。
引用
https://gstreamer.freedesktop.org/documentation/tutorials/basic/time-management.html?gi-language=c
https://gstreamer.freedesktop.org/documentation/additional/design/query.html?gi-language=c
https://gstreamer.freedesktop.org/documentation/gstreamer/gstquery.html?gi-language=c