GStreamer Windows 基辅3：全手动，声像作例，汝作大像也。导演：卡卡，你说老毛子东西啊？应该是 G 公创 Pad 极，1 曰 Video，二曰 Audio

最新推荐文章于 2022-01-10 20:49:02 发布

无厘头编程

最新推荐文章于 2022-01-10 20:49:02 发布

阅读量288

点赞数

分类专栏： C++ 文章标签： visual studio msvc gstreamer c++ 媒体控制接口

本文链接：https://blog.csdn.net/fly_bear_unknown/article/details/114066620

版权

C++ 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

👧：上面那个汝念什么啊？
🙄：此字念如，三声，就是您的意思。
…

☯️1. 开始

👨‍💼：从访问数据看来，泡 C 的人数还挺多的。
如果经过我 “基辅 2” 的锤炼⚒️，跑 “基辅 3” 绝对没有问题。因为 Property Manager 已经到位，每一个项目都是用统一的设定。
run

Pipeline state changed from NULL to READY:
Received new pad 'src_0' from 'source':
It has type 'video/x-raw' which is not raw audio. Ignoring.
Received new pad 'src_1' from 'source':
Link succeeded (type 'audio/x-raw').
Pipeline state changed from READY to PAUSED:
Pipeline state changed from PAUSED to PLAYING:

背景音乐🎶：Sintel
👦：你看得懂上面在干什么吗？
👧：不晓得咧。音乐不错，没有画面，是不是失灵了？

🧪2. 分析

甭管是什么，能跑起来就是好事，接着看🐴：
…

【🔹】公用参数

CustomData 的结构

/* Structure to contain all our information, so we can pass it to callbacks */
typedef struct _CustomData
{
  GstElement *pipeline;
  GstElement *source;
  GstElement *convert;
  GstElement *resample;
  GstElement *sink;
} CustomData;

pipeline：媒体管道
source：源头
convert：翻成电子讯号
resample：重组
sink：输出

全部都是 pointer，就是针👉啦！

/* Handler for the pad-added signal */
static void pad_added_handler (GstElement * src, GstPad * pad,
    CustomData * data);

方程的头 Header： Header 就是在记忆体内占位置，Java 就没有这个问题，但 C/C++ 很注重位置，上面看不见❌👁‍🗨下面的那位。
…

【🔹】main

  CustomData data;
  GstBus *bus;
  GstMessage *msg;
  GstStateChangeReturn ret;
  gboolean terminate = FALSE;

📀 data：媒体资料
🚛 bus：快递车
📧 msg：邮件
▶️ ret：播放状态
❌ terminate：关闭没有？

  /* Initialize GStreamer */
  gst_init (&argc, &argv);

启动

  /* Create the elements */
  data.source = gst_element_factory_make ("uridecodebin", "source");
  data.convert = gst_element_factory_make ("audioconvert", "convert");
  data.resample = gst_element_factory_make ("audioresample", "resample");
  data.sink = gst_element_factory_make ("autoaudiosink", "sink");

uridecodebin 是个好媳妇，一次生了仨：👶🏻sources, 👶🏻demuxer 和 👶🏻decoders，多省时间，第二次申请都免了，🥳party 要礼物一次可以要三份，绝对不是赔钱货啊。
制作元素：其中 convert 和 resample 都是用于播放声音的。

  /* Create the empty pipeline */
  data.pipeline = gst_pipeline_new ("test-pipeline");

  if (!data.pipeline || !data.source || !data.convert || !data.resample
      || !data.sink) {
    g_printerr ("Not all elements could be created.\n");
    return -1;
  }
  
  /* Build the pipeline. Note that we are NOT linking the source at this
   * point. We will do it later. */
  gst_bin_add_many (GST_BIN (data.pipeline), data.source, data.convert,
      data.resample, data.sink, NULL);
  if (!gst_element_link_many (data.convert, data.resample, data.sink, NULL)) {
    g_printerr ("Elements could not be linked.\n");
    gst_object_unref (data.pipeline);
    return -1;
  }

媒体管道制作，这个你都会了。

  /* Set the URI to play */
  g_object_set (data.source, "uri",
      "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm",
      NULL);

媒体资料来源

  /* Connect to the pad-added signal */
  g_signal_connect (data.source, "pad-added", G_CALLBACK (pad_added_handler),
      &data);

连接 pad 的信号。

  /* Start playing */
  ret = gst_element_set_state (data.pipeline, GST_STATE_PLAYING);
  if (ret == GST_STATE_CHANGE_FAILURE) {
    g_printerr ("Unable to set the pipeline to the playing state.\n");
    gst_object_unref (data.pipeline);
    return -1;
  }

播放

  /* Listen to the bus */
  bus = gst_element_get_bus (data.pipeline);
  do {
    msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE,
        GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS);

    /* Parse message */
    if (msg != NULL) {
      GError *err;
      gchar *debug_info;

      switch (GST_MESSAGE_TYPE (msg)) {
        case GST_MESSAGE_ERROR:
          gst_message_parse_error (msg, &err, &debug_info);
          g_printerr ("Error received from element %s: %s\n",
              GST_OBJECT_NAME (msg->src), err->message);
          g_printerr ("Debugging information: %s\n",
              debug_info ? debug_info : "none");
          g_clear_error (&err);
          g_free (debug_info);
          terminate = TRUE;
          break;
        case GST_MESSAGE_EOS:
          g_print ("End-Of-Stream reached.\n");
          terminate = TRUE;
          break;
        case GST_MESSAGE_STATE_CHANGED:
          /* We are only interested in state-changed messages from the pipeline */
          if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data.pipeline)) {
            GstState old_state, new_state, pending_state;
            gst_message_parse_state_changed (msg, &old_state, &new_state,
                &pending_state);
            g_print ("Pipeline state changed from %s to %s:\n",
                gst_element_state_get_name (old_state),
                gst_element_state_get_name (new_state));
          }
          break;
        default:
          /* We should not reach here */
          g_printerr ("Unexpected message received.\n");
          break;
      }
      gst_message_unref (msg);
    }
  } while (!terminate);

上快递，处理邮件：

1️⃣，GST_MESSAGE_STATE_CHANGED
2️⃣，GST_MESSAGE_ERROR
3️⃣，GST_MESSAGE_EOS

下车由 terminate 控制。

  /* Free resources */
  gst_object_unref (bus);
  gst_element_set_state (data.pipeline, GST_STATE_NULL);
  gst_object_unref (data.pipeline);

♻️ 回收 bus 和 pipeline 。
…

【🔸】pad_added_handler

这是新东西 pad 。
制造：

g_signal_connect (
data.source, 
"pad-added", G_CALLBACK (pad_added_handler), 
&data);

进口：

pad_added_handler (GstElement * src, GstPad * new_pad, CustomData * data)

src：源头
new_pad：pad 名是 ‘src_1’ （自动命名）
data：资料

参数：

  GstPad *sink_pad = gst_element_get_static_pad (data->convert, "sink");
  GstPadLinkReturn ret;
  GstCaps *new_pad_caps = NULL;
  GstStructure *new_pad_struct = NULL;
  const gchar *new_pad_type = NULL;

是否连接状态：

  /* If our converter is already linked, we have nothing to do here */
  if (gst_pad_is_linked (sink_pad)) {
    g_print ("We are already linked. Ignoring.\n");
    goto exit;
  }

如果别人帮你把活给干了，去 exit 。

分析类别：

  /* Check the new pad's type */
  new_pad_caps = gst_pad_get_current_caps (new_pad);
  new_pad_struct = gst_caps_get_structure (new_pad_caps, 0);
  new_pad_type = gst_structure_get_name (new_pad_struct);
  if (!g_str_has_prefix (new_pad_type, "audio/x-raw")) {
    g_print ("It has type '%s' which is not raw audio. Ignoring.\n",
        new_pad_type);
    goto exit;
  }

不认得，去 exit 。

连接：

  /* Attempt the link */
  ret = gst_pad_link (new_pad, sink_pad);
  if (GST_PAD_LINK_FAILED (ret)) {
    g_print ("Type is '%s' but link failed.\n", new_pad_type);
  } else {
    g_print ("Link succeeded (type '%s').\n", new_pad_type);
  }

出口：

exit:
  /* Unreference the new pad's caps, if we got them */
  if (new_pad_caps != NULL)
    gst_caps_unref (new_pad_caps);

  /* Unreference the sink pad */
  gst_object_unref (sink_pad);

开头的文字大部分都是这出来的。
实在是看不清作者的意图，加 g_print 自己看。

👓 加料

static void pad_added_handler(GstElement* src, GstPad* new_pad, CustomData* data) {
  GstPad* sink_pad = gst_element_get_static_pad(data->convert, "sink");
  GstPadLinkReturn ret;
  GstCaps* new_pad_caps = NULL;
  GstStructure* new_pad_struct = NULL;
  const gchar* new_pad_type = NULL;

  g_print("\n ==========> Begin to link\n");
  g_print("Received new pad '%s' from '%s':\n", GST_PAD_NAME(new_pad), GST_ELEMENT_NAME(src));

  /* If our converter is already linked, we have nothing to do here */
  if (gst_pad_is_linked(sink_pad)) {
    g_print("We are already linked. Ignoring.\n");
    goto exit;
  }
  else {
    g_print("\n ==========> Not linked\n");
  }

  /* Check the new pad's type */
  new_pad_caps = gst_pad_get_current_caps(new_pad);
  new_pad_struct = gst_caps_get_structure(new_pad_caps, 0);
  new_pad_type = gst_structure_get_name(new_pad_struct);
  if (!g_str_has_prefix(new_pad_type, "audio/x-raw")) {
    g_print("It has type '%s' which is not raw audio. Ignoring.\n", new_pad_type);
    goto exit;
  }

  /* Attempt the link */
  ret = gst_pad_link(new_pad, sink_pad);
  if (GST_PAD_LINK_FAILED(ret)) {
    g_print("Type is '%s' but link failed.\n", new_pad_type);
  }
  else {
    g_print("Link audio succeeded (type '%s').\n", new_pad_type);
  }

exit:
  /* Unreference the new pad's caps, if we got them */
  if (new_pad_caps != NULL)
    gst_caps_unref(new_pad_caps);

  /* Unreference the sink pad */
  gst_object_unref(sink_pad);

  g_print("\n ==========> End of link\n\n");

}

🚴跑啦：

Pipeline state changed from NULL to READY:

 ==========> Begin to link
Received new pad 'src_0' from 'source':

 ==========> Not linked
It has type 'video/x-raw' which is not raw audio. Ignoring.

 ==========> End of link


 ==========> Begin to link
Received new pad 'src_1' from 'source':

 ==========> Not linked
Link audio succeeded (type 'audio/x-raw').

 ==========> End of link

Pipeline state changed from READY to PAUSED:
Pipeline state changed from PAUSED to PLAYING:

👦：懂啦！callback 绕了两圈。
👧：是，开头是测 video 录像种类，接着测 audio 音像种类。

📚3. 写作业

Dynamic pad linking has traditionally been a difficult topic for a lot of programmers. 
Prove that you have achieved its mastery by instantiating an autovideosink (probably with an videoconvert in front) 
and link it to the demuxer when the right pad appears. 
Hint: You are already printing on screen the type of the video pads.

You should now see (and hear) the same movie as in Basic tutorial 1: Hello world!. 
In that tutorial you used playbin, which is a handy element that automatically takes care of all the demuxing and pad linking for you. 
Most of the Playback tutorials are devoted to playbin.

🗿：说什么？难的…证明…一大堆——就是加影像啦！
homework
瞧，这个就是作业。

…

💎 加元素

已经有的是：source——uridecodebin(这里已经包括source， demuxer 和 decoder），sink——autoaudiosink，convert——audioconvert，resample——audioresample。
照图上说，加 videoConvert 与 videoSink 。

多了两个，因此我进行重组。

typedef struct _CustomData
{
  GstElement* pipeline;
  GstElement* source;
  // audio
  GstElement* audioConvert;
  GstElement* audioResample;
  GstElement* audioSink;
  // video
  GstElement* videoConvert;
  GstElement* videoSink;
} CustomData;

…

🏭 工厂制造

…

💎 原材料加工

  /* Create the elements */
  data.source = gst_element_factory_make("uridecodebin", "source");

  // audio
  data.audioConvert = gst_element_factory_make("audioconvert", "convert");
  data.audioResample = gst_element_factory_make("audioresample", "resample");
  data.audioSink = gst_element_factory_make("autoaudiosink", "sink");

  // video
  data.videoConvert = gst_element_factory_make("videoconvert", "videoconvert");
  data.videoSink = gst_element_factory_make("autovideosink", "video-sink");

…

🕵 元素检查

  data.pipeline = gst_pipeline_new("test-pipeline");

  if (!data.pipeline || !data.source || !data.audioConvert || !data.audioResample
    || !data.audioSink || !data.videoConvert || !data.videoSink) {
    g_printerr("Not all elements could be created.\n");
    return -1;
  }

…

🥫 罐装加工

🎵 音像

  gst_bin_add_many(GST_BIN(data.pipeline), data.source, data.audioConvert, data.audioResample, data.audioSink, NULL);
  if (!gst_element_link_many(data.audioConvert, data.audioResample, data.audioSink, NULL)) {
    g_printerr("Audio elements could not be linked.\n");
    gst_object_unref(data.pipeline);
    return -1;
  }

照抄，改名字。

👁‍🗨 影像

  gst_bin_add_many(GST_BIN(data.pipeline), data.source, data.videoConvert, data.videoSink, NULL);
  if (!gst_element_link_many(data.videoConvert, data.videoSink, NULL)) {
    g_printerr("Video elements could not be linked.\n");
    gst_object_unref(data.pipeline);
    return -1;
  }

…

🍜 插 Pad 到 demuxer

pad_added_handler 就是负责这个任务。因为是自动绕圈，不用特别写 video，你带一带路就行。第一圈是查 video；第二圈是查 audio 。

➕ 参数

static void pad_added_handler(GstElement* src, GstPad* new_pad, CustomData* data) {
  GstPad* audioPad = NULL;
  GstPad* videoPad = NULL;
  gboolean audioType = FALSE;

  GstPadLinkReturn ret;
  GstCaps* new_pad_caps = NULL;
  GstStructure* new_pad_struct = NULL;
  const gchar* new_pad_type = NULL;

  g_print("\n ==========> Begin link\n");
  g_print("\nReceived new pad '%s' from '%s':\n", GST_PAD_NAME(new_pad), GST_ELEMENT_NAME(src));

  /* Check the new pad's type */
  new_pad_caps = gst_pad_get_current_caps(new_pad);
  new_pad_struct = gst_caps_get_structure(new_pad_caps, 0);
  new_pad_type = gst_structure_get_name(new_pad_struct);

  g_print("\n\n--------------> Testing audio/video type --------------> \n\n", new_pad_type);

分组：audioPad, videoPad
分类：audioType （区分 audio 和 video）
你瞧大头都是照旧，你可以不看 g_print 的东西。
…

🔗分组连接

  if (g_str_has_prefix(new_pad_type, "audio/x-raw")) {
    g_print("Found Raw Audio Type: '%s'\n", new_pad_type);
    audioType = TRUE;
    goto link;
  }
  else if (g_str_has_prefix(new_pad_type, "video/x-raw")) {
    g_print("Found Raw Video Type: '%s'\n", new_pad_type);
    audioType = FALSE;
    goto link;
  }
  else {
    g_print("It has type '%s' which is not raw audio/video type. Ignoring.\n", new_pad_type);
    goto exit;
  }

link:
  /* Attempt the link */
  if (audioType) {
    audioPad = gst_element_get_static_pad(data->audioConvert, "sink");
    if (gst_pad_is_linked(audioPad)) {
      g_print("We are already linked. Ignoring.\n");
      goto exit;
    }
    // link audio
    ret = gst_pad_link(new_pad, audioPad);
  }
  else {
    videoPad = gst_element_get_static_pad(data->videoConvert, "sink");
    if (gst_pad_is_linked(videoPad)) {
      g_print("We are already linked. Ignoring.\n");
      goto exit;
    }
    // link video
    ret = gst_pad_link(new_pad, videoPad);
  }
  
  if (GST_PAD_LINK_FAILED(ret)) {
    g_print("Type is '%s' but link failed.\n", new_pad_type);
  }
  else {
    g_print("Link succeeded (type '%s').\n", new_pad_type);
  }

  if (audioType) {
    gst_object_unref(audioPad);
  }
  else {
    gst_object_unref(videoPad);
  }

…

🚪 出口

exit:
  /* Unreference the new pad's caps, if we got them */
  if (new_pad_caps != NULL)
    gst_caps_unref(new_pad_caps);

  /* Unreference the sink pad */
  g_print("\n ==========> End link\n\n");

照旧。
…

🏇 跑马啦

(basic-tutorial-3.exe:36416): GStreamer-WARNING **: 21:06:09.233: Name 'source' is not unique in bin 'test-pipeline', not adding
Pipeline state changed from NULL to READY:

 ==========> Begin link

Received new pad 'src_0' from 'source':


--------------> Testing audio/video type -------------->

Found Raw Video Type: 'video/x-raw'
Link succeeded (type 'video/x-raw').

 ==========> End link


 ==========> Begin link

Received new pad 'src_1' from 'source':


--------------> Testing audio/video type -------------->

Found Raw Audio Type: 'audio/x-raw'
Link succeeded (type 'audio/x-raw').

 ==========> End link

Pipeline state changed from READY to PAUSED:
Pipeline state changed from PAUSED to PLAYING:

多了个 warning，傻逼才去管，什么 source 不是独一号？
Yeah！跑起来了。
😅：码看起来有点啰嗦，你们自己改改吧。

🎁 4. 总结

😒：本人有点阅读障碍，因为老是跳行看👀，没见着 uridecodebin* 是孕妇🤰的这马子事。都把 demuxer， decoder 都写好，就是 source 连不上。就回头再读，😵硬着头皮地不跳行读，这个最难啊。
🤹‍♂️：卡卡，这是总结，不是吐槽大会。笑一个。
😆：知识就是力量！Pad 就是把音像、影像连接到导播的杀手。呸呸，台词错了，助手👼🏻。

啪啪啪！导演追着小生：让你念错台词！…