[Gstreamer] gstbasesink 的 QOS 机制

前言：

gstreamer里很多element都提供 QOS 机制，src，filter 和 sink 都有。Sink element 的 QOS 机制由 gstbasesink 统一提供。

qos  (quality of service)  是一种评价机制，这个领域中都有这一概念，比如网络的qos。gstbasesink里的 qos 用来统计传入的 gstbuffer 的数据信息，然后发送 qos event 给上游 element ，从而让上游element 控制对 gstbasesink 的gstbuffer 输入速度，比如送的太快了，则发送一个 qos event 要求上游element 推送 gstbuffer 慢一点，如果推送 gstbuffer 慢了，则发送 qos event 要求上游推送快一点。

可以把 gstbasesink 里 的 qos 工作分为两块：

1. 时间分析；
2. qos event上报。

当qos_enable设置true后，qos event上报功能被打开,默认情况下qos是关闭状态。

#define DEFAULT_QOS                 FALSE



  g_object_class_install_property (gobject_class, PROP_QOS,

      g_param_spec_boolean ("qos", "Qos",

          "Generate Quality-of-Service events upstream", DEFAULT_QOS,

          G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));

时间分析是在确定使用 qos 功能之后才会进行，其实所谓的统计分析仅仅是使用当前GstBuffer的事件戳数据进行计算。 因此从某种层面上说，qos计算使用的数据不会累计，qos不会记录某个区间内的数据然后进行综合计算，只会结合当前GstBuffer 的时间信息和参考时钟信息。

---

---

---

三种 qos event 类型

qos event 的 type 有三种，overflow，underflow，throttle 。

/**
 * GstQOSType:
 * @GST_QOS_TYPE_OVERFLOW: The QoS event type that is produced when upstream
 *    elements are producing data too quickly and the element can't keep up
 *    processing the data. Upstream should reduce their production rate. This
 *    type is also used when buffers arrive early or in time.
 * @GST_QOS_TYPE_UNDERFLOW: The QoS event type that is produced when upstream
 *    elements are producing data too slowly and need to speed up their
 *    production rate.
 * @GST_QOS_TYPE_THROTTLE: The QoS event type that is produced when the
 *    application enabled throttling to limit the data rate.
 *
 * The different types of QoS events that can be given to the
 * gst_event_new_qos() method.
 */
typedef enum {
  GST_QOS_TYPE_OVERFLOW        = 0,
  GST_QOS_TYPE_UNDERFLOW       = 1,
  GST_QOS_TYPE_THROTTLE        = 2
} GstQOSType;

overflow：上游element对自己push gstbuffer过快，当前element可以通过push overflow类型的qos event 给上游，以控制 gstbuffer 流量。

underflow：上游element对自己push gstbuffer过慢，当前element可以通过push underflow类型的qos event 给上游，以要求更多的 gstbuffer 输入。

throttle：一个水位值到达的通知类型，这个类型伴随 “throttle-time” 一起使用。

throttle-time 是sink element用来进行 sync的一个机制，可以用来控制每秒钟传入 render 函数的gstbuffer数量(或者说每秒调用render 函数的次数)。当此属性值大于0，则 throttle 类型的 qos event 会被上传给上游 element ，这个时候即便达到underflow和overflow的标准，也只会上传throttle事件。由于gstreamer element的属性是可以动态设置的，所以可以在运行时把这个属性重置为0来激活 overflow / underflow 类型的 qos 事件。

  /**
   * GstBaseSink:throttle-time:
   *
   * The time to insert between buffers. This property can be used to control
   * the maximum amount of buffers per second to render. Setting this property
   * to a value bigger than 0 will make the sink create THROTTLE QoS events.
   */
  g_object_class_install_property (gobject_class, PROP_THROTTLE_TIME,
      g_param_spec_uint64 ("throttle-time", "Throttle time",
          "The time to keep between rendered buffers (0 = disabled)", 0,
          G_MAXUINT64, DEFAULT_THROTTLE_TIME,
          G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));

---

---

---

qos event上报：

---

调用链

qos event 上报是通过 gst_base_sink_send_qos 完成的。

static gboolean
gst_base_sink_send_qos (GstBaseSink * basesink, GstQOSType type,
    gdouble proportion, GstClockTime time, GstClockTimeDiff diff)

这个函数只有一个调用点，在 gst_base_sink_perform_qos 里。

static void
gst_base_sink_perform_qos (GstBaseSink * sink, gboolean dropped)

gst_base_sink_perform_qos  只会在 gst_base_sink_chain_unlocked 里被调用 

/* with STREAM_LOCK, PREROLL_LOCK
 *
 * Takes a buffer and compare the timestamps with the last segment.
 * If the buffer falls outside of the segment boundaries, drop it.
 * Else send the buffer for preroll and rendering.
 *
 * This function takes ownership of the buffer.
 */
static GstFlowReturn
gst_base_sink_chain_unlocked (GstBaseSink * basesink, GstPad * pad,
    gpointer obj, gboolean is_list)

gst_base_sink_chain_unlocked 会被两个函数调用，分别是 gst_base_sink_chain_main 和 gst_base_sink_loop ，对应这 sink element 的 push mode 和 pull mode。因此只需要关心其中一个即可，一般关心push mode 的gst_base_sink_chain_main。

至此，调用链已经明确。

gst_base_sink_chain_main ---> gst_base_sink_chain_unlocked ---> gst_base_sink_perform_qos  ---> gst_base_sink_send_qos

---

何时上报

在清楚了调用链以后，下面就是根据调用链分析在什么业务逻辑下会触发 qos 事件上报。

gst_base_sink_chain_unlocked

done:
  if (step_end) {
    /* the step ended, check if we need to activate a new step */
    GST_DEBUG_OBJECT (basesink, "step ended");
    stop_stepping (basesink, &basesink->segment, &priv->current_step,
        priv->current_rstart, priv->current_rstop, basesink->eos);
    goto again;
  }

  gst_base_sink_perform_qos (basesink, late);

  GST_DEBUG_OBJECT (basesink, "object unref after render %p", obj);
  gst_mini_object_unref (GST_MINI_OBJECT_CAST (obj));

  return ret;

在gst_base_sink_chain_unlocked中，没有goto done，也就是说每一个 GstBuffer被正常处理后都会进入 done 以及 gst_base_sink_perform_qos ，同时还携带当前 GstBuffer 是否已经 out of segment 且可能被丢弃的 flag -> "late" 。

gst_base_sink_perform_qos

static void
gst_base_sink_perform_qos (GstBaseSink * sink, gboolean dropped)
{

  //这个时间戳表示什么 ？
  start = priv->current_rstart;

  //如果qos功能没打开，则不直接跳过qos流程
  /* if Quality-of-Service disabled, do nothing */
  if (!g_atomic_int_get (&priv->qos_enabled) ||
      !GST_CLOCK_TIME_IS_VALID (start))
    return;

  //这个时间戳表示什么 ？
  stop = priv->current_rstop;

  //当前GstBuffer 的jitter 值
  jitter = priv->current_jitter;

  //计算 GstBuffer 进入 sink 的时间戳 entered
  if (jitter < 0) {
    /* this is the time the buffer entered the sink */
    if (start < -jitter)
      entered = 0;
    else
      entered = start + jitter;
    left = start;
  } else {
    /* this is the time the buffer entered the sink */
    entered = start + jitter;
    /* this is the time the buffer left the sink */
    left = start + jitter;
  }

  //avg processing time 是什么意思？avg_rate是什么意思？
  GST_CAT_DEBUG_OBJECT (GST_CAT_QOS, sink,
      "updated: avg_pt: %" GST_TIME_FORMAT
      ", avg_rate: %g", GST_TIME_ARGS (priv->avg_pt), priv->avg_rate);

  //如果 avg_rate >= 0.0 ，则需要触发 qos event 事件
  if (priv->avg_rate >= 0.0) {
    GstQOSType type;
    GstClockTimeDiff diff;

    /* if we have a valid rate, start sending QoS messages */
    if (priv->current_jitter < 0) {
      /* make sure we never go below 0 when adding the jitter to the
       * timestamp. */
      if (priv->current_rstart < -priv->current_jitter)
        priv->current_jitter = -priv->current_rstart;
    }

    // 1. 如果设置了 throttle 功能，那么发送 THROTTLE 类型的 qos
    if (priv->throttle_time > 0) {
      diff = priv->throttle_time;
      type = GST_QOS_TYPE_THROTTLE;
    } else {
      diff = priv->current_jitter;
      if (diff <= 0)
        type = GST_QOS_TYPE_OVERFLOW;    //如果diff <= 0 ,发送 overflow
      else
        type = GST_QOS_TYPE_UNDERFLOW;
    }

    gst_base_sink_send_qos (sink, type, priv->avg_rate, priv->current_rstart,
        diff);
  }

上面的代码中涉及到一些数值计算，其中牵扯到同步机制，比较复杂，我们只需要关心一行日志， 

GST_CAT_DEBUG_OBJECT (GST_CAT_QOS, sink, "updated: avg_pt: %" GST_TIME_FORMAT", avg_rate: %g", GST_TIME_ARGS (priv->avg_pt), priv->avg_rate);

当发现 avg_rate >= 0 的时候，就说明有 qos 事件会被上传给上游。

gst_base_sink_send_qos

gst_base_sink_send_qos中没有提前退出的逻辑，因此一旦进入则必定会发送给上游。