rtsp 协议流程

rtsp 协议流程 学习

RTP Real-time Transport Protocol 实时传输协议
RTSP Real Time Streaming Protocol 实时流协议
RTSP 和 RTP 开发客户 一般是C/S 模式

整个流媒体传输的 协议流程

整个过程只关注 协议流程 (不关注细节，细节可以在遇到bug时处理)

首先是RTSP 流程

1. 客户端创建socket 添加 服务器端的ip地址和端口信息 连接到服务器端

Start to connect the server..

2. 客户端 在socket上发送 option

OPTIONS rtsp://184.72.239.149/vod/mp4://BigBuckBunny_175k.mov RTSP/1.0

然后会收到服务器端返回的消息 public支持的方法 ，CSeq 表示每次会话中标识的事物id

The response state is: 200
CSeq: 1…
Server: Wowza Streaming Engine 4.7.3.02 build21313…
Cache-Control: no-cache…
Public: DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE, OPTIONS, ANNOUNCE, RECORD, GET_PARAMETER…
Supported: play.basic, con.persistent…

3. 发送DESCRIBE 消息

DESCRIBE rtsp://184.72.239.149/vod/mp4://BigBuckBunny_175k.mov RTSP/1.0

接收到服务器端 返回的消息 包含sdp信息

The response state is: 200
The line is: CSeq: 2…
The line is: Server: Wowza Streaming Engine 4.7.3.02 build21313…
The SDP info: have audio info.. ; the audio track is trackID=1
The SDP info: have video info.. the video track is trackID=2; the video SPS is Z0LAHtkDxWhAAAADAEAAAAwDxYuS; the video PPS is aMuMsg==; the video packetization mode is 1

4. 发送setup 消息 确定传输机制，请求改变传输的参数 (开始等待数据 )

SETUP rtsp://184.72.239.149/vod/mp4://BigBuckBunny_175k.mov/trackID=2 RTSP/1.0

服务器返回信息 告诉客户端 RTP传输使用UDP 单播 客户端端口是55640-55641 还有服务器端的信息

The response state is: 200
CSeq: 3…
Transport: RTP/AVP/UDP;unicast;client_port=55640-55641;source=184.72.239.149;server_port=8260-8261;ssrc=24E1848D..

5. 然后客户端发送 PLAY 请求

PLAY rtsp://184.72.239.149/vod/mp4://BigBuckBunny_175k.mov RTSP/1.0

服务器然后返回信息 ，然后服务器就开始发送RTP包

The response state is: 200
RTP-Info: url=trackID=1;seq=1;rtptime=0,url=rtsp://184.72.239.149/vod/mp4://BigBuckBunny_175k.mov/trackID=2;seq=1;rtptime=0…
CSeq: 4…
The line is: Server: Wowza Streaming Engine 4.7.3.02 build21313…
Cache-Control: no-cache…
Range: npt=0.0-596.458…
Session: 1897418299;timeout=60…

6. 类似于心跳包 发送GET_PARAMETER

GET_PARAMETER rtsp://184.72.239.149/vod/mp4://BigBuckBunny_175k.mov/trackID=2 RTSP/1.0

如果是 从服务器获取参数，目前主要获取时间范围 如果 发送空信息 用来和服务器端保持联系 作为心跳包

如果能了解这些请求方法，后面的流程就不是很难

RTP 流程

在RTSP 发送setup 请求的时候，就已经创建RTP socket 连接，使用buffer 缓冲等待数据， 客户端发送PLAY 请求后，直接接受消息

GStreamer 中RTSP和RTP代码结构分析

本地初始化，构建RTSP 需要的环境

在 gst-plugins-good-1.0-1.5/gst/rtsp/gstrtspsrc.c 代表rtsp客户端
gst_rtspsrc_class_init gst_rtspsrc_init 用来本地初始化
gst_rtspsrc_finalize 本地释放rtsp使用内存空间

RTSP 本地流媒体状态

在 方法gst_rtspsrc_change_state 包含了视频流 从 null到ready，然后pause 最后play整个状态切换，通过command来控制

static GstStateChangeReturn gst_rtspsrc_change_state (GstElement * element, GstStateChange transition)
{

  rtspsrc = GST_RTSPSRC (element);

  switch (transition) {
    case GST_STATE_CHANGE_NULL_TO_READY:
      if (!gst_rtspsrc_start (rtspsrc))
        goto start_failed;
      break;
    case GST_STATE_CHANGE_READY_TO_PAUSED:
      gst_rtspsrc_loop_send_cmd (rtspsrc, CMD_OPEN, 0);
      break;
    case GST_STATE_CHANGE_PAUSED_TO_PLAYING:
      set_manager_buffer_mode (rtspsrc);
    default:
      break;
  }

  switch (transition) {
    case GST_STATE_CHANGE_NULL_TO_READY:
      ret = GST_STATE_CHANGE_SUCCESS;
      break;
    case GST_STATE_CHANGE_READY_TO_PAUSED:
      ret = GST_STATE_CHANGE_NO_PREROLL;
      break;
    case GST_STATE_CHANGE_PAUSED_TO_PLAYING:
      gst_rtspsrc_loop_send_cmd (rtspsrc, CMD_PLAY, 0);
      ret = GST_STATE_CHANGE_SUCCESS;
      break;

    default:
      break;
  }

done:
  return ret;

}

当rtsp客户端从null -> ready 那么启动rtsp，并且初始化 调用 gst_rtspsrc_start ，用来启动一个线程作为Task

static gboolean gst_rtspsrc_start (GstRTSPSrc * src)
{

  src->pending_cmd = CMD_WAIT;

  if (src->task == NULL) {
    src->task = gst_task_new ((GstTaskFunction) gst_rtspsrc_thread, src, NULL); //新建一个task

    gst_task_set_lock (src->task, GST_RTSP_STREAM_GET_LOCK (src));//锁住Task
  }

  return TRUE;
}

这个Task 主要的任务就是初始化loop轮询机制

/* the thread where everything happens */
static void gst_rtspsrc_thread (GstRTSPSrc * src)
{
  cmd = src->pending_cmd;
  if (cmd == CMD_RECONNECT || cmd == CMD_PLAY || cmd == CMD_PAUSE
      || cmd == CMD_LOOP || cmd == CMD_OPEN)
    src->pending_cmd = CMD_LOOP;
  else
    src->pending_cmd = CMD_WAIT;

  switch (cmd) {
    case CMD_CLOSE:
      gst_rtspsrc_close (src, TRUE, FALSE);
      break;
    case CMD_LOOP:
      gst_rtspsrc_loop (src);
      break;

    default:
      break;
  }
}

当 rtsp 客户端设置 从ready 变成 pause 通过 gst_rtspsrc_loop_start_cmd 方法将当前组件的状态同步到 BUS总线，便于通知其他组件实现同步 —–这一步是我看代码的分析

当 rtsp客户端 变成pause 做了两件事
1. 设置buffer 管理
2. 当前状态上报BUS总线

RTSP 协议流程 主要分散在下面这几个方法

1. gst_rtspsrc_open
2. gst_rtspsrc_play
3. gst_rtspsrc_pause
4. gst_rtspsrc_close

gst_rtspsrc_open

static GstRTSPResult gst_rtspsrc_open (GstRTSPSrc * src, gboolean async)
{
  src->methods = GST_RTSP_SETUP | GST_RTSP_PLAY | GST_RTSP_PAUSE | GST_RTSP_TEARDOWN;
  ret = gst_rtspsrc_retrieve_sdp (src, &src->sdp, async)//获取SDP信息
  ret = gst_rtspsrc_open_from_sdp (src, src->sdp, async)//打开SDP
}

获取SDP

static GstRTSPResult
gst_rtspsrc_retrieve_sdp (GstRTSPSrc * src, GstSDPMessage ** sdp,
    gboolean async)
{
  GstRTSPResult res;
  GstRTSPMessage request = { 0 };
  GstRTSPMessage response = { 0 };
  guint8 *data;
  guint size;
  gchar *respcont = NULL;

  if ((res = gst_rtsp_conninfo_connect (src, &src->conninfo, async)) < 0)
    goto connect_failed;

  /* create OPTIONS */
  GST_DEBUG_OBJECT (src, "create options...");
  res = gst_rtsp_message_init_request (&request, GST_RTSP_OPTIONS,src->conninfo.url_str);
  /* send OPTIONS */
  GST_DEBUG_OBJECT (src, "send options...");
  GST_ELEMENT_PROGRESS (src, CONTINUE, "open", ("Retrieving server options"));
  res =gst_rtspsrc_send (src, src->conninfo.connection, &request, &response,NULL)
  /* parse OPTIONS */
  gst_rtspsrc_parse_methods (src, &response)
  /* create DESCRIBE */
  GST_DEBUG_OBJECT (src, "create describe...");
  res = gst_rtsp_message_init_request (&request, GST_RTSP_DESCRIBE,src->conninfo.url_str);
  /* we only accept SDP for now */
  gst_rtsp_message_add_header (&request, GST_RTSP_HDR_ACCEPT,"application/sdp");
  /* send DESCRIBE */
  GST_DEBUG_OBJECT (src, "send describe...");
 GST_ELEMENT_PROGRESS (src, CONTINUE, "open", ("Retrieving media info"));
 res = gst_rtspsrc_send (src, src->conninfo.connection, &request, &response,NULL)
  /* check if reply is SDP */
  gst_rtsp_message_get_header (&response, GST_RTSP_HDR_CONTENT_TYPE, &respcont,0);
  /* get message body and parse as SDP */
  gst_rtsp_message_get_body (&response, &data, &size);
  GST_DEBUG_OBJECT (src, "parse SDP...");
  gst_sdp_message_new (sdp);
  gst_sdp_message_parse_buffer (data, size, *sdp);
  return res;
}

上面的代码流程已经开了rtsp的协议流程，在这之前首先判断创建连接

  if ((res = gst_rtsp_conninfo_connect (src, &src->conninfo, async)) < 0)
    goto connect_failed;

static GstRTSPResult  gst_rtsp_conninfo_connect (GstRTSPSrc * src, GstRTSPConnInfo * info,gboolean async){

  GST_DEBUG_OBJECT (src, "parsing uri (%s)...", info->location);
  res = gst_rtsp_url_parse (info->location, &info->url)
    /* create connection */
    GST_DEBUG_OBJECT (src, "creating connection (%s)...", info->location);
    res = gst_rtsp_connection_create (info->url, &info->connection)

  if (!info->connected) {
    /* connect */
    if (async)
      GST_ELEMENT_PROGRESS (src, CONTINUE, "connect",("Connecting to %s", info->location));
    GST_DEBUG_OBJECT (src, "connecting (%s)...", info->location);
    res = gst_rtsp_connection_connect (info->connection,src->ptcp_timeout)
    info->connected = TRUE;
  }
  return GST_RTSP_OK;
}

创建连接的流程
1. parsing uri 解析url
2. creating connection 将url地址放在info->connection 创建connection
3. 开始连接 connect 并且上报给BUS总线

GstRTSPResult gst_rtsp_connection_connect (GstRTSPConnection * conn, GTimeVal * timeout){

  g_socket_client_set_timeout (conn->client,
      (to + GST_SECOND - 1) / GST_SECOND);

  url = conn->url;

  gst_rtsp_url_get_port (url, &url_port);
  uri = gst_rtsp_url_get_request_uri (url);
  connection = g_socket_client_connect_to_uri (conn->client,uri, url_port, conn->cancellable, &error);

  /* get remote address */
  socket = g_socket_connection_get_socket (connection);
  collect_addresses (socket, &remote_ip, NULL, TRUE, &error)

  conn->remote_ip = remote_ip;
  conn->stream0 = G_IO_STREAM (connection);
  conn->socket0 = socket;
  /* this is our read socket */
  conn->read_socket = conn->socket0;
  conn->write_socket = conn->socket0;
  conn->input_stream = g_io_stream_get_input_stream (conn->stream0);
  conn->output_stream = g_io_stream_get_output_stream (conn->stream0);

  return GST_RTSP_OK;
}

上面在真正建立 socket连接的时候
1. 设置超时
2. 根据 url 和 port 创建socket连接 (使用glib库的socket创建方法)
3. 根据 socket 获取连接对象conn 获取输入/输出 流

到此 创建连接结束

根据rtsp 协议 走rtsp流程 (第一部分)

1. create options

  GST_DEBUG_OBJECT (src, "create options...");
  res = gst_rtsp_message_init_request (&request, GST_RTSP_OPTIONS,src->conninfo.url_str);

2. send OPTIONS

res =gst_rtspsrc_send (src, src->conninfo.connection, &request, &response,NULL);

然后接受 返回的response 消息 parse OPTIONS

gst_rtspsrc_parse_methods (src, &response)

3. create DESCRIBE

 res = gst_rtsp_message_init_request (&request, GST_RTSP_DESCRIBE,src->conninfo.url_str);

添加 header 表明只能接受sdp格式

  /* we only accept SDP for now */
  gst_rtsp_message_add_header (&request, GST_RTSP_HDR_ACCEPT,"application/sdp");

4. send DESCRIBE
不过在此之前 gstreamer 会给总线发送open的状态,告诉BUS 开始接受media info

GST_ELEMENT_PROGRESS (src, CONTINUE, "open", ("Retrieving media info"));

gst_rtspsrc_send (src, src->conninfo.connection, &request, &response, NULL)

发送 DESCRIBE 等待response
开始解析和检查返回的reponse 是否是 sdp

  /* check if reply is SDP */
  gst_rtsp_message_get_header (&response, GST_RTSP_HDR_CONTENT_TYPE, &respcont,0);

然后获取消息体 解析sdp

  /* get message body and parse as SDP */
  gst_rtsp_message_get_body (&response, &data, &size);
  GST_DEBUG_OBJECT (src, "parse SDP...");
  gst_sdp_message_new (sdp);
  gst_sdp_message_parse_buffer (data, size, *sdp);

根据接受sdp打开数据

/* must be called with the RTSP state lock */
static GstRTSPResult gst_rtspsrc_open_from_sdp (GstRTSPSrc * src, GstSDPMessage * sdp,gboolean async){

  /* prepare global stream caps properties */
    src->props = gst_structure_new_empty ("RTSPProperties");

  gst_rtsp_ext_list_parse_sdp (src->extensions, sdp, src->props);

  /* create streams */
  n_streams = gst_sdp_message_medias_len (sdp);
  for (i = 0; i < n_streams; i++) {
    gst_rtspsrc_create_stream (src, sdp, i);
  }
  src->state = GST_RTSP_STATE_INIT;
  /* setup streams */
  if ((res = gst_rtspsrc_setup_streams (src, async)) < 0)
    goto setup_failed;

  src->state = GST_RTSP_STATE_READY;
  return res;
}

根据rtsp 协议 走rtsp流程 (第二部分)

上面的代码根据rtsp 走了setup 确定传输机制，请求改变传输的参数
1. 创建sdp的属性并且解析，创建streams
2. 发送setup request

res = gst_rtspsrc_setup_streams (src, async)

gst_rtspsrc_play

static GstRTSPResult gst_rtspsrc_play (GstRTSPSrc * src, GstSegment * segment, gboolean async){
  GST_DEBUG_OBJECT (src, "PLAY...");
 res = gst_rtspsrc_ensure_open (src, async)

  for (walk = src->streams; walk; walk = g_list_next (walk)) {
    GstRTSPStream *stream = (GstRTSPStream *) walk->data;
    const gchar *setup_url;
    GstRTSPConnection *conn;

    /* do play */
    res = gst_rtsp_message_init_request (&request, GST_RTSP_PLAY, setup_url);

    if (async)
      GST_ELEMENT_PROGRESS (src, CONTINUE, "request", ("Sending PLAY request"));

    if ((res = gst_rtspsrc_send (src, conn, &request, &response, NULL)) < 0)
      goto send_error;

    /* some servers indicate RTCP parameters in PLAY response,
     * rather than properly in SDP */
    if (gst_rtsp_message_get_header (&response, GST_RTSP_HDR_RTCP_INTERVAL,
            &hval, 0) == GST_RTSP_OK)
      gst_rtspsrc_handle_rtcp_interval (src, hval);
  }
  /* set to PLAYING after we have configured the caps, otherwise we
   * might end up calling request_key (with SRTP) while caps are still
   * being configured. */
  gst_rtspsrc_set_state (src, GST_STATE_PLAYING);
  src->state = GST_RTSP_STATE_PLAYING;

  /* mark discont */
  GST_DEBUG_OBJECT (src, "mark DISCONT, we did a seek to another position");
  for (walk = src->streams; walk; walk = g_list_next (walk)) {
    GstRTSPStream *stream = (GstRTSPStream *) walk->data;
    stream->discont = TRUE;
  }

}

上面走的是paly协议流程
1. 初始化创建 play request
2. 发送 play request

    /* do play */
    res = gst_rtsp_message_init_request (&request, GST_RTSP_PLAY, setup_url);
    /*send play*/
    res = gst_rtspsrc_send (src, conn, &request, &response, NULL)

解析获取的response 信息

gst_rtsp_message_get_header (&response, GST_RTSP_HDR_RTCP_INTERVAL, &hval, 0)

3. 设置 为playing 状态
4. 断开连接

gst_rtspsrc_pause

static GstRTSPResult gst_rtspsrc_pause (GstRTSPSrc * src, gboolean async)
{
  GST_DEBUG_OBJECT (src, "PAUSE...");
  /* loop over the streams. We might exit the loop early when we could do an
   * aggregate control */
  for (walk = src->streams; walk; walk = g_list_next (walk)) {
    GstRTSPStream *stream = (GstRTSPStream *) walk->data;

    if (async)
      GST_ELEMENT_PROGRESS (src, CONTINUE, "request",("Sending PAUSE request"));

    if ((res =gst_rtsp_message_init_request (&request, GST_RTSP_PAUSE,
                setup_url)) < 0)
      goto create_request_failed;

    if ((res = gst_rtspsrc_send (src, conn, &request, &response, NULL)) < 0)
      goto send_error;
  }
  /* change element states now */
  gst_rtspsrc_set_state (src, GST_STATE_PAUSED);
}

上面的代码很容易看出来发送pause 请求 然后本地媒体设置为pause状态，停止发送信息过来

gst_rtspsrc_close

static GstRTSPResult gst_rtspsrc_close (GstRTSPSrc * src, gboolean async, gboolean only_close)
{

  for (walk = src->streams; walk; walk = g_list_next (walk)) {
    GstRTSPStream *stream = (GstRTSPStream *) walk->data;


    /* do TEARDOWN */
    res =gst_rtsp_message_init_request (&request, GST_RTSP_TEARDOWN, setup_url);
    if (async)
      GST_ELEMENT_PROGRESS (src, CONTINUE, "close", ("Closing stream"));

    if ((res = gst_rtspsrc_send (src, info->connection, &request, &response, NULL)) < 0)
      goto send_error;
  }
}

上面是rtsp 的teardown 流程代码

至此，一个大概的rtsp 流程已经出现