Live Streaming Datasets--MPEG DASH编码的基于chunk的视频数据集下载

MPEG DASH简介

• Manifest.mpd (MPD:Media Presentation Description) 文件是MPEG DASH编码视频的索引文件,该文件包含了整个mpeg dash码流的构成(manifest以.mpd结尾).包括视频流与音频流信息,视频chunk信息,视频码率,帧率,编码,带宽等信息.类似于HLS编码视频的m3u8文件是.MPD是一个XML Document，通过MPD的内容可以构造出用于HTTP GET下载的URL。 详细介绍请看三种主流的流媒体协议MEPG DASH,HLS，Smooth Streaming及其manifest 文件字段解释
• 其优势就是不需要流媒体服务器缓存该视频的全部视频流,客户端请求视频服务器,服务器将manifest.mpd发送给客户端,客户端可以根据视频的URL+不同的byte range指定(chunk的url)进行任意chunk的下载,更加灵活.详细请看pensieve论文中视频文件的处理.
  
  Figure 1 illustrates the end-to-end process of streaming a video over HTTP today. As shown, a player embedded in a client application first sends a token to a video service provider for authentication. The provider responds with a manifest file that directs the client to a CDN hosting the video and lists the available bitrates for the video. The client then requests video chunks one by one, using an adaptive bitrate (ABR) algorithm. These algorithms use a variety of different inputs (e.g., playback buffer occupancy, throughput measurements, etc.) to select the bitrate for future chunks. As chunks are downloaded, they are played back to the client; note that playback of a given chunk cannot begin until the entire chunk has been downloaded.

MPEG DASH视频的chunk下载

视频来源

目前普遍使用的视频源来自于DASH264 JavaScript reference client test page,其中视频的编码方式是H.264/MPEG-4 AVC

chunk的下载步骤

1. 通过DASH264 JavaScript reference client test page获取所需视频的Manifest.mpd.根据Manifest.mpd中的period字段信息,我们可以获取到该视频一共包括几种视频流,每一种视频流的编码,带宽等信息.
   
2. 通过DASH264 JavaScript reference client test page获取所需视频的base_url
   
3. 根据Manifest.mpd中的period/segmentTemplate信息,构建每一种视频流的每个chunk(.m4s文件,视频片段)的url.然后通过requests.get(url)逐一下载视频片段(chunk).

代码实现

• chunk视频文件下载

# This script scraped all of the video chunks (65 per encod