Media Streaming布署及测试

参考：http://parsa.epfl.ch/cloudsuite/streaming.html

一、软件准备

1.Darwin Streaming Server 服务器安装程序

   下载地址：http://dss.macosforge.org/downloads/DarwinStreamingSrvr6.0.3-Source.tar

2.Faban workload generator kit

  下载地址：http://java.net/projects/faban/

3.Stream driver, RTSP client code, and operation scripts (with the benchmark package)

4.cURL RTSP library

 下载地址：http://curl.haxx.se/download.html

二、安装客户端

1.下载并解压faban-kit-022311.tar，下载地址：http://java.net/projects/faban/，并将其目录重命名为faban-streaming

mv faban faban-streaming

2.将streaming文件夹拷到faban-streaming文件下，streaming文件夹可以从Media Streaming Benchmark package里获得，下载地址：

http://parsa.epfl.ch/cloudsuite/software/streaming.tar.gz

cp -r streaming faban-streaming

3. 确认安装java JDK并设置好JDK的环境变量。

4.Start the Faban master: From Faban main directory (faban-streaming)

cd faban-streaming

./master/bin/startup.sh

5.vi build.properties做如下修改：（操作此步之前要先安装ant；从/opt/faban-streaming/streaming目录下）

• Change faban.home to the path of the main Faban directory (e.g., /home/username/faban-streaming).
• Make sure faban.url is set to http://localhost:9980/.
• Change ant.home to the location of you ant installation.

6. From the streaming directory, run:  sh scripts/new_deploy.sh .

(操作到此步时报错：/opt/faban-streaming/streaming/build.xml:17: The <fileset> type doesn't support the "erroronmissingdir" attribute.

将build.xml的17行有erroronmissingdir的 整行删除)

7.      Prepare the RTSP client

1. • The code is provided in the same streaming directory, faban-streaming/streaming/rtspclientfinal.c, you need to download the cURL version that works on your platform.
   • unpack the downloaded cURL distribution.
   • To install cURL, in the curl directory:
     • mkdir curlinst
     • ./configure --prefix=/path/to/curldirectory/curlinst/ --enable-rtsp
     • make
     • make install
   • Now, compile the RTSP client code (from the /path/to/faban-streaming/streaming)

     gcc -I /path/to/curl/curlinst/include -L /path/to/curl/curlinst/lib/ -lcurl rtspclientfinal.c -o rtspclient.o

   • Make a preliminary test:

     ./rtspclient.o
     If the installation was done correctly, you should receive the following message: 
     "ERROR: enter a valid URL" simply because it expects a parameter indicating where to find the video.

     注：执行./rtspclient.o时报错：error while loading shared libraries: libcurl.so.4: cannot open shared object file: No such file 

               or directory

               解决方法：ln -s /opt/curl-7.24.0/curlinst/lib/libcurl.so.4  /usr/lib64/libcurl.so.4
               参考：http://hi.baidu.com/youngky2008/item/6788192b6a109ac3ddf69a7e

三、安装服务器端

1.解压服务器安装程序

tar xvf DarwinStreamingSrvr6.0.3-Source.tar

2.下载如下两个patches

• Patch one  下载地址：http://dss.macosforge.org/trac/raw-attachment/ticket/6/dss-6.0.3.patch
• Patch two   下载地址：http://dss.macosforge.org/trac/raw-attachment/ticket/6/dss-hh-20080728-1.patch

3.执行如下命令，应用这两个patches

patch -p0 < dss-6.0.3.patch 
patch -p0 < dss-hh-20080728-1.patch

4.替换DarwinStreamingSrvr6.0.3-Source文件夹里的Install文件为如下这个： one 下载地址：http://dss.macosforge.org/trac/raw-attachment/ticket/6/Install ；并添加Install的可执行权限。

chmod +x Install

5.Darwin streaming server要求用qtss组和qtss用户，建立qtss组和用户，命令如下：

sudo groupadd qtss
sudo useradd -g qtss qtss

6.更改/usr/include/bits/typesizes.h文件 里面的FD_SETSIZE 值为65536，使流媒体服务器能达到1024连接数。

  vi /usr/include/bits/typesizes.h

在相应位置添加：FD_SETSIZE=65536

7.Build the server

./Buildit

8.Install the server

./Install

9.调整服务器参数

vi /etc/streaming/streamingserver.xml

进行相应修改如下：

• real_rtsp_timeout to 700
• rtp_timeout to 700
• maximum_connections to -1 (infinite)
• maximum_bandwidth to -1 (infinite)
• movie_folder to the path of your movies directory
• run_num_threads should be set to a value higher than the number of cores you want to utilize (e.g., 8 was enough to saturate 4 cores)
• overbuffer_rate to 1.2

10. Check the limis imposed by the operating system on the maximum number of open file: (e.g., ulimit in Linux ). Set this value to unlimited or to a sufficient number for the number of streams your are simulating.

11.启动服务

/usr/local/sbin/DarwinStreamingServer -dDS 1

The server should print the main statistics every second!

屏幕出现如下图所示，如果提示端口被占用，可以将占用端口的程序杀死再试。

[root@gd88 streaming]# /usr/local/sbin/DarwinStreamingServer -dDS 1
INFO: Module Loaded...QTSSFileModule [static]
INFO: Module Loaded...QTSSReflectorModule [static]
INFO: Module Loaded...QTSSRelayModule [static]
INFO: Module Loaded...QTSSAccessLogModule [static]
INFO: Module Loaded...QTSSFlowControlModule [static]
INFO: Module Loaded...QTSSPosixFileSysModule [static]
INFO: Module Loaded...QTSSAdminModule [static]
INFO: Module Loaded...QTSSMP3StreamingModule [static]
INFO: Module Loaded...QTSSAccessModule [static]
Streaming Server done starting up
     RTP-Conns RTSP-Conns HTTP-Conns  kBits/Sec   Pkts/Sec   RTP-Playing   AvgDelay CurMaxDelay  MaxDelay  AvgQuality  NumThinned      Time
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:15
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:16
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:17
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:18
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:19
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:20
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:21
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:22
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:23
          0          0          0          0          0             0          0          0          0          0          0     2012-07-04 10:44:24

四、生成数据集

1.在服务器上 vim /etc/streaming/streamingserver.xml    将movies存放路径设置为你想要的位置  如:/opt/movies

 2.将streaming.tra.gz包里的视频文件夹streamingVideos_10和stressVideos 拷备到上一步设置的电影路径中，这里设置的为/opt/movies

 3.根据你需要的数据集规模，将streamingVideos_10拷备相应的份数，拷备后的将10改为11 12 13等连续命名文件夹，如你需要15G的数据，这时就需要拷备5份streamingVideos_11 streamingVideos_12 streamingVideos_14 streamingVideos_14

五、Run  the benchmark

1.vim vim scripts/run-test.sh 将faban-streaming文件夹的路径替换为你的faban-streaming所在路径。

2.vim  deploy/run.xml 修改如下几个参数

   1）Host IP address：不需要改，默认值即可

   2）The scale ：客户端并发数

   3）The ramp up, steady state and ramp down time.
Please note that the benchmark is designed to ramp up the clients based on the number of such emulated clients. Therefore, the ramp up time is not critical but should be greater than or equal to 5 seconds. The important parameter for a successful run is the ramp down time. This value should be set to 2* duration of longest video you are willing to use.

4）The number of agents：how many agent processes to control the total number of clients.
Please note that this number affects the speed of preparing the client threads. Along with this parameter, you should change therun-test.sh script by including separate commands for all the agents. For example if you want to use 15 agents instead of 10, you should add 10 extra lines in therun-test.sh.

5）Whether to start the agents concurrently or not: selecting yes starts all the agents at the same time, each creating the number of clients assigned to that agent.  --这个默认的是设置为true

6)The mix of videos (duration and bitrate) you want your clients to use. For example, to stream only short-duration and high bitrate videos you need to set the GetShortHi parameter to 100 and the rest to zero. If you want the requested videos to be 30% short and high and 70% medium duration and low bitrate, you should specify the GetShortHi to 30 and the GetMediumLow to 70 and the rest to zeros, and so on

7）IP Address of the server ：改为你的服务器IP地址

8）Data set scale ：这里我们拷备了5份，所以设置为5

9）Fix duration: is an optional parameter that is provided to give more control over the user run. It limits the duration of each requested video to this number. For example, if you want to use a mix of videos with different durations and bit rates but want to test only 1 minute of each video, you should specify this number as 1. A negative number means ignoring this parameter (by default it is -1 meaning that it is ignored).

10）Protocol code: is the directory where the RTSP client resides. The default case is under/path/to/faban-streaming/streaming.

3.From /path/to/faban-streaming/streaming
sh scripts/start-run.sh <NUM-CLIENTS> <MIX> <SERVER-IP> <DATASET-SCALE>

 

 

六、节点重启后测试

  1）启动服务器程序:在172.16.1.88上运行命令 /usr/local/sbin/DarwinStreamingServer -dDS 1

      这时屏幕每秒种显示服务器信息，这时连接数都为0

  2)在客户端172.16.48.201的/opt/faban-streaming目录下运行命令./master/bin/startup.sh 启动faban master （现在已经是启动状态,如果sg201重启则要再运行此命令)

      cd /opt/faban-streaming

      ./master/bin/startup.sh

  3) cd /opt/faban-streaming/streaming

     sh scripts/new_deploy.sh  (现在已经是启动状态,如果sg201重启则要再运行此命令)

  4）通过RTSP client进行测试

     cd /opt/faban-streaming/streaming

    sh scripts/start-run.sh(这时可以在服务器端看到连接数等信息,服务器的CPU利用率直线上升)

     参数修改：vimdeploy/run.xml  

        注：其中客户端数的参数为scale,目前设置的为7500，可自行修改。

 

 

 

附：

1.change-config.sh 文件内容：

sed "s/fa:scale></fa:scale>$1</g" /home/username/faban-streaming/streaming/deploy/run.xml  $2 > new.xml

sed "s/<ipAddress></<ipAddress>$3</g" new.xml > new2.xml

sed "s/<datasetScale></<datasetScale>$4</g" new2.xml > /home/username/faban-streaming/streaming/deploy/run.xml

rm new.xml

rm new2.xml

2.server端最大连接数调整

ulimit -n查询当前最大连接数

ulimit  -n 90000   #将最大连接数调为9000