mediastream2使用指南(转载)

http://blog.sina.com.cn/s/blog_59d649610100diui.html

 

  定义

Filter: 媒体库中处理数据的组件。一个filter有0到数个输入流和0到数个输出流。

 

下面是可以使用Filter的例子:

   捕获音频或者视频数据.

   播放音频或者显示视频数据.

   发送或者接受RTP数据.

   对音频或者视频数据的编解码

   变化 (视频大小调整, 音频取样等等) 数据.

   复制数据.

   混和音频视频数据.

Graph: 是管理几个连接在一起的filter的组件. 它能够把数据从输出流传输到输入流 并且负责管理这些filters.

 

如何使用媒体库?

媒体库提供许多功能, 它主要用来管理RTP 音频和视频 会话.你将需要使用API函数创建filters,并且把它们连接进一个graph. 然后使用API函数能够用来启动和停止graph.

最简单的graph 例子:

  AUDIO CAPTURE   -->   ENCODE  -->     RTP

      FILTER      -->   FILTER  -->    FILTER

上面的graph 由三个 filters组成. Audio capture filter没有输入,直接从驱动捕获音频数据然后提供给输出流. 然后把输出流连接到encoder filter 的输入流上,coder filter 把这些音频数据进行编码以后然后送给输出流.最后把这个输出流连接到rtp filter的输入流上,然后由rtp filter把数据封装成rtp送出去。

该模块的设计可以让应用开发者使用自己的encode/decode filter 替代已有的"encode/decode filter" . 该媒体库支持g711u, g711a, h263的encode filter. 应用开发者也可以动态加入自己实现的encode filter,用来支持其他的编码格式。

媒体库的初始化

要使用媒体库, 首先需要对他进行初始化:

    ms_init();

媒体库提供许多filters. 这些filter必须被连接在一起以至一个filter的输出能够变成其他filter的输入.

通常, filters 被用来处理音频和视频数据. 他们能够捕获音频和视频数据,播放音频和视频数据, 对这些数据进行编解码, 合成这些数据 , 转换这些数据.最重要的filters 是RTP filters,他能够接收和发送RTP数据.

Graph的例子

要真正使用媒体库进行通话, 就必须构建两个graphs. 当然这两个graph都相当简单。

第一个graph需要三个filter,从声卡捕获数据,然后进行编码,最后把编码后的数据发送给RTP session.

             AUDIO    ->    ENCODER   ->   RTP

            CAPTURE   ->              ->  SENDER

第二个graph同样需要三个filter, 从一个RTP session接收数据,然后解码,最后把解码后的数据发送给播放设备.

        RTP      ->    DECODER   ->   AUDIO

       RECEIVER  ->              ->  PLAYBACK

初始化Graph的例子

为了便于阅读和理解,下面的代码没有差错检查.为了构造一个graph, 需要一些而外的输入输出设备:比如需要选择一个声卡用来捕获和播放音频,同时还需要使用创建一个RTP session用来发送和接受rtp流.

      MSSndCard *sndcard;

      sndcard=ms_snd_card_manager_get_default_card(ms_snd_card_manager_get());

       

      MSFilter *soundread=ms_snd_card_create_reader(captcard);

      MSFilter *soundwrite=ms_snd_card_create_writer(playcard);

      MSFilter *encoder=ms_filter_create_encoder("PCMU");

      MSFilter *decoder=ms_filter_create_decoder("PCMU");

      MSFilter *rtpsend=ms_filter_new(MS_RTP_SEND_ID);

      MSFilter *rtprecv=ms_filter_new(MS_RTP_RECV_ID);

      RtpSession *rtp_session = *** your_ortp_session *** ;

      ms_filter_call_method(rtpsend,MS_RTP_SEND_SET_SESSION,rtp_session);

      ms_filter_call_method(rtprecv,MS_RTP_RECV_SET_SESSION,rtp_session);

      MSFilter *dtmfgen=ms_filter_new(MS_DTMF_GEN_ID);

大多数情况下上面的graph还是不够的: 通常还需要对filter进行一些配置. ,比如还需要设置声卡filters的取样率。

      int sr = 8000;

      int chan=1;

      ms_filter_call_method(soundread,MS_FILTER_SET_SAMPLE_RATE,&sr);

      ms_filter_call_method(soundwrite,MS_FILTER_SET_SAMPLE_RATE,&sr);

      ms_filter_call_method(stream->encoder,MS_FILTER_SET_SAMPLE_RATE,&sr);

      ms_filter_call_method(stream->decoder,MS_FILTER_SET_SAMPLE_RATE,&sr);

      ms_filter_call_method(soundwrite,MS_FILTER_SET_NCHANNELS, &chan);

     

      MSTicker *ticker=ms_ticker_new();

      ms_ticker_attach(ticker,soundread);

      ms_ticker_attach(ticker,rtprecv);

解除filters并且停止graph的例子

      ms_ticker_detach(ticker,soundread);

      ms_ticker_detach(ticker,rtprecv);

      ms_filter_unlink(stream->soundread,0,stream->encoder,0);

      ms_filter_unlink(stream->encoder,0,stream->rtpsend,0);

      ms_filter_unlink(stream->rtprecv,0,stream->decoder,0);

      ms_filter_unlink(stream->decoder,0,stream->dtmfgen,0);

      ms_filter_unlink(stream->dtmfgen,0,stream->soundwrite,0);

      if (rtp_session!=NULL) rtp_session_destroy(rtp_session);

      if (rtpsend!=NULL) ms_filter_destroy(rtpsend);

      if (rtprecv!=NULL) ms_filter_destroy(rtprecv);

      if (soundread!=NULL) ms_filter_destroy(soundread);

      if (soundwrite!=NULL) ms_filter_destroy(soundwrite);

      if (encoder!=NULL) ms_filter_destroy(encoder);

      if (decoder!=NULL) ms_filter_destroy(decoder);

      if (dtmfgen!=NULL) ms_filter_destroy(dtmfgen);

      if (ticker!=NULL) ms_ticker_destroy(ticker);

 

mediastream.c的一些说明

 http://blog.sina.com.cn/s/blog_59d649610100diua.html

2009/6/29
mediastream.c的一些说明





                                       

转载,出处。(http://eatdrinkmanwoman.spaces.live.com/blog/cns!97719476F5BAEDA4!1003.entry)

mediastream.c是mediastream2库自带的一个test,也是最为复杂的一个test,学习它有助于加深对mediastreamer2的理解。

简介一下它的功能
1 利用mediastreamer2库封装的filter完成:从声卡捕捉声音,编码后通过rtp发送给远端主机,同时接收远端主机发来的rtp包,解码到声卡回放。
  filter graph如下:
  soundread -> ec -> encoder -> rtpsend
  rtprecv -> decode -> dtmfgen -> ec -> soundwrite

2 利用mediastreamer2库封装的filter完成:从摄像头捕捉图像,编码后通过rtp发送给远端主机(有本地视频预览),同时接收远端主机发来的rtp包,解码后视频回放。
  filter graph如下:
  source -> pixconv -> tee -> encoder -> rtpsend
                       tee -> output
  rtprecv -> decoder -> output

这个程序没有实现:用2个session来分别同时传送视频和音频。所以不要造成误解。
它实现的是:用1个全双工的session来传送视频或者音频,不管是本机还是远端主机,运行的都是同一个程序,一次只能选择一种payload。
牢 记rfc3550 Page 17中的所说“Separate audio and video streams SHOULD NOT be carried in a single RTP session and demultiplexed based on the payload type or SSRC fields. ”

程序中audio_stream_new() video_stream_new()内使用create_duplex_rtpsession()建立起监听端口。
比较奇怪的是video_stream_start()最后没有attach上rtprecv。而audio_stream_start_full()里有attach rtprecv。

编译的时候,别忘了加-D VIDEO_ENABLED启用视频支持。

程序命令参数
mediastream --local <port> --remote <ip:port> --payload <payload type number>
          [ --fmtp <fmtpline>] [ --jitter <miliseconds>]

这里fmtp和jitter是可选
fmtp的介绍如下:
Sets a send parameters (fmtp) for the PayloadType. This method is provided for applications using RTP with SDP, but actually the ftmp information is not used for RTP processing.
jitter就是设定缓冲时间,也就是队列的阀值。具体可以参见Comer所著TCP/IP 卷一的RTP一章。默认是80ms(还是50ms?),没必要修改它。

举一个使用的例子。
主机A IP 10.10.104.198
主机B IP 10.10.104.199
主机A 运行 ./mediastream --local 5010 --remote 10.10.104.199:6014 --payload 110
主机B 运行 ./mediastream --local 6014 --remote 10.10.104.198:5010 --payload 110

这里我使用的是音频传输,speex_nb编码。视频没有使用,怀疑是SDL有些问题,视频预览的时候是绿屏。

注意:程序代码里提到的音频编码有lpc1015,speex_nb,speex_wb,ilbc等,视频编码有h263_1998,theora,mp4v,x_snow,h264等。但是你却不一定能用得起来。这要看之前编译ffmpeg时,究竟是否指定了如上编码。
如果你的机子上并没有这些库,却又指定了这些库的payload type,那么mediastreamer2初始化的时候,会在终端输出错误信息找不到xxx.so之类,那么请换另一种payload type。
一般来说,speex,theora,xvid(h264)这三个比较容易编译。


关于rtp session,不知道你会不会像我有一些误解(尤其是双工的session)。
我 觉得session是一个难以一言蔽之的概念(否则rfc3550上也不会唠唠叨叨说一大堆,Page 9),虽然也可以精炼说成“The distinguishing feature of an RTP session is that each maintains a full, separate space of SSRC identifiers”,但是这样没什么意义,让人理解起来反而更困难。
我个人认为不必深究session的确切定义,而要细细体会rfc3550 Page 68 Section 11。(我每次有疑问时,就再来看一遍这一节)
1.RTP relies on the underlying protocol(s) to provide demultiplexing of RTP data and RTCP control streams.  For UDP and similar protocols,RTP SHOULD use an even destination port number and the corresponding RTCP stream SHOULD use the next higher (odd) destination port number.
2.For applications in which the RTP and RTCP destination port numbers are specified via explicit, separate parameters (using a signaling protocol or other means), the application MAY disregard the restrictions that the port numbers be even/odd and consecutive although the use of an even/odd port pair is still encouraged. 
3.The RTP and RTCP port numbers MUST NOT be the same since RTP relies on the port numbers to demultiplex the RTP data and RTCP control streams.
4.In a unicast session, both participants need to identify a port pair for receiving RTP and RTCP packets.  Both participants MAY use the same port pair.  A participant MUST NOT assume that the source port of the incoming RTP or RTCP packet can be used as the destination port for outgoing RTP or RTCP packets.
5.RTP data packets contain no length field or other delineation,therefore RTP relies on the underlying protocol(s) to provide a length indication.  The maximum length of RTP packets is limited only by the underlying protocols.
6.(原话找不到了)RTP本身并不知道该使用remote host的什么端口来传输,这需要用Non-RTP means来告知(比如和远端主机之间的信令交互得知),而本程序中没有信令交互,是显式指定。


最后把主机A上的运行结果贴一下

[atom@localhost code]$ ./mediastream --local 5010 --remote 10.10.104.199:6014 --payload 110 > atom
ortp-message-Registering all filters...
ortp-message-Registering all soundcard handlers
ortp-message-Card ALSA: default device added
ortp-message-Card ALSA: Ensoniq AudioPCI added
ortp-message-Card OSS: /dev/dsp added
ortp-message-Card OSS: /dev/dsp added
ortp-message-Loading plugins
ortp-message-Cannot open directory /usr/lib/mediastreamer/plugins: No such file or directory
ortp-message-ms_init() done
ortp-message-Setting audio encoder network bitrate to 8000
ortp-message-ms_filter_link: MSAlsaRead:0x93f0db0,0-->MSSpeexEnc:0x93f0ea0,0
ortp-message-ms_filter_link: MSDtmfGen:0x93f0d30,0-->MSAlsaWrite:0x93f0e28,0
ortp-message-ms_filter_link: MSSpeexEnc:0x93f0ea0,0-->MSRtpSend:0x93f0c18,0
ortp-message-ms_filter_link: MSRtpRecv:0x93f0c88,0-->MSSpeexDec:0x93f0f60,0
ortp-message-ms_filter_link: MSSpeexDec:0x93f0f60,0-->MSDtmfGen:0x93f0d30,0
ortp-message-Using bitrate 2150 for speex encoder.
ortp-message-alsa_open_r: opening default at 8000Hz, bits=16, stereo=0
ortp-message-synchronizing timestamp, diff=960
ortp-message-synchronizing timestamp, diff=320
ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=47
 number of rtp bytes sent=1636 bytes
 number of rtp packet received=49
 number of rtp bytes received=1927 bytes
 number of incoming rtp bytes successfully delivered to the application=1739
 number of times the application queried a packet that didn't exist=107
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=99
 number of rtp bytes sent=3254 bytes
 number of rtp packet received=102
 number of rtp bytes received=3683 bytes
 number of incoming rtp bytes successfully delivered to the application=3629
 number of times the application queried a packet that didn't exist=213
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=152
 number of rtp bytes sent=5554 bytes
 number of rtp packet received=154
 number of rtp bytes received=5077 bytes
 number of incoming rtp bytes successfully delivered to the application=5038
 number of times the application queried a packet that didn't exist=318
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=205
 number of rtp bytes sent=6671 bytes
 number of rtp packet received=207
 number of rtp bytes received=5964 bytes
 number of incoming rtp bytes successfully delivered to the application=5925
 number of times the application queried a packet that didn't exist=424
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=258
 number of rtp bytes sent=8632 bytes
 number of rtp packet received=260
 number of rtp bytes received=7422 bytes
 number of incoming rtp bytes successfully delivered to the application=7292
 number of times the application queried a packet that didn't exist=530
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-Receiving RTCP SR
ortp-message-interarrival jitter=121
ortp-message-Receiving RTCP SDES
ortp-message-Found CNAME=unknown@unknown
ortp-message-Found TOOL=oRTP-0.14.2
ortp-message-Found NOTE=This is free sofware (LGPL) !
ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=312
 number of rtp bytes sent=10280 bytes
 number of rtp packet received=314
 number of rtp bytes received=10202 bytes
 number of incoming rtp bytes successfully delivered to the application=9970
 number of times the application queried a packet that didn't exist=636
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=364
 number of rtp bytes sent=12382 bytes
 number of rtp packet received=365
 number of rtp bytes received=11527 bytes
 number of incoming rtp bytes successfully delivered to the application=11501
 number of times the application queried a packet that didn't exist=742
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=416
 number of rtp bytes sent=14337 bytes
 number of rtp packet received=419
 number of rtp bytes received=14520 bytes
 number of incoming rtp bytes successfully delivered to the application=14346
 number of times the application queried a packet that didn't exist=847
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=470
 number of rtp bytes sent=16216 bytes
 number of rtp packet received=472
 number of rtp bytes received=15832 bytes
 number of incoming rtp bytes successfully delivered to the application=15752
 number of times the application queried a packet that didn't exist=953
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=523
 number of rtp bytes sent=18399 bytes
 number of rtp packet received=524
 number of rtp bytes received=16604 bytes
 number of incoming rtp bytes successfully delivered to the application=16578
 number of times the application queried a packet that didn't exist=1059
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-Receiving RTCP SR
ortp-message-interarrival jitter=133
ortp-message-Receiving RTCP SDES
ortp-message-Found CNAME=unknown@unknown
ortp-message-Found TOOL=oRTP-0.14.2
ortp-message-Found NOTE=This is free sofware (LGPL) !
ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=575
 number of rtp bytes sent=20072 bytes
 number of rtp packet received=578
 number of rtp bytes received=17986 bytes
 number of incoming rtp bytes successfully delivered to the application=17754
 number of times the application queried a packet that didn't exist=1164
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=629
 number of rtp bytes sent=22106 bytes
 number of rtp packet received=630
 number of rtp bytes received=20753 bytes
 number of incoming rtp bytes successfully delivered to the application=20637
 number of times the application queried a packet that didn't exist=1270
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=681
 number of rtp bytes sent=24917 bytes
 number of rtp packet received=683
 number of rtp bytes received=21885 bytes
 number of incoming rtp bytes successfully delivered to the application=21859
 number of times the application queried a packet that didn't exist=1376
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=734
 number of rtp bytes sent=27703 bytes
 number of rtp packet received=736
 number of rtp bytes received=22594 bytes
 number of incoming rtp bytes successfully delivered to the application=22555
 number of times the application queried a packet that didn't exist=1481
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=788
 number of rtp bytes sent=28977 bytes
 number of rtp packet received=789
 number of rtp bytes received=23637 bytes
 number of incoming rtp bytes successfully delivered to the application=23483
 number of times the application queried a packet that didn't exist=1587
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-Receiving RTCP SR
ortp-message-interarrival jitter=127
ortp-message-Receiving RTCP SDES
ortp-message-Found CNAME=unknown@unknown
ortp-message-Found TOOL=oRTP-0.14.2
ortp-message-Found NOTE=This is free sofware (LGPL) !
ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=840
 number of rtp bytes sent=30072 bytes
 number of rtp packet received=837
 number of rtp bytes received=25564 bytes
 number of incoming rtp bytes successfully delivered to the application=25564
 number of times the application queried a packet that didn't exist=1692
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Audio session's RTP statistics :
 number of rtp packet sent=840
 number of rtp bytes sent=30072 bytes
 number of rtp packet received=837
 number of rtp bytes received=25564 bytes
 number of incoming rtp bytes successfully delivered to the application=25564
 number of times the application queried a packet that didn't exist=1693
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-ms_filter_unlink: MSAlsaRead:0x93f0db0,0-->MSSpeexEnc:0x93f0ea0,0
ortp-message-ms_filter_unlink: MSDtmfGen:0x93f0d30,0-->MSAlsaWrite:0x93f0e28,0
ortp-message-ms_filter_unlink: MSSpeexEnc:0x93f0ea0,0-->MSRtpSend:0x93f0c18,0
ortp-message-ms_filter_unlink: MSRtpRecv:0x93f0c88,0-->MSSpeexDec:0x93f0f60,0
ortp-message-ms_filter_unlink: MSSpeexDec:0x93f0f60,0-->MSDtmfGen:0x93f0d30,0
ortp-message-MSTicker thread exiting

Remote addr: ip=10.10.104.199 port=6014
Starting audio stream.
Bandwidth usage: download=26.614873 kbits/sec, upload=24.850787 kbits/sec
Bandwidth usage: download=24.757891 kbits/sec, upload=23.288695 kbits/sec
Bandwidth usage: download=21.915512 kbits/sec, upload=28.845805 kbits/sec
Bandwidth usage: download=18.254627 kbits/sec, upload=19.849715 kbits/sec
Bandwidth usage: download=22.314123 kbits/sec, upload=26.907152 kbits/sec
Bandwidth usage: download=33.280316 kbits/sec, upload=24.266766 kbits/sec
Bandwidth usage: download=21.465322 kbits/sec, upload=27.740203 kbits/sec
Bandwidth usage: download=34.365668 kbits/sec, upload=26.550016 kbits/sec
Bandwidth usage: download=21.669883 kbits/sec, upload=26.035174 kbits/sec
Bandwidth usage: download=17.581689 kbits/sec, upload=28.368437 kbits/sec
Bandwidth usage: download=22.050596 kbits/sec, upload=23.840863 kbits/sec
Bandwidth usage: download=32.469631 kbits/sec, upload=27.263791 kbits/sec
Bandwidth usage: download=19.910096 kbits/sec, upload=32.475279 kbits/sec
Bandwidth usage: download=16.712168 kbits/sec, upload=32.902848 kbits/sec
Bandwidth usage: download=19.092574 kbits/sec, upload=21.455217 kbits/sec
Bandwidth usage: download=24.896035 kbits/sec, upload=19.984568 kbits/sec
stoping all...
 
 
用mediastream+ortp库使用本机摄像头的简单程序
http://blog.sina.com.cn/s/blog_6a9032c10100lyjz.html

用mediastream+ortp库使用本机摄像头的简单程序

#include "mediastreamer2/mediastream.h" 
#include "mediastreamer2/msvideoout.h" 
#include "mediastreamer2/msv4l.h"

int main(int argc, char *argv[]){ 
VideoStream *vs; 
MSWebCam *cam; 
MSVideoSize vsize; 
int i;

vsize.width=MS_VIDEO_SIZE_CIF_W; 
vsize.height=MS_VIDEO_SIZE_CIF_H;

ortp_init(); 
ortp_set_log_level_mask(ORTP_MESSAGE|ORTP_WARNING|ORTP_ERROR|ORTP_FATAL); 
ms_init(); 
cam=ms_web_cam_manager_get_default_cam(ms_web_cam_manager_get()); 
//vs=video_preview_start(cam,vsize); 
//while(1); 
 
for(i=0;i<1;++i){ 
int n; 
vs=video_preview_start(cam,vsize);

for(n=0;n<1000;++n){ 
#ifdef WIN32 
MSG msg; 
Sleep(100); 
while (PeekMessage(&msg, NULL, 0, 0,1)){ 
TranslateMessage(&msg); 
DispatchMessage(&msg); 

#else 
struct timespec ts; 
ts.tv_sec=0; 
ts.tv_nsec=10000000; 
nanosleep(&ts,NULL);

if (vs) video_stream_iterate(vs); 
#endif

 
if (n==400) 

printf("this is 400\n"); 
ms_ticker_detach (vs->ticker, vs->source);

vs->tee = ms_filter_new(MS_TEE_ID);

ms_filter_unlink(vs->pixconv,0, vs->output,0);

ms_filter_link(vs->pixconv,0,vs->tee,0); 
ms_filter_link(vs->tee,0,vs->output,0); 
ms_filter_link(vs->tee,1,vs->output,1); 

//ms_filter_unlink(vs->tee,0,vs->output,0); 
ms_ticker_attach (vs->ticker, vs->source);


if (n==500) 

printf("this is 500\n"); 
int corner=1; 
ms_filter_call_method(vs->output,MS_VIDEO_OUT_SET_CORNER,&corner); 

if (n==600) 

printf("this is 600\n"); 
int corner=2; 
ms_filter_call_method(vs->output,MS_VIDEO_OUT_SET_CORNER,&corner); 

if (n==700) 

printf("this is 700\n"); 
int corner=3; 
ms_filter_call_method(vs->output,MS_VIDEO_OUT_SET_CORNER,&corner); 

if (n==800) 

printf("this is 800\n"); 
int corner=-1; 
ms_filter_call_method(vs->output,MS_VIDEO_OUT_SET_CORNER,&corner); 

if (n==900) 

printf("this is 900\n"); 
ms_ticker_detach (vs->ticker, vs->source);

ms_filter_unlink(vs->pixconv,0,vs->tee,0); 
ms_filter_unlink(vs->tee,0,vs->output,0); 
ms_filter_unlink(vs->tee,1,vs->output,1); 
ms_filter_destroy(vs->tee); 
vs->tee=NULL;

ms_filter_link(vs->pixconv,0, vs->output,0);


ms_ticker_attach (vs->ticker, vs->source); 


video_preview_stop(vs); 

return 0; 

转载于:https://www.cnblogs.com/virusolf/p/5016953.html

首先接到这一个项目,说是要用mediastreamer2做一个网络电话。之前也是从来没有接触过。于是首先开始在百度中搜索一下需要哪些东西,以及那些步骤。最后大致了解了一下,做这个项目最终要的就是需要移植好多的库,每一个库都需要配置,最后在交叉编译好动态库,然后在执行mediastreamer2的时候去调用这些动态库和头文件就OK了。 1、首先meidastream2是基于ortp库的,那么首先就是下载源码,交叉编译。 交叉编译ortp 下载源码:http://savannah.c3sl.ufpr.br/linphone/ortp/sources/?C=S;O=A 我使用0.18.0版本 ortp-0.18.0.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf ortp-0.18.0.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure CC=arm-linux-gcc --host=arm-linux --target=arm-linux --prefix=/home/protocol_stack/install/ make make install 然后用chmod 777 **.sh 执行脚本./**.sh 这样子就完成了配置,编译,安装。(安装目录为/home/protocol_stack/install/,也就是最后生成的头文件,可执行文件,库文件都会在这个目录下) 2、因为项目是要用到SIP协议的,所以我们还需要移植sip的库 osip2和eXosip2协议,这两个协议对应两个库,osip是简单的osip协议,但是因为API少等一系列原因,增加了eXosip2对osip2的补充。 交叉编译osip2 下载源码:http://ftp.gnu.org/gnu/osip/ 我使用的版本是3.6.0 libosip2-3.6.0.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf libosip2-3.6.0.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure --host=arm-linux --target=arm-linux --prefix=/home/protocol_stack/install/ make make install 然后用chmod 777 **.sh 执行脚本./**.sh 交叉编译eXosip2 下载源码:http://ftp.gnu.org/gnu/osip/ 我使用的版本是3.6.0 libeXosip2-3.6.0.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf libeXosip2-3.6.0.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure --host=arm-linux --target=arm-linux --prefix=/home/protocol_stack/install/ PKG_CONFIG_PATH=/home/protocol_stack/install/lib/pkgconfig make make install 然后用chmod 777 **.sh 执行脚本./**.sh 接下来可以编译mediastreamer2了,不过ms2,依赖好多库:ogg、speex、pulseaudio。而pulseaudio又依赖许多库:alsa、json、libtool。 3、交叉编译ogg 下载源码:http://xiph.org/downloads/ 我使用1.3.1版本 libogg-1.3.3.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf libogg-1.3.3.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure CC=arm-linux-gcc --prefix=/home/protocol_stack/install/ --host=arm-linux make make install 然后用chmod 777 **.sh 执行脚本./**.sh 4、交叉编译speex 下载源码:http://www.speex.org/downloads/ 我使用1.2rc1版本 speex-1.2rc1.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf speex-1.2rc1.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure CC=arm-linux-gcc --prefix=/home/protocol_stack/install/ --with-ogg=/home/protocol_stack/install/ --enable-fixed-point --disable-float-api \ --host=arm-linux make make install 然后用chmod 777 **.sh 执行脚本./**.sh 5、交叉编译pulseaudio 下载源码:http://freedesktop.org/software/pulseaudio/releases/ 我使用1.0版本 pulseaudio-1.0.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf pulseaudio-1.0.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure CC=arm-linux-gcc CXX=arm-linux-g++ --prefix=/home/protocol_stack/install --host=arm-linux --disable-rpath --disable-nls --disable-dbus --disable-bluez --disable-samplerate --disable-solaris --disable-gconf --disable-avahi --disable-jack --disable-lirc --disable-glib2 --disable-gtk2 --disable-openssl --disable-ipv6 --disable-asyncns --disable-per-user-esound-socket --disable-oss-output --disable-oss-wrapper --disable-x11 --enable-neon-opt=no --with-database=simple PKG_CONFIG_PATH=/home/protocol_stack/install/lib/pkgconfig CPPFLAGS=-I/home/protocol_stack/install/include LDFLAGS=-L/home/protocol_stack/install/lib CFLAGS=-I/home/protocol_stack/install/include make make install 然后用chmod 777 **.sh 执行脚本./**.sh 错误1: checking for ltdl.h... no configure: error: Unable to find libltdl version 2. Makes sure you have libtool 2.4 or later installed. make: *** No targets specified and no makefile found. Stop. 分析;找不到libltdl。确保你有libtool 2.4及以上的版本。 下载libtool 2.4.2版本 这时需要交叉编译libtool 下载源码:ftp://ftp.gnu.org/gnu/libtool/ 我使用2.4.2版本 libtool-2.4.2.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf libtool-2.4.2.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure --host=arm-linux --prefix =/home/protocol_stack/install/ make make install 然后用chmod 777 **.sh 执行脚本./**.sh 交叉编译alsa: http://www.alsa-project.org/main/index.php/Main_Page 这个库的版本需要根据你嵌入式Linux内核中alsa的版本而定,可以使用命令查看内核中alsa的版本: # cat /proc/asound/version Advanced Linux Sound Architecture Driver Version 1.0.24. 可以到内核中alsa驱动版本是1.0.24,所以我选1.0.24版本 alsa-lib-1.0.24.1.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf speex-1.2rc1.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure --host=arm-linux --prefix =/home/protocol_stack/install/ make make install 然后用chmod 777 **.sh 执行脚本./**.sh 错误:configure: error: Package requirements ( sndfile >= 1.0.20 ) were not met: No package 'sndfile' found 分析:缺少库 libsndfile库,那么接下来再进行交叉编译libsndfile libsndfile-1.0.25.tar.gz http://www.linuxfromscratch.org/blfs/view/svn/multimedia/libsndfile.html 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf libsndfile-1.0.25.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure --host=arm-linux --prefix =/home/protocol_stack/install/ make make install 然后用chmod 777 **.sh 执行脚本./**.sh 7、最后编译mediastreamer2 下载源码:http://ftp.twaren.net/Unix/NonGNU//linphone/mediastreamer/ 我使用2.8版本 mediastreamer-2.8.0.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf mediastreamer-2.8.0.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 把下边这三行写成一个脚本 vim **.sh ./configure CC=arm-linux-gcc --prefix=/home/protocol_stack/install/ PKG_CONFIG_PATH=/home/protocol_stack/install/lib/pkgconfig --disable-gsm --enable-video=no --enable-macsnd=no --disable-static --disable-sdl --disable-x11 --disable-ffmpeg --host=arm-linux --target=arm-linux make make install 然后用chmod 777 **.sh 执行脚本./**.sh 上面的configure选项没有屏蔽v4l1和v4l2,所以还得交叉编译v4l 编译v4l libv4l-0.6.4.tar.gz 下载源码:http://pkgs.fedoraproject.org/repo/pkgs/libv4l/ 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf libv4l-0.6.4.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 我使用0.6.4版本 libv4l-0.6.4.tar.gz make clean make CC=arm-linux-gcc make install PREFIX=/home/protocol_stack/install 编译mediastreamer2出错:(1)checking for LIBCHECK... no checking for LIBJSON... no configure: error: Package requirements ( json >= 0.9 ) were not met: No package 'json' found 解决方法就是交叉编译json 下载源码:http://ftp.debian.org/debian/pool/main/j/json-c/ 分析:缺少json库,那么我们继续交叉编译json库 json-c_0.12.1.orig.tar.gz 然后通过winshare(Windows和Linux的通信)把下载好的库文件拷贝到Linux下, 然后解压 tar zxvf mediastreamer-2.8.0.tar.gz 注意这个时候可能会发生错误,是没有权限的问题,那么就在命令行前边加上sudo 然后配置 ./configure --host=linux-arm \ --prefix =/home/protocol_stack/install/ make && make install 好了,json库已经编译完成了。接下来我们继续编译mediastreamer2 。。。。。 但是还是有问题,怎么办呢?还是哪个问题还是找不到json库。 分析:在json的论坛中,找到了解决方案:把编译生成的/lib/pkgconfig/这个目录下生成了一个json-c.pc。最后mediastreamer2在调用的时候找的是json.pc。那么我们就把这个文件名改为json.pc #mv json-c.pc json.pc OK,这次这个是可以编译的过去了。接下来继续编译 。。。 又出现问题了 /home/protocol_stack/install/lib/libjson.so: undefined reference to `rpl_malloc' /home/protocol_stack/install/lib/libjson.so: undefined reference to `rpl_realloc' 问题分析: 这个错误的原因是因为没有定义 rpl_malloc 和 rpl_realloc 这两个变量。 那么我们应该怎么办么? 那么就在这个目录下进行查这两个变量是在哪里定义的? 于是:#grep "rpl_malloc" -nR * ....... 找到了,原来这两个变量是在这个config的文件中的。是一个宏开关 那么就好办了,我们就直接把这两个宏进行注释。 嗯嗯,继续。。。我们重新编译json库。。。嗯嗯编译好了,接下来继续来编译mediastreamer2 。。。。 又出错了,还是这个原因 /home/protocol_stack/install/lib/libjson.so: undefined reference to `rpl_malloc' /home/protocol_stack/install/lib/libjson.so: undefined reference to `rpl_realloc 嗯嗯,还是这个原因?究竟是为什么呢。再次来到json的目录下,再次看有没有把那两个宏开关给关闭? 嗯哼? 竟然没有关闭? 分析?明白了。原来是我把配置和编译同时执行了。这个宏开关是./configure ...生成的。 那么就只好,这样。把./configure。。。生成的config文件,再进行关闭宏开关。最后直接make && make install -j8 直接编译,安装,是不能再次进行配置的。因为以配置config文件就会再次生成,那么宏开关就又开了。 OK,安装好了,下来继续进行编译mediastreamer2.。。。。。。。。。。。 。。。。。。。。。。。。。。 又出现了问题? error: /user/include/python2.7/pyconfig.h:15:52: fatal error: arm-linux-gnueabi/python2.7/pyconfig.h: No such file or directory compilation terminated. 分析::找不到arm-linux-gnueabi/python2.7/pyconfig.h这文件。那就继续交叉编译python 好吧,继续下载python,然后再进行交叉编译,但是编译Python的时候出来一系列的问题。根本没有办法解决。 那么该怎么办呢?时候一个小时又一个小时的过去? 最后有一个大胆的想法,既然python都编译不下去。那就不要了。 于是,在mediastreamer2的./configure 中添上一项 --without-python 。 。。。再次配置编译。。。。。。。。。。。 error: /user/include/python2.7/pyconfig.h:15:52: fatal error: arm-linux-gnueabi/python2.7/pyconfig.h: No such file or directory compilation termiated. 嗯哼?还是一样的错误。怎么办呢? 于是乎就又在论坛上进行找灵感。。。。。 还是找不到。。。 又一结合前边几个库的配置编译,发现不使能一个模块还可以用另外一个--disable-python 。。。 于是乎 就把--without-python改为了--disable-python 继续编译。。。。 。。。。。。。。。。。。。。。。。。。。。 到了这个节骨眼上了,编译每跳一下,我的心就跟着逗一下。。。。心酸 。。。。。。 。。。。。。 。。。。。。 竟然编译成功了。。。。 哈哈。。。。。。。。。 于是,马上就把编译好的库,拷贝到了开发板。。。 嗯嗯,本来还想把编译好的库目录树拷贝下的,但是太多了,放不下。。。算了吧。。。。 找到编译好的库 在库中的/bin中找到arm-linux-mediastream 然后执行./arm-linux-mediastream 。。。。报错了 问题: error : while loading shared libraries: libmediastreamer.so.1: cannot open shared object file: No such file 答案:分析: 遇到这个问题就是,libmediastreamer.so.1这个动态库,在可执行文件armlinuxmediastreamer执行的时候,会调用这个动态库,但是环境变量中找不到这个动态库。那么我们就是要把我们编译好的动态链接库的目录加到环境变量中 LD_LIBRARY_PATH=$LD_LIBRARY_PATH://arm/lib/这个目录下就是放着我们编译好的所有的动态链接库(包括libmediastreamer.so.1) 执行步骤:LD_LIBRARY_PATH=$LD_LIBRARY_PATH://arm/lib export LD_LIBRARY_PATH ./arm-linux-mediastream mediastream --local --remote --payload [ --fmtp ] [ --jitter ] [ --width ] [ --height ] [ --bitrate ] [ --ec (enable echo canceller) ] [ --ec-tail ] [ --ec-delay ] [ --ec-framesize ] [ --agc (enable automatic gain control) ] [ --ng (enable noise gate)] [ --ng-threshold (noise gate threshold) ] [ --ng-floorgain (gain applied to the signal when its energy is below the threshold.) ] [ --capture-card ] [ --playback-card ] [ --infile <input wav file> specify a wav file to be used for input, instead of soundcard ] [ --outfile specify a wav file to write audio into, instead of soundcard ] [ --camera ] [ --el (enable echo limiter) ] [ --el-speed (gain changes are smoothed with a coefficent) ] [ --el-thres (Threshold above which the system becomes active) ] [ --el-force (The proportional coefficient controlling the mic attenuation) ] [ --el-sustain (Time in milliseconds for which the attenuation is kept unchanged after) ] [ --el-transmit-thres (TO BE DOCUMENTED) ] [ --rc (enable adaptive rate control) ] [ --zrtp (enable zrtp) ] [ --verbose (most verbose messages) ] [ --video-windows-id <video surface:preview surface>] [ --srtp (enable srtp, master key is generated if absent from comand line) [ --netsim-bandwidth (simulates a network download bandwidth limit) 于是按照第一种方式进行 参数添加 ./arm-linux-mediastream --local 8888 --remote 127.0.0.1:88 88 OK运行正常了 下面是运行信息。。。 ortp-message-audio_stream_process_rtcp: interarrival jitter=119 , lost packets percentage since last report=0.000000, round trip time=0.000000 seconds ortp-message-oRTP-stats: RTP stats : ortp-message- number of rtp packet sent=150 ortp-message- number of rtp bytes sent=25800 bytes ortp-message- number of rtp packet received=150 ortp-message- number of rtp bytes received=25800 bytes ortp-message- number of incoming rtp bytes successfully delivered to the application=25284 ortp-message- number of rtp packet lost=0 ortp-message- number of rtp packets received too late=0 ortp-message- number of bad formatted rtp packets=0 ortp-message- number of packet discarded because of queue overflow=0 ortp-message-Bandwidth usage: download=81.290281 kbits/sec, upload=81.288664 kbits/sec ortp-message-Receiving RTCP SR ortp-message-Receiving RTCP SDES ortp-message-Found CNAME=unknown@unknown ortp-message-Found TOOL=oRTP-0.18.0 ortp-message-Found NOTE=This is free sofware (LGPL) ! ortp-message-Quality indicator : 4.888437 运行正常了。。。。。。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值