Android RTSP IPC的使用

RTSP(Real Time Streaming Protocol)简介

https://baike.baidu.com/item/RTSP/1276768?fr=aladdin

Android支持

APP Process=>VideoView=>MediaPlayer=>
mediaserver Process=>NuPlayerDriver=>NuPlayer=>RTSPSource=>AnotherPacketSource

frameworks\av\media\libstagefright\rtsp\MyHandler.h读取rtsp数据
=>processAccessUnitQueue=>postQueueAccessUnit=>
        case MyHandler::kWhatAccessUnit:
=>ATSParser::feedTSPacket
=>AnotherPacketSource 或者 ATSParser （支持mpeg2ts）

Android原生就支持RTSP的播放，上述代码流程就可以读取rtsp数据，存放到AnotherPacketSource，由NuPlayer调研解码器进行解码播放。

调用堆栈

01-12 09:49:33.109  5818  5818 I AEE_AED :     #10 pc 0008114f  /system/lib/libmediaplayerservice.so (android::AAVCAssembler::addNALUnit(android::sp<android::ARTPSource> const&)+294) (BuildId: f42ef5df58cf44e362a6075ce68d66eb)
01-12 09:49:33.109  5818  5818 I AEE_AED :     #11 pc 00081739  /system/lib/libmediaplayerservice.so (android::AAVCAssembler::assembleMore(android::sp<android::ARTPSource> const&)+4) (BuildId: f42ef5df58cf44e362a6075ce68d66eb)
01-12 09:49:33.109  5818  5818 I AEE_AED :     #12 pc 0007e7bb  /system/lib/libmediaplayerservice.so (android::ARTPAssembler::onPacketReceived(android::sp<android::ARTPSource> const&)+42) (BuildId: f42ef5df58cf44e362a6075ce68d66eb)
01-12 09:49:33.109  5818  5818 I AEE_AED :     #13 pc 00080467  /system/lib/libmediaplayerservice.so (android::ARTPSource::processRTPPacket(android::sp<android::ABuffer> const&)+62) (BuildId: f42ef5df58cf44e362a6075ce68d66eb)
01-12 09:49:33.109  5818  5818 I AEE_AED :     #14 pc 0007fb9b  /system/lib/libmediaplayerservice.so (android::ARTPConnection::parseRTP(android::ARTPConnection::StreamInfo*, android::sp<android::ABuffer> const&)+338) (BuildId: f42ef5df58cf44e362a6075ce68d66eb)
01-12 09:49:33.110  5818  5818 I AEE_AED :     #15 pc 0007f9ef  /system/lib/libmediaplayerservice.so (android::ARTPConnection::receive(android::ARTPConnection::StreamInfo*, bool)+182) (BuildId: f42ef5df58cf44e362a6075ce68d66eb)
01-12 09:49:33.110  5818  5818 I AEE_AED :     #16 pc 0007f517  /system/lib/libmediaplayerservice.so (android::ARTPConnection::onPollStreams()+258) (BuildId: f42ef5df58cf44e362a6075ce68d66eb)
01-12 09:49:33.110  5818  5818 I AEE_AED :     #17 pc 0000ff6d  /system/lib/libstagefright_foundation.so (android::AHandler::deliverMessage(android::sp<android::AMessage> const&)+24) (BuildId: b7def77b0a6eb9d32f954c8682733384)
01-12 09:49:33.110  5818  5818 I AEE_AED :     #18 pc 0001234d  /system/lib/libstagefright_foundation.so (android::AMessage::deliver()+64) (BuildId: b7def77b0a6eb9d32f954c8682733384)
01-12 09:49:33.110  5818  5818 I AEE_AED :     #19 pc 000106c7  /system/lib/libstagefright_foundation.so (android::ALooper::loop()+498) (BuildId: b7def77b0a6eb9d32f954c8682733384)
01-12 09:49:33.110  5818  5818 I AEE_AED :     #20 pc 0000d8db  /system/lib/libutils.so (android::Thread::_threadLoop(void*)+318) (BuildId: 61529cc15275456efab07e70acf12d44)
01-12 09:49:33.110  5818  5818 I AEE_AED :     #21 pc 000a60b3  /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+20) (BuildId: 7329cc0764665065af8bebfd78bb43fe)
01-12 09:49:33.110  5818  5818 I AEE_AED :     #22 pc 00060783  /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30) (BuildId: 7329cc0764665065af8bebfd78bb43fe)
01-12 09:49:33.110  5818  5818 I AEE_AED : ne_direct_unwind end

log

	Line 73304: 01-13 08:49:52.602   663  5466 I RTSPSource: loadcustomheaders mRTSPIPCCamera 1 mRTSPIPCCameraID 88 this 0xea345260
	Line 73305: 01-13 08:49:52.602   663  5466 I RTSPSource: RTSPSource mRTSPIPCCamera 1 mRTSPIPCCameraID 88 this 0xea345260
	Line 73341: 01-13 08:49:52.610   663  6142 I RTSPSource: connection request completed with result 0 (Success)
	Line 73344: 01-13 08:49:52.612   663  5466 I RTSPSource: loadcustomheaders mRTSPIPCCamera 1 mRTSPIPCCameraID 89 this 0xea3450a0
	Line 73345: 01-13 08:49:52.612   663  5466 I RTSPSource: RTSPSource mRTSPIPCCamera 1 mRTSPIPCCameraID 89 this 0xea3450a0
	Line 73365: 01-13 08:49:52.618   663  6147 I RTSPSource: connection request completed with result 0 (Success)
	Line 74103: 01-13 08:49:52.680   663  6142 I RTSPSource: DESCRIBE completed with result 0 (Success)
	Line 74244: 01-13 08:49:52.690   663  6147 I RTSPSource: DESCRIBE completed with result 0 (Success)
	Line 74440: 01-13 08:49:52.716   663  6142 I RTSPSource: SETUP(1) completed with result 0 (Success)
	Line 74441: 01-13 08:49:52.716   663  6147 I RTSPSource: SETUP(1) completed with result 0 (Success)
	Line 74442: 01-13 08:49:52.716   663  6147 I RTSPSource: server specified timeout of 60 secs.
	Line 74443: 01-13 08:49:52.716   663  6142 I RTSPSource: server specified timeout of 60 secs.
	Line 74444: 01-13 08:49:52.716   663  6147 W RTSPSource: Missing 'source' field in Transport response. Using RTSP endpoint address.
	Line 74445: 01-13 08:49:52.716   663  6142 W RTSPSource: Missing 'source' field in Transport response. Using RTSP endpoint address.
	Line 74547: 01-13 08:49:52.744   663  6147 I RTSPSource: SETUP(2) completed with result 0 (Success)
	Line 74548: 01-13 08:49:52.744   663  6147 I RTSPSource: server specified timeout of 60 secs.
	Line 74549: 01-13 08:49:52.744   663  6147 W RTSPSource: Missing 'source' field in Transport response. Using RTSP endpoint address.
	Line 74743: 01-13 08:49:52.759   663  6147 I RTSPSource: PLAY completed with result 0 (Success)
	Line 74744: 01-13 08:49:52.759   663  6147 I RTSPSource: This is a live stream
	Line 75533: 01-13 08:49:52.837   663  6142 I RTSPSource: SETUP(2) completed with result 0 (Success)
	Line 75534: 01-13 08:49:52.837   663  6142 I RTSPSource: server specified timeout of 60 secs.
	Line 75535: 01-13 08:49:52.837   663  6142 W RTSPSource: Missing 'source' field in Transport response. Using RTSP endpoint address.
	Line 75569: 01-13 08:49:52.844   663  6142 I RTSPSource: PLAY completed with result 0 (Success)
	Line 75570: 01-13 08:49:52.844   663  6142 I RTSPSource: This is a live stream
	Line 89515: 01-13 08:49:57.818   663  6147 I RTSPSource: Time now established for all tracks.
	Line 92958: 01-13 08:50:02.845   663  6142 W RTSPSource: We received some RTCP packets, but time could not be established on all tracks, now using fake timestamps
	Line 92959: 01-13 08:50:02.845   663  6142 I RTSPSource: Time now established for all tracks.
	Line 101959: 01-13 08:50:16.869   663  6147 W RTSPSource: This is a live stream, ignoring pause request.
	Line 102070: 01-13 08:50:17.869   663  6147 W RTSPSource: This is a live stream, ignoring pause request.
	Line 102881: 01-13 08:50:18.870   663  6147 W RTSPSource: This is a live stream, ignoring pause request.

码流流程

void ARTSPConnection::onReceiveResponse() {
  bool ARTSPConnection::receiveRTSPReponse() {
    sp<AMessage> notify = new AMessage('biny', this);
    void ARTPConnection::injectPacket(int index, const sp<ABuffer> &buffer) {
      void ARTPConnection::onInjectPacket(const sp<AMessage> &msg) {
        status_t ARTPConnection::parseRTP(StreamInfo *s, const sp<ABuffer> &buffer) {
			void ARTPSource::processRTPPacket(const sp<ABuffer> &buffer) {
  				void ARTPAssembler::onPacketReceived(const sp<ARTPSource> &source) {
    				ARTPAssembler::AssemblyStatus AAVCAssembler::assembleMore(
     					ARTPAssembler::AssemblyStatus AAVCAssembler::addNALUnit(
       						void AAVCAssembler::addSingleNALUnit(const sp<ABuffer> &buffer) {
								void AAVCAssembler::submitAccessUnit() {
								sp<AMessage> msg = mNotifyMsg->dup();
								msg->setBuffer("access-unit", accessUnit);
这一步很远，一个notify msg的回调
MyHandler.h
      virtual void onMessageReceived(const sp<AMessage> &msg) {
            case 'accu':
    void onAccessUnitComplete(
      status_t processAccessUnitQueue(int32_t trackIndex) {
            void postQueueAccessUnit(
            size_t trackIndex, const sp<ABuffer> &accessUnit) {
				void NuPlayer::RTSPSource::onMessageReceived(const sp<AMessage> &msg) {
        		case MyHandler::kWhatAccessUnit:

以上从RTSP收到的数据，汇聚到ATSParser或者AnotherPacketSource，然后稍后走到MediaPlayer的datasource，就可以解码播放

status_t ATSParser::feedTSPacket(const void *data, size_t size,
或者
void AnotherPacketSource::queueAccessUnit(const sp<ABuffer> &buffer) {

以下是RTCP的流程：
status_t ARTPConnection::parseRTCP(StreamInfo *s, const sp<ABuffer> &buffer) {
	    void onTimeUpdate(int32_t trackIndex, uint32_t rtpTime, uint64_t ntpTime) {

            if (allTracksHaveTime) {
                mAllTracksHaveTime = true;
                ALOGI("Time now established for all tracks.");
            }
以上需要通过RTCP来获取到track的时间戳，否则

        if (!mAllTracksHaveTime) {
            ALOGW("storing accessUnit, no time established yet");
            return;
        }
会挡住onAccessUnitComplete，不feedback packet数据回去解码器，导致不解码

问题

显示慢/延时大

随机断开

01-14 10:56:43.111 17000 21658 I RTSPSource: stream ended? aborting.
01-14 10:56:43.112 17000 21658 E NetworkUtils: Failed untagging socket 14 (My UID=0)
01-14 10:56:43.115 17000 21658 E NetworkUtils: Failed untagging socket 25 (My UID=0)
01-14 10:56:43.170 17000 21658 I ARTSPConnection: status: RTSP/1.0 200 OK
01-14 10:56:43.170 17000 21658 I RTSPSource: TEARDOWN completed with result 0 (Success)

参考

https://blog.csdn.net/chen495810242/article/details/39207305
https://blog.csdn.net/sdsdfser/article/details/21519291
http://itindex.net/detail/51966-%E6%B5%B7%E5%BA%B7-rtsp-%E5%AE%A2%E6%88%B7%E7%AB%AF