iPhone/iPad札记2012/03/11 - 1 音频，混音

1. conference音频问题总结

＊ 音频双向都通过，但不稳定，时好时坏（录音只成功过一次，播放曾有过连续工作20－30分钟一切正常的记录，但重启系统后就出现问题）

    Q Action: to see how often the callback would be called ?

        A: playing: 10 ms, recording 3ms 

＊ 是否应该使用Audio Unit而非Audio Queue来实现VOIP功能？

     ＃ 据资料说，Audio Queue Service的callback访问时间在0.5秒以上

     ＃ Audio Unit资料说它比较适用于需要low-lantency的音频应用，如VOIP( used high-priority thread)

      REFERENCE: iOS provides audio processing plug-ins that support mixing, equalization, format conversion, and realtime input/output for recording, playback, offline rendering, and live conversation such as for VoIP (Voice over Internet Protocol). 

* 一个新的思路：

      # 另起一个thread去读session层的audio data，并把它们放进一个queue

      # Audio Queue callback从新的queue里拿数据去放

＊实验：

      ＃为什么iOS主线程readAudioData会有问题，而单独的thread则没有问题？

      ＃ 在windows/Android环境里重现相关speex的问题

＊Audio Engine

      ＃审阅相关代码

 ＊明天建立一个内网上的测试环境，杜绝公网上的网络状况的影响

  * 还可以做的尝试：把Speex换成G711

 

2. MixerHost sample code 

* configureAndInitializeAudioProcessingGraph

registration as follows 

        // Setup the struture that contains the input render callback 

        AURenderCallbackStruct inputCallbackStruct;

        inputCallbackStruct.inputProc        = &inputRenderCallback;

        inputCallbackStruct.inputProcRefCon  = soundStructArray;

  ^ this is an array of sound data 

        // Set a callback for the specified node's specified input

        result = AUGraphSetNodeInputCallback (

                     processingGraph,

                     mixerNode,

                     busNumber,

                     &inputCallbackStruct

                 );

sound struct 

typedef struct {

    BOOL                 isStereo;           // set to true if there is data in the audioDataRight member

    UInt32               frameCount;         // the total number of frames in the audio data

    UInt32               sampleNumber;       // the next audio sample to play

    AudioUnitSampleType  *audioDataLeft;     // the complete left (or mono) channel of audio data read from an audio file

    AudioUnitSampleType  *audioDataRight;    // the complete right channel of audio data read from an audio file

    

} soundStruct, *soundStructPtr;

3. Reading "Audio Unit " related guides - Component Service Reference 

* AURenderCallback 

typedef OSStatus (*AURenderCallback) (
   void                        *inRefCon,
   AudioUnitRenderActionFlags  *ioActionFlags,
   const AudioTimeStamp        *inTimeStamp,
   UInt32                      inBusNumber,
   UInt32                      inNumberFrames,
   AudioBufferList             *ioData
);

inRefCon

Custom data that you provided when registering your callback with the audio unit.

4. Test555程序

＊ 试验callback被回调的频率

    Q Action: to see how often the callback would be called ?

        A: playing: 10 ms, recording 3ms 

    Result: 

notification: Invalid mode encountered. The stream is corrupted.

2012-03-11 10:52:10.209 test555[324:207] AQBufferCallback...

2012-03-11 10:52:10.219 test555[324:207] AQBufferCallback...

2012-03-11 10:52:10.229 test555[324:207] AQBufferCallback...

2012-03-11 10:52:10.239 test555[324:207] AQBufferCallback...

[Session started at 2012-03-11 10:58:34 +0800.]

2012-03-11 10:58:34.707 test555[368:7303] InputBufferHandler...

2012-03-11 10:58:34.710 test555[368:7303] audio data sent to audio engine...

2012-03-11 10:58:34.710 test555[368:7303] InputBufferHandler...

2012-03-11 10:58:34.712 test555[368:7303] audio data sent to audio engine...

2012-03-11 10:58:34.713 test555[368:7303] InputBufferHandler...

2012-03-11 10:58:34.714 test555[368:7303] audio data sent to audio engine...

2012-03-11 10:58:34.715 test555[368:7303] InputBufferHandler...

2012-03-11 10:58:34.717 test555[368:7303] audio data sent to audio engine...