1. 几个问题:
Q:从Session拿到的音频数据的格式?codec?
A:通过一个C语言库方法readAudioData(...) 2012/03/12
Q:Session如何提供多路音频的?
A: readAudioData(...)中有参数可以选择
Q:captured audio data如何传给Session?
A:通过C库sendAudioData(...)发送。
Q:Audio Queue只支持几种codec(AAC, MP3, ALAC),播放前是否需要转码?
A:Audio Queue支持LPCM,这也是Audio Engine解码后的格式,无需再转码了 2012/03/12
2. Core Audio Overview
* AudioStreamBasicDescription - an important universal data type in Core Audio
ASBD
struct AudioStreamBasicDescription {
Float64 mSampleRate;
UInt32 mFormatID;
UInt32 mFormatFlags;
UInt32 mBytesPerPacket;
UInt32 mFramesPerPacket;
UInt32 mBytesPerFrame;
UInt32 mChannelsPerFrame;
UInt32 mBitsPerChannel;
UInt32 mReserved;
};
* packets
Smallest meaningful set of frames ( may have 2 channels data for a specific time ) for a given data audio format.
* 3 types of audio data format
CBR ( constant bit rate ) - PCM
VBR ( variable bit rate ) - AAC, MP3
VFR ( variable frame rate ) - NA
* Data Format Conversions
3 types available:
# Decoding ( like AAC to linear PCM)
# Coding ( like linear PCM to AAC )
# converting between different variants of linear PCM ( e.g, 16-bit signed integer linear PCM to 8.24 fixed-point linear PCM)
* AVAudioPlayer
not for audio data captured from network.
* Audio Queue Services
Via Audio Queue, you could
# use hardware recording and playback devices without knowledges of hardware interfaces
# use complex codec without the relevant knowledges
It is a high-level interface with advanced features.
* Audio Unit 4 iOS
# features:
low-latency input/ouput
DSP
# format: 8.24-bit fixed point linear PCM
# examples
Multichannel mixer unit
Converter unit
* Codecs 4 iOS
# unrestricted Playback
iLBC ( Internet Low Bitrate Codec, also speech codec )
IMA/ADPCM ( also known as IMA-4)
Linear PCM
uLaw and aLaw
# restricted Playback
AAC
Apple Lossless
MP3
# Recording
Apple Lossless
iLBC
IMA/ADPCM
Linear PCM
uLaw and aLaw
Note: no MP3 and AAC recording support due to high CPU overhead and consequent battery drain.
* Audio Processing Graph
3. Sample code: SpeakHere
Q: Why SpeakerHereController::awakeFromNib is called ?
#0 -[SpeakHereController awakeFromNib] (self=0x91182c0, _cmd=0x6dc7a2) at /Users/frankzou/study/xcode-doc/SpeakHere/Classes/SpeakHereController.mm:254
#1 0x004be4f0 in -[UINib instantiateWithOwner:options:] ()
#2 0x004c0081 in -[NSBundle(UINSBundleAdditions) loadNibNamed:owner:options:] ()
#3 0x00378a94 in -[UIViewController _loadViewFromNibNamed:bundle:] ()
#4 0x00376709 in -[UIViewController loadView] ()
#5 0x003765e3 in -[UIViewController view] ()
#6 0x000027a1 in -[SpeakHereAppDelegate applicationDidFinishLaunching:] (self=0x4e27070, _cmd=0x6d926b, application=0x4b11490) at /Users/frankzou/study/xcode-doc/SpeakHere/Classes/SpeakHereAppDelegate.m:62