本文来讲述在Android中使用AudioRecord和AudioTrack来进行音频录制播放,并使用speex进行音频压缩编码,再采用TCP传输思路和方法。
1 开篇
2 相关知识
3 多线程音频读取思路
android-recorder项目中的两个线程处理方法非常好。一个线程启动录制,然后循环调用Read方法,并将数据压倒另外一个线程的数据处理队列中,在另外一个线程的run里面,循环从队列里面取出一条,并写入到文件里面。代码示例如下:
线程1
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
public
void run
(
)
{
PcmWriter pcmWriter = new PcmWriter ( ) ; pcmWriter. init ( ) ; Thread writerThread = new Thread (pcmWriter ) ; pcmWriter. setRecording ( true ) ; writerThread. start ( ) ; synchronized (mutex ) { while ( ! this. isRecording ) { try { mutex. wait ( ) ; } catch ( InterruptedException e ) { throw new IllegalStateException ( "Wait() interrupted!", e ) ; } } } android. os. Process . setThreadPriority (android. os. Process. THREAD_PRIORITY_URGENT_AUDIO ) ; int bufferRead = 0 ; int bufferSize = AudioRecord. getMinBufferSize (frequency, AudioFormat. CHANNEL_IN_MONO, audioEncoding ) ; short [ ] tempBuffer = new short [bufferSize ] ; AudioRecord recordInstance = new AudioRecord ( MediaRecorder. AudioSource. MIC, frequency, AudioFormat. CHANNEL_IN_MONO, audioEncoding, bufferSize ) ; recordInstance. startRecording ( ) ; while ( this. isRecording ) { bufferRead = recordInstance. read (tempBuffer, 0, bufferSize ) ; if (bufferRead == AudioRecord. ERROR_INVALID_OPERATION ) { throw new IllegalStateException ( "read() returned AudioRecord.ERROR_INVALID_OPERATION" ) ; } else if (bufferRead == AudioRecord. ERROR_BAD_VALUE ) { throw new IllegalStateException ( "read() returned AudioRecord.ERROR_BAD_VALUE" ) ; } else if (bufferRead == AudioRecord. ERROR_INVALID_OPERATION ) { throw new IllegalStateException ( "read() returned AudioRecord.ERROR_INVALID_OPERATION" ) ; } pcmWriter. putData (tempBuffer, bufferRead ) ; log. debug ( "put data done!" ) ; } recordInstance. stop ( ) ; pcmWriter. setRecording ( false ) ; } |
线程2
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
public
void run
(
)
{
log. error ( "pcmwriter thread runing" ) ; while ( this. isRecording ( ) ) { if (list. size ( ) > ; 0 ) { rawData = list. remove ( 0 ) ; try { for ( int i = 0 ; i < ; rawData. size ; ++i ) { dataOutputStreamInstance. writeShort (rawData. pcm [i ] ) ; } } catch ( IOException e ) { e. printStackTrace ( ) ; } } else { try { Thread. sleep ( 200 ) ; } catch ( InterruptedException e ) { e. printStackTrace ( ) ; } } } stop ( ) ; } |
上面看着代码很多,实际核心代码为:
1
2 3 4 5 6 7 8 9 10 11 |
short
[
] tempBuffer
=
new
short
[bufferSize
]
;
AudioRecord recordInstance = new AudioRecord ( MediaRecorder. AudioSource. MIC, frequency, AudioFormat. CHANNEL_IN_MONO, audioEncoding, bufferSize ) ; recordInstance. startRecording ( ) ; while ( this. isRecording ) { bufferRead = recordInstance. read (tempBuffer, 0, bufferSize ) ; pcmWriter. putData (tempBuffer, bufferRead ) ; } recordInstance. stop ( ) ; |
这里需要解释一下几个问题:
1
|
public AudioRecord
(
int audioSource,
int sampleRateInHz,
int channelConfig,
int audioFormat,
int bufferSizeInBytes
)
|
Since: API Level 3 Class constructor.
构造函数参数说明:
audioSource 录制源,参考 MediaRecorder.AudioSource的定义
4 播放方法
1
2 3 4 5 6 7 8 9 10 |
AudioTrack mAudioTrack
;
mAudioTrack = new AudioTrack (AudioManager. STREAM_MUSIC,mFrequency,mChannel,mSampBit,minBufSize,AudioTrack. MODE_STREAM ) ; //AudioTrack中有MODE_STATIC和MODE_STREAM两种分类。 //STREAM的意思是由用户在应用程序通过write方式把数据一次一次得写到audiotrack中。 //这个和我们在socket中发送数据一样,应用层从某个地方获取数据,例如通过编解码得到PCM数据,然后write到audiotrack。 //这种方式的坏处就是总是在JAVA层和Native层交互,效率损失较大。 //而STATIC的意思是一开始创建的时候,就把音频数据放到一个固定的buffer,然后直接传给audiotrack, //后续就不用一次次得write了。AudioTrack会自己播放这个buffer中的数据。 //这种方法对于铃声等内存占用较小,延时要求较高的声音来说很适用。 mAudioTrack. play ( ) ; |
5 jni使用speex进行压缩编码
官方文档里面提到了,如果使用C/C++调用speex进行音频压缩及解压的代码。
压缩代码:
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
#include
#include /*The frame size in hardcoded for this sample code but it doesn't have to be*/ #define FRAME_SIZE 160 int main ( int argc , char **argv ) { char *inFile ; FILE *fin ; short in [FRAME_SIZE ] ; float input [FRAME_SIZE ] ; char cbits [ 200 ] ; int nbBytes ; /*Holds the state of the encoder*/ void *state ; /*Holds bits so they can be read and written to by the Speex routines*/ SpeexBits bits ; int i , tmp ; /*Create a new encoder state in narrowband mode*/ state = speex_encoder_init ( & ;speex_nb_mode ) ; /*Set the quality to 8 (15 kbps)*/ tmp = 8 ; speex_encoder_ctl (state , SPEEX_SET_QUALITY , & ;tmp ) ; inFile = argv [ 1 ] ; fin = fopen (inFile , "r" ) ; /*Initialization of the structure that holds the bits*/ speex_bits_init ( & ;bits ) ; while ( 1 ) { /*Read a 16 bits/sample audio frame*/ fread (in , sizeof ( short ) , FRAME_SIZE , fin ) ; if ( feof (fin ) ) break ; /*Copy the 16 bits values to float so Speex can work on them*/ for (i = 0 ;i input [i ] =in [i ] ; /*Flush all the bits in the struct so we can encode a new frame*/ speex_bits_reset ( & ;bits ) ; /*Encode the frame*/ speex_encode (state , input , & ;bits ) ; /*Copy the bits to an array of char that can be written*/ nbBytes = speex_bits_write ( & ;bits , cbits , 200 ) ; /*Write the size of the frame first. This is what sampledec expects but it's likely to be different in your own application*/ fwrite ( & ;nbBytes , sizeof ( int ) , 1 , stdout ) ; /*Write the compressed data*/ fwrite (cbits , 1 , nbBytes , stdout ) ; } /*Destroy the encoder state*/ speex_encoder_destroy (state ) ; /*Destroy the bit-packing struct*/ speex_bits_destroy ( & ;bits ) ; fclose (fin ) ; return 0 ; } |
解压代码:
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
#include
#include /*The frame size in hardcoded for this sample code but it doesn't have to be*/ #define FRAME_SIZE 160 int main ( int argc , char **argv ) { char *outFile ; FILE *fout ; /*Holds the audio that will be written to file (16 bits per sample)*/ short out [FRAME_SIZE ] ; /*Speex handle samples as float, so we need an array of floats*/ float output [FRAME_SIZE ] ; char cbits [ 200 ] ; int nbBytes ; /*Holds the state of the decoder*/ void *state ; /*Holds bits so they can be read and written to by the Speex routines*/ SpeexBits bits ; int i , tmp ; /*Create a new decoder state in narrowband mode*/ state = speex_decoder_init ( & ;speex_nb_mode ) ; /*Set the perceptual enhancement on*/ tmp = 1 ; speex_decoder_ctl (state , SPEEX_SET_ENH , & ;tmp ) ; outFile = argv [ 1 ] ; fout = fopen (outFile , "w" ) ; /*Initialization of the structure that holds the bits*/ speex_bits_init ( & ;bits ) ; while ( 1 ) { /*Read the size encoded by sampleenc, this part will likely be different in your application*/ fread ( & ;nbBytes , sizeof ( int ) , 1 , stdin ) ; fprintf (stderr , "nbBytes: %dn" , nbBytes ) ; if ( feof (stdin ) ) break ; /*Read the "packet" encoded by sampleenc*/ fread (cbits , 1 , nbBytes , stdin ) ; /*Copy the data into the bit-stream struct*/ speex_bits_read_from ( & ;bits , cbits , nbBytes ) ; /*Decode the data*/ speex_decode (state , & ;bits , output ) ; /*Copy from float to short (16 bits) for output*/ for (i = 0 ;i out [i ] =output [i ] ; /*Write the decoded audio to file*/ fwrite (out , sizeof ( short ) , FRAME_SIZE , fout ) ; } /*Destroy the decoder state*/ speex_decoder_destroy (state ) ; /*Destroy the bit-stream truct*/ speex_bits_destroy ( & ;bits ) ; fclose (fout ) ; return 0 ; } |
通过上面的范例,我们就可以通过JNI的方式包含speex库,并撰写相应的android.mk文件,然后写JNI的encode和decode代码了。
下面的代码摘自android-recoder-6
[code lang="c"]
#include
#include
#include
#include
static int codec_open = 0;
static int dec_frame_size;
static int enc_frame_size;
static SpeexBits ebits, dbits;
void *enc_state;
void *dec_state;
static JavaVM *gJavaVM;
//打开编码函数
extern "C"
JNIEXPORT jint JNICALL Java_com_ryong21_encode_Speex_open
(JNIEnv *env, jobject obj, jint compression) {
int tmp;
if (codec_open++ != 0)
return (jint)0;
speex_bits_init(&ebits);
speex_bits_init(&dbits);
enc_state = speex_encoder_init(&speex_nb_mode);
dec_state = speex_decoder_init(&speex_nb_mode);
tmp = compression;
speex_encoder_ctl(enc_state, SPEEX_SET_QUALITY, &tmp);
speex_encoder_ctl(enc_state, SPEEX_GET_FRAME_SIZE, &enc_frame_size);
speex_decoder_ctl(dec_state, SPEEX_GET_FRAME_SIZE, &dec_frame_size);
return (jint)0;
}
//编码函数实现
extern "C"
JNIEXPORT jint JNICALL Java_com_ryong21_encode_Speex_encode
(JNIEnv *env, jobject obj, jshortArray lin, jint offset, jbyteArray encoded, jint size) {
jshort buffer[enc_frame_size];
jbyte output_buffer[enc_frame_size];
int nsamples = (size-1)/enc_frame_size + 1;
int i, tot_bytes = 0;
if (!codec_open)
return 0;
speex_bits_reset(&ebits);
for (i = 0; i < nsamples; i++) { env->GetShortArrayRegion(lin, offset + i*enc_frame_size, enc_frame_size, buffer);
speex_encode_int(enc_state, buffer, &ebits);
}
//env->GetShortArrayRegion(lin, offset, enc_frame_size, buffer);
//speex_encode_int(enc_state, buffer, &ebits);
tot_bytes = speex_bits_write(&ebits, (char *)output_buffer,
enc_frame_size);
env->SetByteArrayRegion(encoded, 0, tot_bytes,
output_buffer);
return (jint)tot_bytes;
}
//解码函数实现
extern "C"
JNIEXPORT jint JNICALL Java_com_ryong21_encode_Speex_decode
(JNIEnv *env, jobject obj, jbyteArray encoded, jshortArray lin, jint size) {
jbyte buffer[dec_frame_size];
jshort output_buffer[dec_frame_size];
jsize encoded_length = size;
if (!codec_open)
return 0;
env->GetByteArrayRegion(encoded, 0, encoded_length, buffer);
speex_bits_read_from(&dbits, (char *)buffer, encoded_length);
speex_decode_int(dec_state, &dbits, output_buffer);
env->SetShortArrayRegion(lin, 0, dec_frame_size,
output_buffer);
return (jint)dec_frame_size;
}
//获取帧大小
extern "C"
JNIEXPORT jint JNICALL Java_com_ryong21_encode_Speex_getFrameSize
(JNIEnv *env, jobject obj) {
if (!codec_open)
return 0;
return (jint)enc_frame_size;
}
//关闭解码器
extern "C"
JNIEXPORT void JNICALL Java_com_ryong21_encode_Speex_close
(JNIEnv *env, jobject obj) {
if (--codec_open != 0)
return;
speex_bits_destroy(&ebits);
speex_bits_destroy(&dbits);
speex_decoder_destroy(dec_state);
speex_encoder_destroy(enc_state);
}
[/code]
java对jni的导入实现为:
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
class Speex
{
/* quality * 1 : 4kbps (very noticeable artifacts, usually intelligible) * 2 : 6kbps (very noticeable artifacts, good intelligibility) * 4 : 8kbps (noticeable artifacts sometimes) * 6 : 11kpbs (artifacts usually only noticeable with headphones) * 8 : 15kbps (artifacts not usually noticeable) */ private static final int DEFAULT_COMPRESSION = 8 ; private Logger log = LoggerFactory. getLogger (Speex. class ) ; Speex ( ) { } public void init ( ) { load ( ) ; open (DEFAULT_COMPRESSION ) ; log. info ( "speex opened" ) ; } private void load ( ) { try { System. loadLibrary ( "speex" ) ; } catch ( Throwable e ) { e. printStackTrace ( ) ; } } public native int open ( int compression ) ; public native int getFrameSize ( ) ; public native int decode ( byte encoded [ ], short lin [ ], int size ) ; public native int encode ( short lin [ ], int offset, byte encoded [ ], int size ) ; public native void close ( ) ; } |
然后我们就可以在开发代码中使用了。
6 TCP传输
在TCP传输端,可以采取本博客之前的Server端框架,参考 Android多客户端TCP Server框架代码 或者 Android中TCP Server的模型代码
7 接受端还原并播放
播放的方式,我们可以使用AudioTrack或者其他的方式。代码可以参考第四部分。
8 尾声
本博客整体的代码是从android-recorder中学习而来的,一些实现也是他的摘录,算是对音频方面的总结吧。在这里感谢原开源作者。另外对于网上音频聊天的实现,一般都通过服务器中转,如red5,实现方面有的UDP,有的地方TCP。见仁见智了,同时也看具体的业务逻辑而定。
9 代码下载
代码我附上google上面的最新代码,值得阅读。虽然在代码里面他将音频转为flv格式播放,但不影响我们自主封装。
android-recorder-5.1
10 参考文章
1 http://www.eoeandroid.com/thread-101317-1-1.html
2 http://code.google.com/p/android-recorder/
3 http://code.google.com/p/spydroid-ipcamera/
4 https://github.com/pdwryst/SoundSwap/tree/c87e7a2d1c63589f928e653f467c6801bbbb1d20
5 https://github.com/RaabsIn513/RecordAudio
6 https://github.com/trashmaxx/AudioRecorder
7 http://hi.baidu.com/lzhts/blog/item/6773d3ca165b7a49f31fe744.html
8 http://blog.csdn.net/hellogv/article/details/6026455
9 http://www.speex.org/
10 http://www.eoeandroid.com/thread-70014-1-1.html