Android 语音遥控器的整体分析

今天蓝牙遥控器的导入终于完成了，分别梳理和记录一下部分语音和蓝牙相关的知识，首先是主机端上层语音部分：

一、应用层使用MediaRecorder的过程（应用层）

1.创建一个MediaRecorder

 mRecorder = new MediaRecorder();

2.设置录音来源

 mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);

3.设置录音比特率

 mRemainingTimeCalculator.setBitRate(SoundRecorder.BITRATE_3GPP);

4.设置采样率

mRecorder.setAudioSamplingRate(highQuality ? 44100 : 22050);

5.设置音频输出格式

  mRecorder.setOutputFormat(outputfileformat);

6.设置音频编码格式

 mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);

7.设置音频输出文件路径

mRecorder.setOutputFile(path);

8.设置异常监听

 mRecorder.setOnErrorListener(this);

9.调用prepare做录音准备工作

mRecorder.prepare();

10.正式开始录音

 mRecorder.start();

11.开始录音后可以调用WakeLock，方式屏幕关闭

mWakeLock.acquire();

12.可以通过广播改变UI状态

sendStateBroadcast();

二、应用层使用MediaRecorder的分析（框架层）

1.mRecorder.setOutputFormat(outputfileformat);中的outputfileformat是MediaRecorder.OutputFormat类型，新的音频格式可以在这里追加

    /**
     * Defines the output format. These constants are used with
     * {@link MediaRecorder#setOutputFormat(int)}.
     */
    public final class OutputFormat {
      /* Do not change these values without updating their counterparts
       * in include/media/mediarecorder.h!
       */
        private OutputFormat() {}
        public static final int DEFAULT = 0;
        /** 3GPP media file format*/
        public static final int THREE_GPP = 1;
        /** MPEG4 media file format*/
        public static final int MPEG_4 = 2;

        /** The following formats are audio only .aac or .amr formats */

        /**
         * AMR NB file format
         * @deprecated  Deprecated in favor of MediaRecorder.OutputFormat.AMR_NB
         */
        public static final int RAW_AMR = 3;

        /** AMR NB file format */
        public static final int AMR_NB = 3;

        /** AMR WB file format */
        public static final int AMR_WB = 4;

        /** @hide AAC ADIF file format */
        public static final int AAC_ADIF = 5;

        /** AAC ADTS file format */
        public static final int AAC_ADTS = 6;

        /** @hide Stream over a socket, limited to a single stream */
        public static final int OUTPUT_FORMAT_RTP_AVP = 7;

        /** @hide H.264/AAC data encapsulated in MPEG2/TS */
        public static final int OUTPUT_FORMAT_MPEG2TS = 8;
    };

2.   mRecorder.setAudioSamplingRate(highQuality ? 44100 : 22050);是设置音频的采样频率在prepare（）之前调用，因为在prepare中会对采样频率的可行性做检测。采样频率和音频的编码格式有关，比如AAC 编码的介于8k到96kHZ，AMRNB编码的为8kHZ，AMRWB为16kHZ，根据实际需要来设置。

   /**
     * Sets the audio sampling rate for recording. Call this method before prepare().
     * Prepare() may perform additional checks on the parameter to make sure whether
     * the specified audio sampling rate is applicable. The sampling rate really depends
     * on the format for the audio recording, as well as the capabilities of the platform.
     * For instance, the sampling rate supported by AAC audio coding standard ranges
     * from 8 to 96 kHz, the sampling rate supported by AMRNB is 8kHz, and the sampling
     * rate supported by AMRWB is 16kHz. Please consult with the related audio coding
     * standard for the supported audio sampling rate.
     *
     * @param samplingRate the sampling rate for audio in samples per second.
     */
    public void setAudioSamplingRate(int samplingRate) {
        if (samplingRate <= 0) {
            throw new IllegalArgumentException("Audio sampling rate is not positive");
        }
        setParameter("audio-param-sampling-rate=" + samplingRate);
    }

3. mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);就是设置前面设置采样频率时提到的编码格式了。这里编码格式定义在MediaRecorder.AudioEncoder这个内部类中，AudioDecoder也是定义在这里

   /**
     * Defines the audio encoding. These constants are used with
     * {@link MediaRecorder#setAudioEncoder(int)}.
     */
    public final class AudioEncoder {
      /* Do not change these values without updating their counterparts
       * in include/media/mediarecorder.h!
       */
        private AudioEncoder() {}
        public static final int DEFAULT = 0;
        /** AMR (Narrowband) audio codec */
        public static final int AMR_NB = 1;
        /** AMR (Wideband) audio codec */
        public static final int AMR_WB = 2;
        /** AAC Low Complexity (AAC-LC) audio codec */
        public static final int AAC = 3;
        /** High Efficiency AAC (HE-AAC) audio codec */
        public static final int HE_AAC = 4;
        /** Enhanced Low Delay AAC (AAC-ELD) audio codec */
        public static final int AAC_ELD = 5;
    }

    /**
     * Defines the video encoding. These constants are used with
     * {@link MediaRecorder#setVideoEncoder(int)}.
     */
    public final class VideoEncoder {
      /* Do not change these values without updating their counterparts
       * in include/media/mediarecorder.h!
       */
        private VideoEncoder() {}
        public static final int DEFAULT = 0;
        public static final int H263 = 1;
        public static final int H264 = 2;
        public static final int MPEG_4_SP = 3;
    }

4.  mRecorder.prepare();中又做了什么？这里是调用jni函数去实现了。前面说过可以自定义编码格式，

    /**
     * Prepares the recorder to begin capturing and encoding data. This method
     * must be called after setting up the desired audio and video sources,
     * encoders, file format, etc., but before start().
     *
     * @throws IllegalStateException if it is called after
     * start() or before setOutputFormat().
     * @throws IOException if prepare fails otherwise.
     */
    public void prepare() throws IllegalStateException, IOException
    {
        _prepare();
    }

jni部分的实现是在\frameworks\av\media\libmedia中

status_t MediaRecorder::prepare()
{
    ALOGV("prepare");
    if (mMediaRecorder == NULL) {
        ALOGE("media recorder is not initialized yet");
        return INVALID_OPERATION;
    }
    if (!(mCurrentState & MEDIA_RECORDER_DATASOURCE_CONFIGURED)) {
        ALOGE("prepare called in an invalid state: %d", mCurrentState);
        return INVALID_OPERATION;
    }
    if (mIsAudioSourceSet != mIsAudioEncoderSet) {
        if (mIsAudioSourceSet) {
            ALOGE("audio source is set, but audio encoder is not set");
        } else {  // must not happen, since setAudioEncoder checks this already
            ALOGE("audio encoder is set, but audio source is not set");
        }
        return INVALID_OPERATION;
    }

    if (mIsVideoSourceSet != mIsVideoEncoderSet) {
        if (mIsVideoSourceSet) {
            ALOGE("video source is set, but video encoder is not set");
        } else {  // must not happen, since setVideoEncoder checks this already
            ALOGE("video encoder is set, but video source is not set");
        }
        return INVALID_OPERATION;
    }

    status_t ret = mMediaRecorder->prepare();
    if (OK != ret) {
        ALOGE("prepare failed: %d", ret);
        mCurrentState = MEDIA_RECORDER_ERROR;
        return ret;
    }
    mCurrentState = MEDIA_RECORDER_PREPARED;
    return ret;
}

主要是调用mMediaRecorder去实现
这个mMediaRecorder是个Mediarecorder的成员变量，是个强指针（注意，不是智能指针）

sp<IMediaRecorder>          mMediaRecorder;

5.现在要看这个IMediaRecorder类型的 mMediaRecorder->prepare()调用到了哪里，我们需要弄清楚MediaRecorder这一部分的类图：

 

显然是调用到了MediaRecorderClient.prepare()中

status_t MediaRecorderClient::prepare()
{
    ALOGV("prepare");
    Mutex::Autolock lock(mLock);
    if (mRecorder == NULL) {
        ALOGE("recorder is not initialized");
        return NO_INIT;
    }
    return mRecorder->prepare();
}

这里的mRecorder是一个MediaRecorderBase类型的成员，整个Android系统中目前还只有一个派生类StagefightRecorder。

所以根据继承关系知道最终是调用到了StagefightRecorder.prepare()和StagefrightRecorder.start()中，这里就正式开始录音咯！！可以看到前面设置的音频类型在这里会用到

status_t StagefrightRecorder::start() {
    // Get UID here for permission checking
    mClientUid = IPCThreadState::self()->getCallingUid();
    if (mWriter != NULL) {
        ALOGE("File writer is not avaialble");
        return UNKNOWN_ERROR;
    }

    status_t status = OK;

    switch (mOutputFormat) {
        case OUTPUT_FORMAT_DEFAULT:
        case OUTPUT_FORMAT_THREE_GPP:
        case OUTPUT_FORMAT_MPEG_4:
            status = startMPEG4Recording();
            break;

        case OUTPUT_FORMAT_AMR_NB:
        case OUTPUT_FORMAT_AMR_WB:
            status = startAMRRecording();
            break;

        case OUTPUT_FORMAT_AAC_ADIF:
        case OUTPUT_FORMAT_AAC_ADTS:
            status = startAACRecording();
            break;

        case OUTPUT_FORMAT_RTP_AVP:
            status = startRTPRecording();
            break;

        case OUTPUT_FORMAT_MPEG2TS:
            status = startMPEG2TSRecording();
            break;

          default:
            ALOGE("Unsupported output file format: %d", mOutputFormat);
            status = UNKNOWN_ERROR;
            break;
    }

    if ((status == OK) && (!mStarted)) {
        mStarted = true;

        uint32_t params = IMediaPlayerService::kBatteryDataCodecStarted;
        if (mAudioSource != AUDIO_SOURCE_CNT) {
            params |= IMediaPlayerService::kBatteryDataTrackAudio;
        }
        if (mVideoSource != VIDEO_SOURCE_LIST_END) {
            params |= IMediaPlayerService::kBatteryDataTrackVideo;
        }

        addBatteryData(params);
    }

    return status;
}

现在分析下startAMRRecording（）;的具体实现

status_t StagefrightRecorder::startAMRRecording() {
    CHECK(mOutputFormat == OUTPUT_FORMAT_AMR_NB ||
          mOutputFormat == OUTPUT_FORMAT_AMR_WB);

    if (mOutputFormat == OUTPUT_FORMAT_AMR_NB) {
        if (mAudioEncoder != AUDIO_ENCODER_DEFAULT &&
            mAudioEncoder != AUDIO_ENCODER_AMR_NB) {
            ALOGE("Invalid encoder %d used for AMRNB recording",
                    mAudioEncoder);
            return BAD_VALUE;
        }
    } else {  // mOutputFormat must be OUTPUT_FORMAT_AMR_WB
        if (mAudioEncoder != AUDIO_ENCODER_AMR_WB) {
            ALOGE("Invlaid encoder %d used for AMRWB recording",
                    mAudioEncoder);
            return BAD_VALUE;
        }
    }

    mWriter = new AMRWriter(mOutputFd);
    status_t status = startRawAudioRecording();
    if (status != OK) {
        mWriter.clear();
        mWriter = NULL;
    }
    return status;
}

正在干活的是这里：

status_t StagefrightRecorder::startRawAudioRecording() {
    if (mAudioSource >= AUDIO_SOURCE_CNT) {
        ALOGE("Invalid audio source: %d", mAudioSource);
        return BAD_VALUE;
    }

    status_t status = BAD_VALUE;
    if (OK != (status = checkAudioEncoderCapabilities())) {
        return status;
    }

    sp<MediaSource> audioEncoder = createAudioSource();
    if (audioEncoder == NULL) {
        return UNKNOWN_ERROR;
    }

    CHECK(mWriter != 0);
    mWriter->addSource(audioEncoder);

    if (mMaxFileDurationUs != 0) {
        mWriter->setMaxFileDuration(mMaxFileDurationUs);
    }
    if (mMaxFileSizeBytes != 0) {
        mWriter->setMaxFileSize(mMaxFileSizeBytes);
    }
    mWriter->setListener(mListener);
    mWriter->start();

    return OK;
}

6.startRawAudioRecording（)

(1)首先检测波特率采样率和通道数是否符合要求

status_t StagefrightRecorder::checkAudioEncoderCapabilities() {
    clipAudioBitRate();
    clipAudioSampleRate();
    clipNumberOfAudioChannels();
    return OK;
}

（2）然后设置一个MediaWriter的对象，真正录音的对象就是这个mWriter

可以分析其中一个派生类的具体实现来看怎么录音的。
正在的录音是在这里：

status_t AMRWriter::threadFunc() {
    mEstimatedDurationUs = 0;
    mEstimatedSizeBytes = 0;
    bool stoppedPrematurely = true;
    int64_t previousPausedDurationUs = 0;
    int64_t maxTimestampUs = 0;
    status_t err = OK;

    prctl(PR_SET_NAME, (unsigned long)"AMRWriter", 0, 0, 0);
    while (!mDone) {
        MediaBuffer *buffer;
        err = mSource->read(&buffer);

        if (err != OK) {
            break;
        }

        if (mPaused) {
            buffer->release();
            buffer = NULL;
            continue;
        }

        mEstimatedSizeBytes += buffer->range_length();
        if (exceedsFileSizeLimit()) {
            buffer->release();
            buffer = NULL;
            notify(MEDIA_RECORDER_EVENT_INFO, MEDIA_RECORDER_INFO_MAX_FILESIZE_REACHED, 0);
            break;
        }

        int64_t timestampUs;
        CHECK(buffer->meta_data()->findInt64(kKeyTime, ×tampUs));
        if (timestampUs > mEstimatedDurationUs) {
            mEstimatedDurationUs = timestampUs;
        }
        if (mResumed) {
            previousPausedDurationUs += (timestampUs - maxTimestampUs - 20000);
            mResumed = false;
        }
        timestampUs -= previousPausedDurationUs;
        ALOGV("time stamp: %lld, previous paused duration: %lld",
                timestampUs, previousPausedDurationUs);
        if (timestampUs > maxTimestampUs) {
            maxTimestampUs = timestampUs;
        }

        if (exceedsFileDurationLimit()) {
            buffer->release();
            buffer = NULL;
            notify(MEDIA_RECORDER_EVENT_INFO, MEDIA_RECORDER_INFO_MAX_DURATION_REACHED, 0);
            break;
        }
        ssize_t n = write(mFd,
                        (const uint8_t *)buffer->data() + buffer->range_offset(),
                        buffer->range_length());

        if (n < (ssize_t)buffer->range_length()) {
            buffer->release();
            buffer = NULL;
            err = ERROR_IO;
            break;
        }

        if (err != OK) {
            break;
        }

        if (stoppedPrematurely) {
            stoppedPrematurely = false;
        }

        buffer->release();
        buffer = NULL;
    }

    if ((err == OK || err == ERROR_END_OF_STREAM) && stoppedPrematurely) {
        err = ERROR_MALFORMED;
    }

    close(mFd);
    mFd = -1;
    mReachedEOS = true;
    if (err == ERROR_END_OF_STREAM) {
        return OK;
    }
    return err;
}

我们关注的是两个点：

1.MediaBuffer *buffer;音频最后都是来源于这里：err = mSource->read(&buffer);。分析音频格式，可以从这个buffer入手。

2. mFd = open(filename, O_CREAT | O_LARGEFILE | O_TRUNC | O_RDWR, S_IRUSR | S_IWUSR);

最后音频内容是保存在这个里面了。

 

后面会针对这个buffer来分析Android 的音频底层实现机制并添加自己的音频编解码。