Android音视频开发入门(五)

任务目标

使用MediaExtractor和MediaMuxer API解析和封装Mp4文件

MediaExtractor

这两个API相对来说内容很少,具体请参考MediaExtractor官方文档。文档给了一个简单的示例,如下:

MediaExtractor extractor = new MediaExtractor();
 extractor.setDataSource(...);
 int numTracks = extractor.getTrackCount();
 for (int i = 0; i < numTracks; ++i) {
   MediaFormat format = extractor.getTrackFormat(i);
   String mime = format.getString(MediaFormat.KEY_MIME);
   if (weAreInterestedInThisTrack) {
     extractor.selectTrack(i);
   }
 }
 ByteBuffer inputBuffer = ByteBuffer.allocate(...)
 while (extractor.readSampleData(inputBuffer, ...) >= 0) {
   int trackIndex = extractor.getSampleTrackIndex();
   long presentationTimeUs = extractor.getSampleTime();
   ...
   extractor.advance();
 }

 extractor.release();
 extractor = null;

简单介绍下方法

  1. setDataSource(…) 设置输入文件的路径,可以是本地也可以是网络文件,如果是网络文件需要网络权限
  2. getTrackCount() 获取通道数
  3. getTrackFormat(…) 获取通道格式
  4. selectTrack(…) 选中追踪通道
  5. readSampleData(ByteBuffer byteBuf, int offset) 把指定通道的数据按偏移量读取到ByteBuffer中
  6. getSampleTime() 获取当前时间戳
  7. advance() 读取下一帧数据
  8. release() 释放资源

MediaMuxer

MediaMuxer官方文档内容也比较简单,现在支持Mp4、Webp、3GP输出格式,示例如下:

MediaMuxer muxer = new MediaMuxer("temp.mp4", OutputFormat.MUXER_OUTPUT_MPEG_4);
 // More often, the MediaFormat will be retrieved from MediaCodec.getOutputFormat()
 // or MediaExtractor.getTrackFormat().
 MediaFormat audioFormat = new MediaFormat(...);
 MediaFormat videoFormat = new MediaFormat(...);
 int audioTrackIndex = muxer.addTrack(audioFormat);
 int videoTrackIndex = muxer.addTrack(videoFormat);
 ByteBuffer inputBuffer = ByteBuffer.allocate(bufferSize);
 boolean finished = false;
 BufferInfo bufferInfo = new BufferInfo();

 muxer.start();
 while(!finished) {
   // getInputBuffer() will fill the inputBuffer with one frame of encoded
   // sample from either MediaCodec or MediaExtractor, set isAudioSample to
   // true when the sample is audio data, set up all the fields of bufferInfo,
   // and return true if there are no more samples.
   finished = getInputBuffer(inputBuffer, isAudioSample, bufferInfo);
   if (!finished) {
     int currentTrackIndex = isAudioSample ? audioTrackIndex : videoTrackIndex;
     muxer.writeSampleData(currentTrackIndex, inputBuffer, bufferInfo);
   }
 };
 muxer.stop();
 muxer.release();
  1. MediaMuxer(String path, int format) 第一个参数是输出路径,第二个是输出格式,支持的格式上面已说明
  2. addTrack(MediaFormat format) 添加通道,本文使用Extractor.getTrackFormat(int index)来获取MediaFormat
  3. start() 开始合成,要在addTrack()之后调用
  4. writeSampleData(int trackIndex, ByteBuffer byteBuf, MediaCodec.BufferInfo bufferInfo) 把ByteBuffer中的数据写入到文件
  5. stop() 停止合成文件
  6. release() 释放资源

抽取视频的示例代码

	/**
     * 抽取视频
     */
    private void extractorVideo() {
        int videoTrackIndex = -1;
        mExtractor = new MediaExtractor();
        try {
            mExtractor.setDataSource(inputPath);
            //获取通道数
            int tractCount = mExtractor.getTrackCount();
            for (int i = 0; i < tractCount; i++) {
                MediaFormat mediaFormat = mExtractor.getTrackFormat(i);
                String mimeType = mediaFormat.getString(MediaFormat.KEY_MIME);
                if (mimeType.startsWith("video/")) {
                    videoTrackIndex = i;
                    break;
                }
            }
            //设置视频通道
            mExtractor.selectTrack(videoTrackIndex);
            //初始化MediaMuxer
            mMuxer = new MediaMuxer(outVideoPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
            //获取MediaFormat
            MediaFormat videoFormat = mExtractor.getTrackFormat(videoTrackIndex);
            //添加通道给muxer
            mMuxer.addTrack(videoFormat);

            ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
            mMuxer.start();
            //获取相邻视频帧的时间间隔
            long videoSampleTime = getSampleTime(mExtractor, byteBuffer, videoTrackIndex);

            MediaCodec.BufferInfo videoInfo = new MediaCodec.BufferInfo();

            while (true) {
                int readSize = mExtractor.readSampleData(byteBuffer, 0);
                if (readSize < 0) {
                    break;
                }
                mExtractor.advance();
                videoInfo.offset = 0;
                videoInfo.size = readSize;
                videoInfo.flags = mExtractor.getSampleFlags();
                videoInfo.presentationTimeUs += videoSampleTime;
                mMuxer.writeSampleData(videoTrackIndex, byteBuffer, videoInfo);
            }
            mMuxer.stop();
            mMuxer.release();
            mExtractor.release();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

我发现有人用帧率计算时间,我也试了下,报错了,可能用帧率计算时间不是通用的;还有抽取视频后发现视频文件不是很完整,但是可以播放,不知道具体什么原因,希望知道的留言告知。 另外试过这篇文章的方式,分离后的视频不能正常播放,不知道是不是我操作不当,但是我直接copy的作者代码没有改动过。

抽取音频的实例代码

音频跟视频的抽取是一样的

	/**
     * 抽取音频
     */
    private void extractorAudio() {
        int audioTrackIndex = -1;
        mExtractor = new MediaExtractor();
        try {
            //设置输入源
            mExtractor.setDataSource(inputPath);
            //获取通道数
            int trackCount = mExtractor.getTrackCount();
            for (int i = 0; i < trackCount; i++) {
                MediaFormat mediaFormat = mExtractor.getTrackFormat(i);
                String mime = mediaFormat.getString(MediaFormat.KEY_MIME);
                if (mime.startsWith("audio/")) {
                    audioTrackIndex = i;
                    break;
                }
            }
            //设置音频通道
            mExtractor.selectTrack(audioTrackIndex);
            MediaFormat audioFormat = mExtractor.getTrackFormat(audioTrackIndex);
            mMuxer = new MediaMuxer(outAudioPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
            //添加通道
            int writeAudio = mMuxer.addTrack(audioFormat);
            ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
            mMuxer.start();

            long audioSampleTime = getSampleTime(mExtractor, byteBuffer, audioTrackIndex);
            MediaCodec.BufferInfo audioInfo = new MediaCodec.BufferInfo();

            while (true) {
                int readSize = mExtractor.readSampleData(byteBuffer, 0);
                if (readSize < 0) {
                    break;
                }
                mExtractor.advance();
                audioInfo.offset = 0;
                audioInfo.size = readSize;
                audioInfo.flags = mExtractor.getSampleFlags();
                audioInfo.presentationTimeUs += audioSampleTime;
                mMuxer.writeSampleData(writeAudio, byteBuffer, audioInfo);
            }
            mMuxer.stop();
            mMuxer.release();
            mExtractor.release();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

用抽取的音视频重新合成一个MP4文件

	/**
     * 合成视频音频
     */
    public void muxerVideoAudio(View view) {
        mMuxerStatus.setText("开始合成");
        MediaExtractor videoExtractor = new MediaExtractor();
        MediaExtractor audioExtractor = new MediaExtractor();
        try {
            videoExtractor.setDataSource(outVideoPath);
            audioExtractor.setDataSource(outAudioPath);
            //获取要追踪的通道
            int videoTrackIndex = getTrack(videoExtractor, "video/");
            int audioTrackIndex = getTrack(audioExtractor, "audio/");
            //选中通道
            videoExtractor.selectTrack(videoTrackIndex);
            audioExtractor.selectTrack(audioTrackIndex);

            MediaCodec.BufferInfo videoInfo = new MediaCodec.BufferInfo();
            MediaCodec.BufferInfo audioInfo = new MediaCodec.BufferInfo();
            //创建缓冲区
            ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
            mMuxer = new MediaMuxer("/storage/emulated/0/merge.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
            //添加通道
            int writeVideoTrack = mMuxer.addTrack(videoExtractor.getTrackFormat(videoTrackIndex));
            int writeAudioTrack = mMuxer.addTrack(audioExtractor.getTrackFormat(audioTrackIndex));
            mMuxer.start();

            long videoSampleTime = getSampleTime(videoExtractor, byteBuffer, videoTrackIndex);

            //顺序写入数据
            while (true) {
                int videoSize = videoExtractor.readSampleData(byteBuffer, 0);
                if (videoSize < 0) {
                    break;
                }
                videoInfo.offset = 0;
                videoInfo.size = videoSize;
                videoInfo.flags = videoExtractor.getSampleFlags();
                videoInfo.presentationTimeUs += videoSampleTime;
                mMuxer.writeSampleData(writeVideoTrack, byteBuffer, videoInfo);
                videoExtractor.advance();
            }

            long audioSampleTime = getSampleTime(audioExtractor, byteBuffer, audioTrackIndex);

            while (true) {
                int audioSize = audioExtractor.readSampleData(byteBuffer, 0);
                if (audioSize < 0) {
                    break;
                }
                audioInfo.offset = 0;
                audioInfo.size = audioSize;
                audioInfo.flags = audioExtractor.getSampleFlags();
                audioInfo.presentationTimeUs += audioSampleTime;
                mMuxer.writeSampleData(writeAudioTrack, byteBuffer, audioInfo);
                audioExtractor.advance();
            }
            SystemClock.sleep(500);
            mMuxer.stop();
            mMuxer.release();
            videoExtractor.release();
            audioExtractor.release();
            mMuxerStatus.setText("合成完毕");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    
	/**
     * 获取要追踪的信道
     * @param extractor {@link MediaExtractor}
     * @return track
     */
    private int getTrack(MediaExtractor extractor, String mimeType) {
        int trackCount = extractor.getTrackCount();
        int trackIndex = -1;
        for (int i = 0; i < trackCount; i++) {
            MediaFormat format = extractor.getTrackFormat(i);
            String mime = format.getString(MediaFormat.KEY_MIME);
            if (mime.startsWith(mimeType)) {
                trackIndex = i;
                break;
            }
        }
        return trackIndex;
    }
    
	//获取每一帧的时间差
    private long getSampleTime(MediaExtractor extractor, ByteBuffer byteBuffer, int trackIndex) {
        extractor.readSampleData(byteBuffer, 0);
        //跳过I帧,要P帧(视频是由个别I帧和很多P帧组成)h264编码中有IBP帧 I为关键帧。
        if (extractor.getSampleFlags() == MediaExtractor.SAMPLE_FLAG_SYNC) {
            extractor.advance();//跳过
        }
        extractor.readSampleData(byteBuffer, 0);
        //得到第一个P帧的时间戳
        long firstPTS = extractor.getSampleTime();
        //下一帧
        extractor.advance();
        extractor.readSampleData(byteBuffer, 0);
        long secondPTS = extractor.getSampleTime();
        long sampleTime = Math.abs(secondPTS - firstPTS);
        // 重新切换此信道,不然上面跳过了3帧,造成前面的帧数模糊
        extractor.unselectTrack(trackIndex);
        extractor.selectTrack(trackIndex);
        return sampleTime;
    }

音视频合成的时候有一点需要注意while循环里有这么一句audioExtractor.advance(),开始我放到if语句后面,运行报错了。可能有一点小问题就会引起java.lang.IllegalStateException: Failed to stop the muxer。报错了不要急,可能就是你的代码跟别人的有一点顺序不太一样。

最后代码已上传GitHub

参考

  1. Android音视频开发-入门(四):使用 MediaExtractor 和 MediaMuxer API 解析和封装 mp4 文件
  2. Android 音视频开发(五):使用 MediaExtractor 和 MediaMuxer API 解析和封装 mp4 文件
发布了48 篇原创文章 · 获赞 55 · 访问量 13万+
展开阅读全文

没有更多推荐了,返回首页

©️2019 CSDN 皮肤主题: 编程工作室 设计师: CSDN官方博客

分享到微信朋友圈

×

扫一扫,手机浏览