音频的裁剪、合成、给MP4文件混合背景音频

学习目标：

音频的裁剪、合成、小例子

学习内容：

一、音频的裁剪
1.程序思路
把一段mp3文件裁剪生成一个新的mp3文件，我们通过对一段视频进行解析出pcm码流，然后输出对应的时间段的pcm视频流，然后在这段视频流前面添加wav头部，生成mp3文件，这样就裁剪完成了。
有关添加wav头部的实现请看下面的链接:
音频之WAV格式编码解析
解析mp3文件代码如下:

MediaCodec mediaCodec = null;
            MediaExtractor mediaExtractor = null;
            File pcmFile = null;
            try {
                mediaExtractor = new MediaExtractor();
                mediaExtractor.setDataSource(this.resPath);
                int audioTrack = findTrack(mediaExtractor, "audio/");
                mediaExtractor.selectTrack(audioTrack);
                mediaExtractor.seekTo(startTime, MediaExtractor.SEEK_TO_CLOSEST_SYNC);
                MediaFormat audioFormat = mediaExtractor.getTrackFormat(audioTrack);
                int maxBufferSize = 100_000;
                if (audioFormat.containsKey(MediaFormat.KEY_MAX_INPUT_SIZE)) {
                    maxBufferSize = audioFormat.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE);
                }
                ByteBuffer buffer = ByteBuffer.allocateDirect(maxBufferSize);
                //初始化解码器
                mediaCodec = MediaCodec.createDecoderByType(audioFormat.getString(MediaFormat.KEY_MIME));
                mediaCodec.configure(audioFormat, null, null, 0);
                mediaCodec.start();
                MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
                int decodeInputIndex, outPutIndex;
                pcmFile = new File(Environment.getExternalStorageDirectory(), "out.pcm");
                pcmChannel = new FileOutputStream(pcmFile).getChannel();
                while (isStart) {
                    decodeInputIndex = mediaCodec.dequeueInputBuffer(100_000);
                    if (decodeInputIndex >= 0) {
                        long sampleTimeUs = mediaExtractor.getSampleTime();
                        if (sampleTimeUs == -1) {
                            break;
                        } else if (sampleTimeUs < startTime) {
                            mediaExtractor.advance();
                            continue;
                        } else if (sampleTimeUs > endTime) {
                            break;
                        }
                        //音频数据放入buffer
                        info.size = mediaExtractor.readSampleData(buffer, 0);
                        info.presentationTimeUs = sampleTimeUs;
                        info.flags = mediaExtractor.getSampleFlags();
                        //放入mediaCodec解析
                        ByteBuffer inputBuffer = mediaCodec.getInputBuffer(decodeInputIndex);
                        byte[] content = new byte[buffer.remaining()];
                        buffer.get(content);
                        inputBuffer.put(content);
                        mediaCodec.queueInputBuffer(decodeInputIndex, 0, info.size, info.presentationTimeUs, info.flags);
                        mediaExtractor.advance();
                    }
                    outPutIndex = mediaCodec.dequeueOutputBuffer(info, 100_000);
                    //读取数据
                    while (outPutIndex >= 0) {
                        ByteBuffer outBuffer = mediaCodec.getOutputBuffer(outPutIndex);
                        pcmChannel.write(outBuffer);
                        mediaCodec.releaseOutputBuffer(outPutIndex, false);
                        outPutIndex = mediaCodec.dequeueOutputBuffer(info, 100_000);
                    }
                }
            } catch (IOException e) {
                e.printStackTrace();
                stopClip();
            }

            try {
                if (pcmChannel != null) {
                    pcmChannel.close();
                }
                if (mediaExtractor != null) {
                    mediaExtractor.release();
                }
                if (mediaCodec != null) {
                    mediaCodec.stop();
                    mediaCodec.release();
                }
            } catch (Exception e) {
                e.printStackTrace();
            } 
            //添加wav头部  
            new PcmToWavUtil(44100,  AudioFormat.CHANNEL_IN_STEREO,
                    2, AudioFormat.ENCODING_PCM_16BIT).pcmToWav(pcmFile.getAbsolutePath()
                    , this.outPath);

具体代码demo如下
demo代码链接

二、音频的合成
1.程序思路
这里我们读取视频里的音频轨道和一段音频，进行合并，当然还是需要添加wav头部，这样就完成了。
合并mp3关键代码如下:

//合成两个pcm文件
public class AudioMixThread {

    String musicPath;
    String videoPath;
    String mixedPath;
    int startTime;
    int endTime;
    int musicVolume;
    int videoVolume;

    public AudioMixThread(String musicPath, String videoPath, String mixedPath, int startTime, int endTime, int musicVolume, int videoVolume) {
        this.musicPath = musicPath;
        this.videoPath = videoPath;
        this.startTime = startTime;
        this.endTime = endTime;
        this.musicVolume = musicVolume;
        this.videoVolume = videoVolume;
        this.mixedPath = mixedPath;
    }

    public void mix() {
        if (startTime < 0 || endTime < 0 || startTime > endTime) return;
        FileInputStream musicInput = null;
        FileInputStream videoInput = null;
        FileOutputStream mixedStream = null;
        try {
            float musicVol = formatVolume(musicVolume);
            float videoVol = formatVolume(videoVolume);
            byte[] musicVolBuffer = new byte[2048];
            byte[] videoVolBuffer = new byte[2048];
            byte[] result = new byte[2048];
            musicInput = new FileInputStream(musicPath);
            videoInput = new FileInputStream(videoPath);
            mixedStream = new FileOutputStream(mixedPath);
            short tempVideo, tempMusic;
            int tempResult;
            boolean endMusic = false, endVideo = false;
            while (!endMusic || !endVideo) {
                if (!endMusic) {
                    endMusic = (musicInput.read(musicVolBuffer) == -1);
                    //音乐的pcm数据  写入到 buffer3
                    System.arraycopy(musicVolBuffer, 0, result, 0, musicVolBuffer.length);
                }
                if (!endVideo) {
                    endVideo = (videoInput.read(videoVolBuffer) == -1);
                    //声音的值  跳过下一个声音的值    一个声音 2 个字节
                    for (int i = 0; i < videoVolBuffer.length; i += 2) {
                        //或运算
                        tempMusic = (short) ((musicVolBuffer[i] & 0xff) | (musicVolBuffer[i + 1] & 0xff) << 8);
                        tempVideo = (short) ((videoVolBuffer[i] & 0xff) | (videoVolBuffer[i + 1] & 0xff) << 8);
                        tempResult = (int) (tempMusic * musicVol + tempVideo * videoVol);//音乐和 视频声音 各占一半
                        if (tempResult > Short.MAX_VALUE) {
                            tempResult = Short.MAX_VALUE;
                        } else if (tempResult < Short.MIN_VALUE) {
                            tempResult = Short.MIN_VALUE;
                        }
                        result[i] = (byte) (tempResult & 0xFF);
                        result[i + 1] = (byte) ((tempResult >>> 8) & 0xFF);
                    }
                    mixedStream.write(result);
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                if (musicInput != null) {
                    musicInput.close();
                }
                if (videoInput != null) {
                    videoInput.close();
                }
                if (mixedStream != null) {
                    mixedStream.close();
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

    //音量格式化&提高精度
    private static float formatVolume(int volume) {
        return volume / 100f * 1;
    }
}

具体代码demo如下
Demo链接

三、给MP4文件混合背景音频
1.程序思路
这里我们读取视频里的音频轨道和一段mp3音频，进行合并输出到新的音频轨道，读取视频里的视频轨道输出到新的视频轨道，这样就完成了。
关键代码如下:

public class MusicProcess {
    private static int TIMEOUT = 1000;

    public static void mixAudioTrack(
            final Activity activity,
            final String musicPath,//音频路径
            final String videoPath,//视频路径
            final String mixedPath,//生成结果路径
            final Integer startTimeUs,//截取开始时间
            final Integer endTimeUs,//截取结束时间
            int musicVolume,//音频声音大小
            int videoVolume,//视频声音大小l
            final String mixedMusicAndAudioPath//新生成的视频
    ) throws Exception {

        ExecutorService service = Executors.newFixedThreadPool(3);
        final CountDownLatch latch = new CountDownLatch(2);
        final File videoPcmFile = new File(Environment.getExternalStorageDirectory(), "video.pcm");
        final File musicPcmFile = new File(Environment.getExternalStorageDirectory(), "music.pcm");
        final File mixPcmFile = new File(Environment.getExternalStorageDirectory(), "mixed.pcm");
        //解析pcm
        service.execute(() -> {
            try {
                decodeToPCM(videoPath, videoPcmFile.getAbsolutePath(), startTimeUs, endTimeUs);
            } catch (Exception e) {
                e.printStackTrace();
            } finally {
                latch.countDown();
            }
        });
        //解析pcm
        service.execute(() -> {
            try {
                decodeToPCM(musicPath, musicPcmFile.getAbsolutePath(), startTimeUs, endTimeUs);
            } catch (Exception e) {
                e.printStackTrace();
            } finally {
                latch.countDown();
            }
        });
        latch.await();
        service.execute(() -> {
            //开始混合两个pcm
            AudioMixThread thread = new AudioMixThread(musicPcmFile.getAbsolutePath(), videoPcmFile.getAbsolutePath(), mixPcmFile.getAbsolutePath(), startTimeUs, endTimeUs, musicVolume, videoVolume);
            thread.mix();
            //生成混合后的mp3文件
            new PcmToWavUtil(44100, AudioFormat.CHANNEL_IN_STEREO,
                    2, AudioFormat.ENCODING_PCM_16BIT).pcmToWav(mixPcmFile.getAbsolutePath()
                    , mixedPath);
            try {
                mixVideoAndMusic(videoPath, mixedMusicAndAudioPath, startTimeUs, endTimeUs, new File(mixedPath));
                ((MainActivity)activity).startPlay();
            } catch (IOException e) {
                e.printStackTrace();
            }
            Log.d("oicq", "mix sucessful!");
        });
    }

    //    MP3 截取并且输出  pcm
    @SuppressLint("WrongConstant")
    public static void decodeToPCM(String musicPath, String outPath, int startTime, int endTime) throws Exception {
        if (endTime < startTime) {
            return;
        }
//    MP3  （zip  rar    ） ----> aac   封装个事 1   编码格式
//        jie  MediaExtractor = 360 解压 工具
        MediaExtractor mediaExtractor = new MediaExtractor();

        mediaExtractor.setDataSource(musicPath);
        int audioTrack = selectTrack(mediaExtractor, "audio/");

        mediaExtractor.selectTrack(audioTrack);
// 视频 和音频
        mediaExtractor.seekTo(startTime, MediaExtractor.SEEK_TO_CLOSEST_SYNC);
// 轨道信息  都记录 编码器
        MediaFormat oriAudioFormat = mediaExtractor.getTrackFormat(audioTrack);
        int maxBufferSize;
        if (oriAudioFormat.containsKey(MediaFormat.KEY_MAX_INPUT_SIZE)) {
            maxBufferSize = oriAudioFormat.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE);
        } else {
            maxBufferSize = 100 * 1000;
        }
        ByteBuffer buffer = ByteBuffer.allocateDirect(maxBufferSize);
//        h264   H265  音频
        MediaCodec mediaCodec = MediaCodec.createDecoderByType(oriAudioFormat.getString((MediaFormat.KEY_MIME)));
//        设置解码器信息    直接从 音频文件
        mediaCodec.configure(oriAudioFormat, null, null, 0);
        File pcmFile = new File(outPath);
        FileChannel writeChannel = new FileOutputStream(pcmFile).getChannel();
        mediaCodec.start();
        MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
        int outputBufferIndex = -1;
        while (true) {
            int decodeInputIndex = mediaCodec.dequeueInputBuffer(100000);
            if (decodeInputIndex >= 0) {
                long sampleTimeUs = mediaExtractor.getSampleTime();

                if (sampleTimeUs == -1) {
                    break;
                } else if (sampleTimeUs < startTime) {
//                    丢掉 不用了
                    mediaExtractor.advance();
                    continue;
                } else if (sampleTimeUs > endTime) {
                    break;
                }
//                获取到压缩数据
                info.size = mediaExtractor.readSampleData(buffer, 0);
                info.presentationTimeUs = sampleTimeUs;
                info.flags = mediaExtractor.getSampleFlags();

//                下面放数据  到dsp解码
                byte[] content = new byte[buffer.remaining()];
                buffer.get(content);
//                输出文件  方便查看
//                FileUtils.writeContent(content);
//                解码
                ByteBuffer inputBuffer = mediaCodec.getInputBuffer(decodeInputIndex);
                inputBuffer.put(content);
                mediaCodec.queueInputBuffer(decodeInputIndex, 0, info.size, info.presentationTimeUs, info.flags);
//                释放上一帧的压缩数据
                mediaExtractor.advance();
            }

            outputBufferIndex = mediaCodec.dequeueOutputBuffer(info, 100_000);
            while (outputBufferIndex >= 0) {
                ByteBuffer decodeOutputBuffer = mediaCodec.getOutputBuffer(outputBufferIndex);
                writeChannel.write(decodeOutputBuffer);//MP3  1   pcm2
                mediaCodec.releaseOutputBuffer(outputBufferIndex, false);
                outputBufferIndex = mediaCodec.dequeueOutputBuffer(info, 100_000);
            }
        }
        writeChannel.close();
        mediaExtractor.release();
        mediaCodec.stop();
        mediaCodec.release();
    }

    //找出对应的轨道
    private static int selectTrack(MediaExtractor mediaExtractor, String fo) {
        //获取每条轨道
        int numTracks = mediaExtractor.getTrackCount();
        for (int i = 0; i < numTracks; i++) {
            MediaFormat format = mediaExtractor.getTrackFormat(i);
            String mime = format.getString(MediaFormat.KEY_MIME);
            if (mime.startsWith(fo)) {
                return i;
            }
        }
        return -1;
    }

    public static void mixVideoAndMusic(String videoInput, String output, Integer startTimeUs, Integer endTimeUs, File wavFile) throws IOException {
        //初始化一个视频封装容器
        MediaMuxer mediaMuxer = new MediaMuxer(output, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);

        MediaExtractor mediaExtractor = new MediaExtractor();
        mediaExtractor.setDataSource(videoInput);

        //视频的视频轨道
        int video_videoIndex = selectTrack(mediaExtractor, "video/");
        //视频的音频轨道
        int video_audioIndex = selectTrack(mediaExtractor, "audio/");

        //新建视频轨道
        MediaFormat videoFormat = mediaExtractor.getTrackFormat(video_videoIndex);
        int mixedVideoTrack = mediaMuxer.addTrack(videoFormat);

        //新建音频轨道
        MediaFormat audioFormat1 = mediaExtractor.getTrackFormat(video_audioIndex);
        int audioBitRate = 16_000;
        if(audioFormat1.containsKey(MediaFormat.KEY_BIT_RATE)) {
            audioBitRate = audioFormat1.getInteger(MediaFormat.KEY_BIT_RATE);
        }
        audioFormat1.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_AAC);
        //添加一个空的音频轨道  轨道格式取自 视频文件，跟视频所有信息一样
        int mixedAudioTrack = mediaMuxer.addTrack(audioFormat1);
        mediaMuxer.start();



        //处理音频
        MediaExtractor audioExtrator = new MediaExtractor();
        audioExtrator.setDataSource(wavFile.getAbsolutePath());
        int audioTrack = selectTrack(audioExtrator,"audio/");
        audioExtrator.selectTrack(audioTrack);
        MediaFormat audioFormat = audioExtrator.getTrackFormat(audioTrack);
        //最大一帧的大小
        int maxBufferSize;
        if (audioFormat.containsKey(MediaFormat.KEY_MAX_INPUT_SIZE)) {
            maxBufferSize = audioFormat.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE);
        } else {
            maxBufferSize = 100 * 1000;
        }

        MediaFormat encodeFormat = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, 44100, 2);
        //比特率
        encodeFormat.setInteger(MediaFormat.KEY_BIT_RATE, audioBitRate);
        //音质等级
        encodeFormat.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC);
        //最大一帧大小
        encodeFormat.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, maxBufferSize);
        //初始化音频编码器
        MediaCodec audioMediaCodecEncoder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC);
        audioMediaCodecEncoder.configure(encodeFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
        audioMediaCodecEncoder.start();
        ByteBuffer buffer = ByteBuffer.allocateDirect(maxBufferSize);
        MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
        boolean encodeDone = false;
        while (!encodeDone) {
            int inputBufferIndex = audioMediaCodecEncoder.dequeueInputBuffer(10000);
            if (inputBufferIndex >= 0) {
                long sampleTime = audioExtrator.getSampleTime();
                if (sampleTime < 0) {
                    //pts小于0 来到了文件末尾 通知编码器 不用编码了
                    audioMediaCodecEncoder.queueInputBuffer(inputBufferIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                } else {
                    int flags = audioExtrator.getSampleFlags();
                    int size = audioExtrator.readSampleData(buffer, 0);
                    ByteBuffer inputBuffer = audioMediaCodecEncoder.getInputBuffer(inputBufferIndex);
                    inputBuffer.clear();
                    inputBuffer.put(buffer);
                    inputBuffer.position(0);
                    audioMediaCodecEncoder.queueInputBuffer(inputBufferIndex, 0, size, sampleTime, flags);
                    //前进到下一个样本
                    audioExtrator.advance();
                }
            }
            //获取编码完的数据
            int outputBufferIndex = audioMediaCodecEncoder.dequeueOutputBuffer(info, TIMEOUT);
            while (outputBufferIndex >= 0) {
                if (info.flags == MediaCodec.BUFFER_FLAG_END_OF_STREAM) {
                    encodeDone = true;
                    break;
                }
                ByteBuffer encodeOutputBuffer = audioMediaCodecEncoder.getOutputBuffer(outputBufferIndex);
                //将编码好的数据(aac)输出
                mediaMuxer.writeSampleData(mixedAudioTrack, encodeOutputBuffer, info);
                encodeOutputBuffer.clear();
                audioMediaCodecEncoder.releaseOutputBuffer(outputBufferIndex, false);
                outputBufferIndex = audioMediaCodecEncoder.dequeueOutputBuffer(info, TIMEOUT);
            }
        }

        //处理视频
        if (video_audioIndex >= 0) {
            mediaExtractor.unselectTrack(video_audioIndex);
        }
        mediaExtractor.selectTrack(video_videoIndex);
        mediaExtractor.seekTo(startTimeUs, MediaExtractor.SEEK_TO_PREVIOUS_SYNC);
        maxBufferSize = videoFormat.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE);
        buffer = ByteBuffer.allocateDirect(maxBufferSize);
        while (true) {
            long sampleTimeUs = mediaExtractor.getSampleTime();
            if (sampleTimeUs == -1) {
                break;
            }
            if (sampleTimeUs < startTimeUs) {
                mediaExtractor.advance();
                continue;
            }
            if (endTimeUs != null && sampleTimeUs > endTimeUs) {
                break;
            }
            info.presentationTimeUs = sampleTimeUs - startTimeUs + 600;
            info.flags = mediaExtractor.getSampleFlags();
            //读取视频文件的数据  画面 数据
            info.size = mediaExtractor.readSampleData(buffer, 0);
            if (info.size < 0) {
                break;
            }
            //视频轨道  画面写完了
            mediaMuxer.writeSampleData(mixedVideoTrack, buffer, info);
            mediaExtractor.advance();
        }
        try {
            audioExtrator.release();
            mediaExtractor.release();
            audioMediaCodecEncoder.stop();
            audioMediaCodecEncoder.release();
            mediaMuxer.release();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

这里生成的新的mp4文件，用windows10自带播放器播放只有音频没有画面，用Vlan播放器可以播放。

Demo链接