JavaCV合并音频到视频封装成MP4

将音频合并到视频有两种情况：

一)音频的播放时长大于视频的播放时长

这种情况相对简单只要在控制音频的播放时长与视频对齐则可，这里主要难点主要在如何使用JavaCV来获取音频时长，代码如下：

		String imageInput = this.home + "01.mp4";//
		FFmpegFrameGrabber imageGrabber = new FFmpegFrameGrabber(imageInput);
		imageGrabber.start();
		String audioInput = this.home + "xiaolaba.mp3";
		FFmpegFrameGrabber audioGrabber = new FFmpegFrameGrabber(audioInput);
		audioGrabber.start();
		String outputPath = this.home + "testAudio.mp4";
		// 流媒体输出地址，分辨率（长，高），是否录制音频（0:单声道/1:立体声）
		FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(outputPath, imageGrabber.getImageWidth(), imageGrabber.getImageHeight(), 1);
		recorder.setInterleaved(true);
		recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
		recorder.setFormat("mp4");
		recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P); // yuv420p
		int frameRate = 25;
		recorder.setFrameRate(frameRate);
		recorder.setGopSize(frameRate * 2);
		recorder.setAudioOption("crf", "0");
		recorder.setAudioQuality(0);// 最高质量
		recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
		recorder.setAudioChannels(2);
		recorder.setSampleRate(44100);
		recorder.start();
		long videoTime = imageGrabber.getLengthInTime();
		Frame imageFrame = null;
		while ((imageFrame = imageGrabber.grabImage()) != null) {
			recorder.record(imageFrame);
		}
		Frame sampleFrame = null;
		while ((sampleFrame = audioGrabber.grabSamples()) != null) {
			recorder.record(sampleFrame);
			if (audioGrabber.getTimestamp() >= videoTime) {
				break;
			}
		}
		recorder.close();
		audioGrabber.close();
		imageGrabber.close();
	

二)视频的播放时长大于音频的播放时长

当音频流播放结束时sampleFrame将为空，此时调用音频抓取器的#restart方法可以重新操作，这样就能达到重新播放的效果，代码如下：

	
		String imageInput = this.home + "01.mp4";//
		String audioInput = this.home + "tt21.aac";
		String outputPath = this.home + "testAudio.mp4";
		// 流媒体输出地址，分辨率（长，高），是否录制音频（0:单声道/1:立体声）
		try (FFmpegFrameGrabber imageGrabber = new FFmpegFrameGrabber(imageInput);
				FFmpegFrameGrabber audioGrabber = new FFmpegFrameGrabber(audioInput)) {
			imageGrabber.start();
			audioGrabber.start();
			try (FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(outputPath, imageGrabber.getImageWidth(), imageGrabber.getImageHeight(),
					1);) {

				recorder.setInterleaved(true);
				recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
				recorder.setFormat("mp4");
				recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P); // yuv420p
				int frameRate = 25;
				recorder.setFrameRate(frameRate);
				recorder.setGopSize(frameRate * 2);
				recorder.setAudioOption("crf", "0");
				recorder.setAudioQuality(0);// 最高质量
				recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
				recorder.setAudioChannels(2);
				recorder.setSampleRate(44100);
				recorder.start();
				long videoTime = imageGrabber.getLengthInTime();
				Frame imageFrame = null;
				while ((imageFrame = imageGrabber.grabImage()) != null) {
					recorder.record(imageFrame);
				}
				long audioPlayTime = 0L;
				Frame sampleFrame = null;
				while ((sampleFrame = audioGrabber.grabSamples()) != null || audioPlayTime < videoTime) {
					if (sampleFrame == null) {
						audioGrabber.restart();//重新开始
						sampleFrame = audioGrabber.grabSamples();
						videoTime -= audioPlayTime;
					}
					recorder.record(sampleFrame);
					audioPlayTime = audioGrabber.getTimestamp();
					if (audioPlayTime >= videoTime) {
						break;
					}
				}
			}
		} catch (Exception e) {
			throw new RuntimeException(e);
		}

	
	

总结

上述的两种情况都音视频是同步抓取的，如果想提高时效性可以并行抓取即音频和视频采用不同的线程并行抓取（注意只能是采集的时候并行)，FFMpeg不能并行输出，但这样做缺点也是明显就是要耗费更多的内存，故实际需权衡利弊。