学习笔记- AVFoundation Programming Guide - Editing

Editing

AVFoundation 提供了功能丰富的类方便编辑音视频assets。AVFoundation的编辑API的关键在于compositions。compositions简单的吧一个或多个不同媒体的assets集合到一起。

AVMutableComposition 类提供了插入和删除tracks的接口，以及管理他们的时间排序。

AVMutableAudioMix 类可以在compositon的audio tracks上自定义音频处理。

AVMutableVideoComposition 类可以让你在composition上直接编辑video tracks。

通过 video Composition 指令（例如AVMutableVideoCompositionInstruction 类），你能修改视频的背景颜色和layer 指令。

layer指令（AVMutableVideoCompositionLayerInstruction 类）可用于应用变换，ramps转换，不透明度到你的composition里的视频tracks。

video composition 类也支持Core Animation，使用 animationTool 属性。

你可以用 AVAssetExportSession 合并你的音视频 composition。

1. Createing a Composition

创建Composition，需要用 AVMutableComposition类。向你的Composition添加媒体数据，必须添加一个或多个composition tracks，例如 AVMutableCompositionTrack类。最简单的方法通过一个视频 track 和一个音频track创建mutable composition。

code：

AVMutableComposition *mutableComposition = [AVMutableComposition composition];
// Create the video composition track.
AVMutableCompositionTrack *mutableCompositionVideoTrack = [mutableComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
// Create the audio composition track.
AVMutableCompositionTrack *mutableCompositionAudioTrack = [mutableComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];

Options for Initializing a Composition Track

当向一个composition添加新的tracks时，必须提供媒体类型和track id。虽然音频和视频是最常用的媒体类型，但您可以指定其他媒体类型，如：

AVMediaTypeSubtitle or AVMediaTypeText。

每个track 相关联的视听数据都有一个唯一标识符称作 track ID。如果你指定kCMPersistentTrackID_Invalid座位track ID，将会为你自动生成一个唯一标识符并与track相关联。

 

2. Adding Audiovisual Data to a Composition

一旦你有一个composition（需要包含一个或多个tracks），就可以开始添加媒体数据到适当的tracks了。

添加媒体数据到composition track，你需要访问AVAsset对象媒体数据的位置。

你可以试用 mutable composition track 接口来让多个相同类型的tracks放到相同track上。

下面示例演示如何在相同composition track上添加两个不同video asset tracks。

code：

// You can retrieve AVAssets from a number of places, like the camera roll for example.
AVAsset *videoAsset = <#AVAsset with at least one video track#>;
AVAsset *anotherVideoAsset = <#another AVAsset with at least one video track#>;
// Get the first video track from each asset.
AVAssetTrack *videoAssetTrack = [[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
AVAssetTrack *anotherVideoAssetTrack = [[anotherVideoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
// Add them both to the composition.
[mutableCompositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero,videoAssetTrack.timeRange.duration) ofTrack:videoAssetTrack atTime:kCMTimeZero error:nil];
[mutableCompositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero,anotherVideoAssetTrack.timeRange.duration) ofTrack:anotherVideoAssetTrack atTime:videoAssetTrack.timeRange.duration error:nil];

检索兼容Composition Tracks

如果可能的话，应该为每种媒体类型只有一个composition track。这种兼容asset tracks 会使资源使用量最小。

当创新呈现媒体数据时，你应该将同一类型的任何媒体数据放到同一个composition track上。

你可以查询mutable composition 来找出是否与你的asset track 不兼容。

code：

AVMutableCompositionTrack *compatibleCompositionTrack = [mutableComposition mutableTrackCompatibleWithTrack:<#the AVAssetTrack you want to insert#>];
if (compatibleCompositionTrack) {
    // Implementation continues.
}

注：在相同的composition track 上放置多个视频片段，在视频片段切换时可能会丢帧，尤其是嵌入式设备。composition tracks 上视频片段的数量完全取决于app和他的平台。

3. Generating a Volume Ramp

AVMutableAudioMix 对象可以对你的composition 的所有的音频tracks进行自定义处理。

下面的示例将显示如何在特定的音频track上设置音量ramp，以在composition的持续时间缓慢淡出音频

code：

AVMutableAudioMix *mutableAudioMix = [AVMutableAudioMix audioMix];
// Create the audio mix input parameters object.
AVMutableAudioMixInputParameters *mixParameters = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:mutableCompositionAudioTrack];
// Set the volume ramp to slowly fade the audio out over the duration of the composition.
[mixParameters setVolumeRampFromStartVolume:1.f toEndVolume:0.f timeRange:CMTimeRangeMake(kCMTimeZero, mutableComposition.duration)];
// Attach the input parameters to the audio mix.
mutableAudioMix.inputParameters = @[mixParameters];

4. Perfroming Custom Video Processing

你只需要一个AVMutableVideoComposition 对象，在你的composition video tracks 上执行自定义视频处理。

使用视频组合，您可以直接在你的composition video tracks 上设置适当的渲染大小，缩放，和帧速率。

1）Changing the Composition's Background Color

所有的 video composition 必须有一个AVVideoCompositionInstruction 对象的数组，至少有一个 video composition instruction。

用AVMutableVideoCompositionInstruction 类创建你的video composition instructions。使用video composition instruction，你可以修改composition的背景颜色，指定是否需要后期处理或者应用 layer instructions。

code：

// 创建video composition instruction ，改变整个composit背景颜色

AVMutableVideoCompositionInstruction *mutableVideoCompositionInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
mutableVideoCompositionInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, mutableComposition.duration);
mutableVideoCompositionInstruction.backgroundColor = [[UIColor redColor] CGColor];

2）Applying Opacity Ramps

video composition instruction 也可以用来申请 video composition layer instruction。

AVMutableVideoCompositionLayerInstruction 对象可以应用变换，ramps变换，不透明度和ramps不透明度到composition的视频track里。

下面代码演示了如何设置一个不透明的ramp从第一个视频慢慢淡出到第二个视频：

 code：

AVAsset *firstVideoAssetTrack = <#AVAssetTrack representing the first video segment played in the composition#>;
AVAsset *secondVideoAssetTrack = <#AVAssetTrack representing the second video segment played in the composition#>;

// Create the first video composition instruction.
AVMutableVideoCompositionInstruction *firstVideoCompositionInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
// Set its time range to span the duration of the first video track.
firstVideoCompositionInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, firstVideoAssetTrack.timeRange.duration);

// Create the layer instruction and associate it with the composition video track.
AVMutableVideoCompositionLayerInstruction *firstVideoLayerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:mutableCompositionVideoTrack];
// Create the opacity ramp to fade out the first video track over its entire duration.
[firstVideoLayerInstruction setOpacityRampFromStartOpacity:1.f toEndOpacity:0.f timeRange:CMTimeRangeMake(kCMTimeZero, firstVideoAssetTrack.timeRange.duration)];

// Create the second video composition instruction so that the second video track isn't transparent.
AVMutableVideoCompositionInstruction *secondVideoCompositionInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
// Set its time range to span the duration of the second video track.
secondVideoCompositionInstruction.timeRange = CMTimeRangeMake(firstVideoAssetTrack.timeRange.duration, CMTimeAdd(firstVideoAssetTrack.timeRange.duration, secondVideoAssetTrack.timeRange.duration));

// Create the second layer instruction and associate it with the composition video track.
AVMutableVideoCompositionLayerInstruction *secondVideoLayerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:mutableCompositionVideoTrack];

// Attach the first layer instruction to the first video composition instruction.
firstVideoCompositionInstruction.layerInstructions = @[firstVideoLayerInstruction];
// Attach the second layer instruction to the second video composition instruction.
secondVideoCompositionInstruction.layerInstructions = @[secondVideoLayerInstruction];

// Attach both of the video composition instructions to the video composition.
AVMutableVideoComposition *mutableVideoComposition = [AVMutableVideoComposition videoComposition];
mutableVideoComposition.instructions = @[firstVideoCompositionInstruction, secondVideoCompositionInstruction];

3）Incorporating Core Animation Effects

video composition 可以通过 animationTool 属性添加CoreAnimation。通过animation tool，你可以完成入水印视频和添加片头或遮盖。使用Core Animation 可以用两种不同的方式：添加一个 Core Animation layer 作为自己的 composition track，或者直接在你的 composition 上的视频帧上绘制Core Animation 特效（使用一个Core Animation layer)。

code：

// 使用上面说的第二种方法在视频中心添加一个水印

CALayer *watermarkLayer = <#CALayer representing your desired watermark image#>;
CALayer *parentLayer = [CALayer layer];
CALayer *videoLayer = [CALayer layer];
parentLayer.frame = CGRectMake(0, 0, mutableVideoComposition.renderSize.width, mutableVideoComposition.renderSize.height);
videoLayer.frame = CGRectMake(0, 0, mutableVideoComposition.renderSize.width, mutableVideoComposition.renderSize.height);
[parentLayer addSublayer:videoLayer];
watermarkLayer.position = CGPointMake(mutableVideoComposition.renderSize.width/2, mutableVideoComposition.renderSize.height/4);
[parentLayer addSublayer:watermarkLayer];
mutableVideoComposition.animationTool = [AVVideoCompositionCoreAnimationTool videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayer:videoLayer inLayer:parentLayer];

5. 合并多个Asset并且保存结果到Camera Roll

下面演示了如何把2个视频asset tracks 和一个音频asset track 接合起来并创建一个单一文件。步骤：

a. 创建一个 AVMutableComposition 对象，并添加多个 AVMutableCompositionTrack 对象；

b. 添加AVAssetTrack 对象的时间范围到 composition tracks；

c. 检测视频 asset track 的 preferredTransform 属性，确定视频的方向；

d. 用 AVMutableVIdeoCompositionLayerInstruction 对象转换composition 里的视频 tracks；

e. 设置 video composition 的 renderSize 和 frameDuration 属性；

 f. 用video composition 导出视频文件；

g. 保存视频文件到 Camera Roll。

1）Creating the Composition

code：

// 创建 Composition 并添加一个音频track和一个视频track

AVMutableComposition *mutableComposition = [AVMutableComposition composition];
AVMutableCompositionTrack *videoCompositionTrack = [mutableComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
AVMutableCompositionTrack *audioCompositionTrack = [mutableComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];

2) Adding the Assets

code：

// 添加2个视频 asset tracks 和一个音频 asset track 到 composition

AVAssetTrack *firstVideoAssetTrack = [[firstVideoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
AVAssetTrack *secondVideoAssetTrack = [[secondVideoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];

[videoCompositionTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, firstVideoAssetTrack.timeRange.duration) ofTrack:firstVideoAssetTrack atTime:kCMTimeZero error:nil];

[videoCompositionTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, secondVideoAssetTrack.timeRange.duration) ofTrack:secondVideoAssetTrack atTime:firstVideoAssetTrack.timeRange.duration error:nil];

[audioCompositionTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, CMTimeAdd(firstVideoAssetTrack.timeRange.duration, secondVideoAssetTrack.timeRange.duration)) ofTrack:[[audioAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] atTime:kCMTimeZero error:nil];

3) Checking the Video Orientations

一旦添加了视频tracks和音频tracks到composition后，需要确定video tracks 的方向是正确的。

默认情况下，所有视频tracks 被假定为 landscape 模式。

如果你的视频track是在纵向模式拍摄的，视频播放时将不能正确定位。

同样，如果尝试合并横向拍摄的视频和纵向拍摄的视频时， export session 将会失败。

code：

BOOL isFirstVideoPortrait = NO;
CGAffineTransform firstTransform = firstVideoAssetTrack.preferredTransform;

// Check the first video track's preferred transform to determine if it was recorded in portrait mode.
if (firstTransform.a == 0 && firstTransform.d == 0 && (firstTransform.b == 1.0 || firstTransform.b == -1.0) && (firstTransform.c == 1.0 || firstTransform.c == -1.0)) {
    isFirstVideoPortrait = YES;
}

BOOL isSecondVideoPortrait = NO;
CGAffineTransform secondTransform = secondVideoAssetTrack.preferredTransform;

// Check the second video track's preferred transform to determine if it was recorded in portrait mode.
if (secondTransform.a == 0 && secondTransform.d == 0 && (secondTransform.b == 1.0 || secondTransform.b == -1.0) && (secondTransform.c == 1.0 || secondTransform.c == -1.0)) {
    isSecondVideoPortrait = YES;
}

if ((isFirstVideoAssetPortrait && !isSecondVideoAssetPortrait) || (!isFirstVideoAssetPortrait && isSecondVideoAssetPortrait)) {
    UIAlertView *incompatibleVideoOrientationAlert = [[UIAlertView alloc] initWithTitle:@"Error!" message:@"Cannot combine a video shot in portrait mode with a video shot in landscape mode." delegate:self cancelButtonTitle:@"Dismiss" otherButtonTitles:nil];
    [incompatibleVideoOrientationAlert show];
    return;
}

4) Applying the Video Composition Layer Instructions

一旦验证了视频方向的兼容后，你可以使用layer instructions，并添加到video composition 的layer instructions中。

code：

AVMutableVideoCompositionInstruction *firstVideoCompositionInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
// Set the time range of the first instruction to span the duration of the first video track.
firstVideoCompositionInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, firstVideoAssetTrack.timeRange.duration);

AVMutableVideoCompositionInstruction * secondVideoCompositionInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
// Set the time range of the second instruction to span the duration of the second video track.
secondVideoCompositionInstruction.timeRange = CMTimeRangeMake(firstVideoAssetTrack.timeRange.duration, CMTimeAdd(firstVideoAssetTrack.timeRange.duration, secondVideoAssetTrack.timeRange.duration));

AVMutableVideoCompositionLayerInstruction *firstVideoLayerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:videoCompositionTrack];
// Set the transform of the first layer instruction to the preferred transform of the first video track.
[firstVideoLayerInstruction setTransform:firstTransform atTime:kCMTimeZero];

AVMutableVideoCompositionLayerInstruction *secondVideoLayerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:videoCompositionTrack];
// Set the transform of the second layer instruction to the preferred transform of the second video track.
[secondVideoLayerInstruction setTransform:secondTransform atTime:firstVideoAssetTrack.timeRange.duration];

firstVideoCompositionInstruction.layerInstructions = @[firstVideoLayerInstruction];
secondVideoCompositionInstruction.layerInstructions = @[secondVideoLayerInstruction];

AVMutableVideoComposition *mutableVideoComposition = [AVMutableVideoComposition videoComposition];
mutableVideoComposition.instructions = @[firstVideoCompositionInstruction, secondVideoCompositionInstruction];

所有 AVAssetTrack 对象有 preferredTransform 属性，包含asset track 的方向信息。当asset track 显示在屏幕上时转换。

前面的代码中，layer instruction 的转换被设置到 asset track 的转换，以便当你调整绘制大小时，视频的新 composition 的显示正确。

5) Setting the Render Size and Frame Duration

视频方向确认后，你必须调整 renderSize 属性。也应该适当选择 frameDuration 属性（如30帧/秒）。默认，renderScale 为1.0， 这是适当的组合。

code：

CGSize naturalSizeFirst, naturalSizeSecond;
// If the first video asset was shot in portrait mode, then so was the second one if we made it here.
if (isFirstVideoAssetPortrait) {
// Invert the width and height for the video tracks to ensure that they display properly.
    naturalSizeFirst = CGSizeMake(firstVideoAssetTrack.naturalSize.height, firstVideoAssetTrack.naturalSize.width);
    naturalSizeSecond = CGSizeMake(secondVideoAssetTrack.naturalSize.height, secondVideoAssetTrack.naturalSize.width);
}
else {
// If the videos weren't shot in portrait mode, we can just use their natural sizes.
    naturalSizeFirst = firstVideoAssetTrack.naturalSize;
    naturalSizeSecond = secondVideoAssetTrack.naturalSize;
}
float renderWidth, renderHeight;
// Set the renderWidth and renderHeight to the max of the two videos widths and heights.
if (naturalSizeFirst.width > naturalSizeSecond.width) {
    renderWidth = naturalSizeFirst.width;
}
else {
    renderWidth = naturalSizeSecond.width;
}
if (naturalSizeFirst.height > naturalSizeSecond.height) {
    renderHeight = naturalSizeFirst.height;
}
else {
    renderHeight = naturalSizeSecond.height;
}
mutableVideoComposition.renderSize = CGSizeMake(renderWidth, renderHeight);
// Set the frame duration to an appropriate value (i.e. 30 frames per second for video).
mutableVideoComposition.frameDuration = CMTimeMake(1,30);

6) Exporting the Composition and Saving it to the Camera Roll

最后一步使用 AVAssetExportSession 对象通过URL创建一个新的视频文件，然后使用 ALAssetsLibrary 类保存到 Camera Roll 中。

code：

// Create a static date formatter so we only have to initialize it once.
static NSDateFormatter *kDateFormatter;
if (!kDateFormatter) {
    kDateFormatter = [[NSDateFormatter alloc] init];
    kDateFormatter.dateStyle = NSDateFormatterMediumStyle;
    kDateFormatter.timeStyle = NSDateFormatterShortStyle;
}

// Create the export session with the composition and set the preset to the highest quality.
AVAssetExportSession *exporter = [[AVAssetExportSession alloc] initWithAsset:mutableComposition presetName:AVAssetExportPresetHighestQuality];
// Set the desired output URL for the file created by the export process.
exporter.outputURL = [[[[NSFileManager defaultManager] URLForDirectory:NSDocumentDirectory inDomain:NSUserDomainMask appropriateForURL:nil create:@YES error:nil] URLByAppendingPathComponent:[kDateFormatter stringFromDate:[NSDate date]]] URLByAppendingPathExtension:CFBridgingRelease(UTTypeCopyPreferredTagWithClass((CFStringRef)AVFileTypeQuickTimeMovie, kUTTagClassFilenameExtension))];
// Set the output file type to be a QuickTime movie.
exporter.outputFileType = AVFileTypeQuickTimeMovie;
exporter.shouldOptimizeForNetworkUse = YES;
exporter.videoComposition = mutableVideoComposition;

// Asynchronously export the composition to a video file and save this file to the camera roll once export completes.
[exporter exportAsynchronouslyWithCompletionHandler:^{
    dispatch_async(dispatch_get_main_queue(), ^{
        if (exporter.status == AVAssetExportSessionStatusCompleted) {
            ALAssetsLibrary *assetsLibrary = [[ALAssetsLibrary alloc] init];
            if ([assetsLibrary videoAtPathIsCompatibleWithSavedPhotosAlbum:exporter.outputURL]) {
                [assetsLibrary writeVideoAtPathToSavedPhotosAlbum:exporter.outputURL completionBlock:NULL];
            }
        }
    });
}];