【最佳实践】OSS移动端断点下载简单实现

• 概要
• 技术点
• 最佳实践
• 改进
• 参考

概要

所谓断点下载，其实是客户端在从网络上下载资源时,由于某种原因中断下载。再次开启下载时可以从已经下载的部分开始继续下载未完成的部分，从而节省时间和流量。

应用场景：当我们在手机端使用视频软件下载视频时,下载期间网络模式从WIFI切换到移动网络,默认App都会中断下载。当再次切换到WIFI网络时，由用户手动重新开启下载任务,此时就用到了断点下载。

优点：节省时间和流量。

技术点

HTTP1.1中新增了Range头的支持，用于指定获取数据的范围。Range的格式一般分为以下几种：

• Range: bytes=100- 从 101 bytes 之后开始传，一直传到最后。
• Range: bytes=100-200 指定开始到结束这一段的长度，记住 Range 是从 0 计数 的，所以这个是要求服务器从 101 字节开始传，一直到 201 字节结束。这个一般用来特别大的文件分片传输，比如视频。
• Range: bytes=-100 如果范围没有指定开始位置，就是要服务器端传递倒数 100 字节的内容。而不是从 0 开始的 100 字节。
• Range: bytes=0-100, 200-300 也可以同时指定多个范围的内容，这种情况不是很常见。

另外断点续传时需要验证服务器上面的文件是否发生变化，此时用到了If-Match头。If-Match对应的是Etag的值。

客户端在发起请求时在Header中携带Range,If-Match,OSS服务器在收到请求后会验证'If-Match'中的Etag值,如果不匹配,则会返回412 precondition 状态码。

OSS的服务器针对getObject这个开放API已经支持了Range, If-Match,If-None-Match,If-Modified-Since,If-Unmodified-Since，所有我们能够在移动端实践OSS资源的断点下载功能。

最佳实践

首先来看一下流程图

效果图：

这里以iOS下载实现思路为例，参考代码如下 仅供参考,切不可用于生产！

#import "DownloadService.h"
#import "OSSTestMacros.h"

@implementation DownloadRequest

@end

@implementation Checkpoint

- (instancetype)copyWithZone:(NSZone *)zone {
    Checkpoint *other = [[[self class] allocWithZone:zone] init];
    
    other.etag = self.etag;
    other.totalExpectedLength = self.totalExpectedLength;
    
    return other;
}

@end


@interface DownloadService()<NSURLSessionTaskDelegate, NSURLSessionDataDelegate>

@property (nonatomic, strong) NSURLSession *session;         //网络会话
@property (nonatomic, strong) NSURLSessionDataTask *dataTask;   //数据请求任务
@property (nonatomic, copy) DownloadFailureBlock failure;    //请求出错
@property (nonatomic, copy) DownloadSuccessBlock success;    //请求成功
@property (nonatomic, copy) DownloadProgressBlock progress;  //下载进度
@property (nonatomic, copy) Checkpoint *checkpoint;        //检查节点
@property (nonatomic, copy) NSString *requestURLString;    //文件资源地址,用于下载请求
@property (nonatomic, copy) NSString *headURLString;       //文件资源地址,用于head请求
@property (nonatomic, copy) NSString *targetPath;     //文件存储路径
@property (nonatomic, assign) unsigned long long totalReceivedContentLength; //已下载大小
@property (nonatomic, strong) dispatch_semaphore_t semaphore;

@end

@implementation DownloadService

- (instancetype)init
{
    self = [super init];
    if (self) {
        NSURLSessionConfiguration *conf = [NSURLSessionConfiguration defaultSessionConfiguration];
        conf.timeoutIntervalForRequest = 15;
        
        NSOperationQueue *processQueue = [NSOperationQueue new];
        _session = [NSURLSession sessionWithConfiguration:conf delegate:self delegateQueue:processQueue];
        _semaphore = dispatch_semaphore_create(0);
        _checkpoint = [[Checkpoint alloc] init];
    }
    return self;
}

+ (instancetype)downloadServiceWithRequest:(DownloadRequest *)request {
    DownloadService *service = [[DownloadService alloc] init];
    if (service) {
        service.failure = request.failure;
        service.success = request.success;
        service.requestURLString = request.sourceURLString;
        service.headURLString = request.headURLString;
        service.targetPath = request.downloadFilePath;
        service.progress = request.downloadProgress;
        if (request.checkpoint) {
            service.checkpoint = request.checkpoint;
        }
    }
    return service;
}

/**
 * head文件信息，取出来文件的etag和本地checkpoint中保存的etag进行对比,并且将结果返回
 */
- (BOOL)getFileInfo {
    __block BOOL resumable = NO;
    NSURL *url = [NSURL URLWithString:self.headURLString];
    NSMutableURLRequest *request = [[NSMutableURLRequest alloc]initWithURL:url];
    [request setHTTPMethod:@"HEAD"];
    
    NSURLSessionDataTask *task = [[NSURLSession sharedSession] dataTaskWithRequest:request completionHandler:^(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error) {
        if (error) {
            NSLog(@"获取文件meta信息失败,error : %@", error);
        } else {
            NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *)response;
            NSString *etag = [httpResponse.allHeaderFields objectForKey:@"Etag"];
            if ([self.checkpoint.etag isEqualToString:etag]) {
                resumable = YES;
            } else {
                resumable = NO;
            }
        }
        dispatch_semaphore_signal(self.semaphore);
    }];
    [task resume];
    
    dispatch_semaphore_wait(self.semaphore, DISPATCH_TIME_FOREVER);
    return resumable;
}

/**
 * 用于获取本地文件的大小
 */
- (unsigned long long)fileSizeAtPath:(NSString *)filePath {
    unsigned long long fileSize = 0;
    NSFileManager *dfm = [NSFileManager defaultManager];
    if ([dfm fileExistsAtPath:filePath]) {
        NSError *error = nil;
        NSDictionary *attributes = [dfm attributesOfItemAtPath:filePath error:&error];
        if (!error && attributes) {
            fileSize = attributes.fileSize;
        } else if (error) {
            NSLog(@"error: %@", error);
        }
    }

    return fileSize;
}

- (void)resume {
    NSURL *url = [NSURL URLWithString:self.requestURLString];
    NSMutableURLRequest *request = [[NSMutableURLRequest alloc]initWithURL:url];
    [request setHTTPMethod:@"GET"];
    
    BOOL resumable = [self getFileInfo];    // 如果resumable为NO,则证明不能断点续传,否则走续传逻辑。
    if (resumable) {
        self.totalReceivedContentLength = [self fileSizeAtPath:self.targetPath];
        NSString *requestRange = [NSString stringWithFormat:@"bytes=%llu-", self.totalReceivedContentLength];
        [request setValue:requestRange forHTTPHeaderField:@"Range"];
    } else {
        self.totalReceivedContentLength = 0;
    }
    
    if (self.totalReceivedContentLength == 0) {
        [[NSFileManager defaultManager] createFileAtPath:self.targetPath contents:nil attributes:nil];
    }
    
    self.dataTask = [self.session dataTaskWithRequest:request];
    [self.dataTask resume];
}

- (void)pause {
    [self.dataTask cancel];
    self.dataTask = nil;
}

- (void)cancel {
    [self.dataTask cancel];
    self.dataTask = nil;
    [self removeFileAtPath: self.targetPath];
}

- (void)removeFileAtPath:(NSString *)filePath {
    NSError *error = nil;
    [[NSFileManager defaultManager] removeItemAtPath:self.targetPath error:&error];
    if (error) {
        NSLog(@"remove file with error : %@", error);
    }
}

#pragma mark - NSURLSessionDataDelegate

- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task
didCompleteWithError:(nullable NSError *)error {
    NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *)task.response;
    if ([httpResponse isKindOfClass:[NSHTTPURLResponse class]]) {
        if (httpResponse.statusCode == 200) {
            self.checkpoint.etag = [[httpResponse allHeaderFields] objectForKey:@"Etag"];
            self.checkpoint.totalExpectedLength = httpResponse.expectedContentLength;
        } else if (httpResponse.statusCode == 206) {
            self.checkpoint.etag = [[httpResponse allHeaderFields] objectForKey:@"Etag"];
            self.checkpoint.totalExpectedLength = self.totalReceivedContentLength + httpResponse.expectedContentLength;
        }
    }
    
    if (error) {
        if (self.failure) {
            NSMutableDictionary *userInfo = [NSMutableDictionary dictionaryWithDictionary:error.userInfo];
            [userInfo oss_setObject:self.checkpoint forKey:@"checkpoint"];
            
            NSError *tError = [NSError errorWithDomain:error.domain code:error.code userInfo:userInfo];
            self.failure(tError);
        }
    } else if (self.success) {
        self.success(@{@"status": @"success"});
    }
}

- (void)URLSession:(NSURLSession *)session dataTask:(NSURLSessionDataTask *)dataTask didReceiveResponse:(NSURLResponse *)response completionHandler:(void (^)(NSURLSessionResponseDisposition))completionHandler
{
    NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *)dataTask.response;
    if ([httpResponse isKindOfClass:[NSHTTPURLResponse class]]) {
        if (httpResponse.statusCode == 200) {
            self.checkpoint.totalExpectedLength = httpResponse.expectedContentLength;
        } else if (httpResponse.statusCode == 206) {
            self.checkpoint.totalExpectedLength = self.totalReceivedContentLength +  httpResponse.expectedContentLength;
        }
    }
    
    completionHandler(NSURLSessionResponseAllow);
}

- (void)URLSession:(NSURLSession *)session dataTask:(NSURLSessionDataTask *)dataTask didReceiveData:(NSData *)data {
    
    NSFileHandle *fileHandle = [NSFileHandle fileHandleForWritingAtPath:self.targetPath];
    [fileHandle seekToEndOfFile];
    [fileHandle writeData:data];
    [fileHandle closeFile];
    
    self.totalReceivedContentLength += data.length;
    if (self.progress) {
        self.progress(data.length, self.totalReceivedContentLength, self.checkpoint.totalExpectedLength);
    }
}

@end

上面展示的是从网络接收数据的处理逻辑，DownloadService是下载逻辑的核心。在URLSession:dataTask:didReceiveData中将接收到的网络数据按照追加的方式写入到文件中,并更新下载进度。在URLSession:task:didCompleteWithError判断下载任务是否完成,然后将结果返回给上层业务。URLSession:dataTask:didReceiveResponse:completionHandler代理方法中将object相关的信息,比如etag用于断点续传是的precheck,content-length用于计算下载进度。

#import <Foundation/Foundation.h>

typedef void(^DownloadProgressBlock)(int64_t bytesReceived, int64_t totalBytesReceived, int64_t totalBytesExpectToReceived);
typedef void(^DownloadFailureBlock)(NSError *error);
typedef void(^DownloadSuccessBlock)(NSDictionary *result);

@interface Checkpoint : NSObject<NSCopying>

@property (nonatomic, copy) NSString *etag;     // 资源的etag值
@property (nonatomic, assign) unsigned long long totalExpectedLength;    //文件总大小

@end

@interface DownloadRequest : NSObject

@property (nonatomic, copy) NSString *sourceURLString;      // 用于下载的url

@property (nonatomic, copy) NSString *headURLString;        // 用于获取文件原信息的url

@property (nonatomic, copy) NSString *downloadFilePath;     // 文件的本地存储地址

@property (nonatomic, copy) DownloadProgressBlock downloadProgress; // 下载进度

@property (nonatomic, copy) DownloadFailureBlock failure;   // 下载成功的回调

@property (nonatomic, copy) DownloadSuccessBlock success;   // 下载失败的回调

@property (nonatomic, copy) Checkpoint *checkpoint;         // checkpoint,用于存储文件的etag

@end


@interface DownloadService : NSObject

+ (instancetype)downloadServiceWithRequest:(DownloadRequest *)request;

/**
 * 启动下载
 */
- (void)resume;

/**
 * 暂停下载
 */
- (void)pause;

/**
 * 取消下载
 */
- (void)cancel;

@end

上面这一部分是DownloadRequest的定义。

- (void)initDownloadURLs {
    OSSPlainTextAKSKPairCredentialProvider *pCredential = [[OSSPlainTextAKSKPairCredentialProvider alloc] initWithPlainTextAccessKey:OSS_ACCESSKEY_ID secretKey:OSS_SECRETKEY_ID];
    _mClient = [[OSSClient alloc] initWithEndpoint:OSS_ENDPOINT credentialProvider:pCredential];
    
    // 生成用于get请求的带签名的url
    OSSTask *downloadURLTask = [_mClient presignConstrainURLWithBucketName:@"aliyun-dhc-shanghai" withObjectKey:OSS_DOWNLOAD_FILE_NAME withExpirationInterval:1800];
    [downloadURLTask waitUntilFinished];
    _downloadURLString = downloadURLTask.result;
    
    // 生成用于head请求的带签名的url
    OSSTask *headURLTask = [_mClient presignConstrainURLWithBucketName:@"aliyun-dhc-shanghai" withObjectKey:OSS_DOWNLOAD_FILE_NAME httpMethod:@"HEAD" withExpirationInterval:1800 withParameters:nil];
    [headURLTask waitUntilFinished];
    
    _headURLString = headURLTask.result;
}

- (IBAction)resumeDownloadClicked:(id)sender {
    _downloadRequest = [DownloadRequest new];
    _downloadRequest.sourceURLString = _downloadURLString;       // 设置资源的url
    _downloadRequest.headURLString = _headURLString;
    NSString *documentPath = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES).firstObject;
    _downloadRequest.downloadFilePath = [documentPath stringByAppendingPathComponent:OSS_DOWNLOAD_FILE_NAME];   //设置下载文件的本地保存路径
    
    __weak typeof(self) wSelf = self;
    _downloadRequest.downloadProgress = ^(int64_t bytesReceived, int64_t totalBytesReceived, int64_t totalBytesExpectToReceived) {
        // totalBytesReceived是当前客户端已经缓存了的字节数,totalBytesExpectToReceived是总共需要下载的字节数。
        dispatch_async(dispatch_get_main_queue(), ^{
            __strong typeof(self) sSelf = wSelf;
            CGFloat fProgress = totalBytesReceived * 1.f / totalBytesExpectToReceived;
            sSelf.progressLab.text = [NSString stringWithFormat:@"%.2f%%", fProgress * 100];
            sSelf.progressBar.progress = fProgress;
        });
    };
    _downloadRequest.failure = ^(NSError *error) {
        __strong typeof(self) sSelf = wSelf;
        sSelf.checkpoint = error.userInfo[@"checkpoint"];
    };
    _downloadRequest.success = ^(NSDictionary *result) {
        NSLog(@"下载成功");
    };
    _downloadRequest.checkpoint = self.checkpoint;
    
    NSString *titleText = [[_downloadButton titleLabel] text];
    if ([titleText isEqualToString:@"download"]) {
        [_downloadButton setTitle:@"pause" forState: UIControlStateNormal];
        _downloadService = [DownloadService downloadServiceWithRequest:_downloadRequest];
        [_downloadService resume];
    } else {
        [_downloadButton setTitle:@"download" forState: UIControlStateNormal];
        [_downloadService pause];
    }
}

- (IBAction)cancelDownloadClicked:(id)sender {
    [_downloadButton setTitle:@"download" forState: UIControlStateNormal];
    [_downloadService cancel];
}

这一部分是在上层业务的调用。暂停或取消上传时均能从failure回调中获取到checkpoint,重启下载时可以将checkpoint传到DownloadRequest，然后DownloadService内部会使用checkpoint做一致性的校验。

关于安卓里面实现对OSS Object的断点下载可以参考 基于okhttp3实现的断点下载功能实现 这个开源工程。这里列出如何使用这个示例工程下载OSS资源。

//1. 首先使用sdk获取object的下载链接
String signedURLString = ossClient.presignConstrainedObjectURL(bucket, object, expires);

//2. 添加下载任务

        mDownloadManager = DownloadManager.getInstance();
        mDownloadManager.add(signedURLString, new DownloadListner() {
            @Override
            public void onFinished() {
                Toast.makeText(MainActivity.this, "下载完成!", Toast.LENGTH_SHORT).show();
            }

            @Override
            public void onProgress(float progress) {
                pb_progress1.setProgress((int) (progress * 100));
                tv_progress1.setText(String.format("%.2f", progress * 100) + "%");
            }

            @Override
            public void onPause() {
                Toast.makeText(MainActivity.this, "暂停了!", Toast.LENGTH_SHORT).show();
            }

            @Override
            public void onCancel() {
                tv_progress1.setText("0%");
                pb_progress1.setProgress(0);
                btn_download1.setText("下载");
                Toast.makeText(MainActivity.this, "下载已取消!", Toast.LENGTH_SHORT).show();
            }
        });
        
//3.开启下载
        mDownloadManager.download(signedURLString);
        
//4.暂停下载
        mDownloadManager.cancel(signedURLString);
        
//5.继续下载
        mDownloadManager.download(signedURLString);

改进

HTTP1.1里的If-Range,我们来看针对该header的描述:

If-Range HTTP 请求头字段用来使得 Range 头字段在一定条件下起作用：当字段值中的条件得到满足时，Range 头字段才会起作用，同时服务器回复206 部分内容状态码，以及Range 头字段请求的相应部分；如果字段值中的条件没有得到满足，服务器将会返回 200 OK 状态码，并返回完整的请求资源。

字段值中既可以用 Last-Modified 时间值用作验证，也可以用ETag标记作为验证，但不能将两者同时使用。

If-Range 头字段通常用于断点续传的下载过程中，用来自从上次中断后，确保下载的资源没有发生改变。

使用If-Range的优点：客户端只需要一次网络请求,而前面所讲的If-Unmodified-Since或者If-Match在条件判断失败时，会返回412前置条件检查失败状态码，客户端不得不再开启一个请求来获取资源。

目前OSS Server还不支持If-Range字段。而在iOS中使用NSURLSessionDownloadTask实现断点下载时会发送If-Range给服务器以确认文件是否发生过变化。所以目前还不能使用NSURLSessionDownloadTask去实现断点下载。

参考

https://developer.mozilla.org/zh-CN/docs/Web/HTTP/Status/412

https://help.aliyun.com/document_detail/31856.html

https://tools.ietf.org/html/rfc2616