问题经过
公司新分了一个项目,项目中使用OSS做文件存储;CTO考虑说:为了不浪费后端网络带宽,文件的上传下载由web端和app端直接对接OSS,并且要支持断点续传和断点下载,服务端来提供临时访问令牌。
Coder:没问题,保证完成任务!
问题1
Web端访问OSS,由于不在同域下,报出异常如下:
<Code>AccessForbidden</Code>
<Message>
CORSResponse: This CORS request is not allowed. This is usually because the evalution of Origin,
request method Access-Control-Request-Method or Access-Control-Requet-Headers are not whitelisted by the resource's CORS spec.
</Message>
需要配置OSS跨越
这里是dev环境所对应的bucket设置的跨域配置
OSS详细跨域配置链接:
阿里云OSS跨域资源共享
问题2
随着时间的流逝,我们的项目经历了N轮的测试,终于要发布演示(stage)环境了。这个时候app端出现了问题:
<Error>
<Code>AccessDenied</Code>
<Message>Access denied by authorizer's policy.</Message>
<RequestId>62EB5D472C1E93303271BA2F</RequestId>
<HostId>xxx-test.oss-cn-hangzhou.aliyuncs.com</HostId>
</Error>
这个问题大概意思是 授权人的策略拒绝访问
,定睛一看,这是说STS的权限策略有问题呀!但是这个权限策略是我司资深运维配置的怎么会有问题呢?(不知道怎么配置的小伙伴点这个链接)
先检查后端生成临时生成token的代码,发现没有问题。。。。。
就在这时,发现一个问题!为什么我们配置的bucket名称是xxx-stage,但是异常抛的却是xxx-test呢?
那么重点来了:刚才文章头部说了,需要支持断点下载。
那我们走OSS的下载方法就是这个:
private DownloadFileResult downloadFileWithCheckpoint(DownloadFileRequest downloadFileRequest) throws Throwable {
DownloadFileResult downloadFileResult = new DownloadFileResult();
DownloadCheckPoint downloadCheckPoint = new DownloadCheckPoint();
// The checkpoint is enabled, downloads the parts download results from
// checkpoint file.
if (downloadFileRequest.isEnableCheckpoint()) {
// read the last download result. If checkpoint file dosx not exist,
// or the file is updated/corrupted,
// re-download again.
try {
downloadCheckPoint.load(downloadFileRequest.getCheckpointFile());
} catch (Exception e) {
remove(downloadFileRequest.getCheckpointFile());
}
// The download checkpoint is corrupted, download again.
if (!downloadCheckPoint.isValid(objectOperation, downloadFileRequest)) {
prepare(downloadCheckPoint, downloadFileRequest);
remove(downloadFileRequest.getCheckpointFile());
}
} else {
// The checkpoint is not enabled, download the file again.
prepare(downloadCheckPoint, downloadFileRequest);
}
// Progress listen starts tracking the progress.
ProgressListener listener = downloadFileRequest.getProgressListener();
ProgressPublisher.publishProgress(listener, ProgressEventType.TRANSFER_STARTED_EVENT);
// Concurrently download parts.
DownloadResult downloadResult = download(downloadCheckPoint, downloadFileRequest);
Long serverCRC = null;
for (PartResult partResult : downloadResult.getPartResults()) {
if (partResult.getServerCRC() != null) {
serverCRC = partResult.getServerCRC();
}
if (partResult.isFailed()) {
ProgressPublisher.publishProgress(listener, ProgressEventType.TRANSFER_PART_FAILED_EVENT);
throw partResult.getException();
}
}
// check crc64
if(objectOperation.getInnerClient().getClientConfiguration().isCrcCheckEnabled() &&
!hasRangeInRequest(downloadFileRequest)) {
Long clientCRC = calcObjectCRCFromParts(downloadResult.getPartResults());
try {
OSSUtils.checkChecksum(clientCRC, serverCRC, downloadResult.getObjectMetadata().getRequestId());
} catch (Exception e) {
ProgressPublisher.publishProgress(listener, ProgressEventType.TRANSFER_FAILED_EVENT);
throw new InconsistentException(clientCRC, serverCRC, downloadResult.getObjectMetadata().getRequestId());
}
}
// Publish the complete status.
ProgressPublisher.publishProgress(listener, ProgressEventType.TRANSFER_COMPLETED_EVENT);
// rename the temp file.
renameTo(downloadFileRequest.getTempDownloadFile(), downloadFileRequest.getDownloadFile());
// The checkpoint is enabled, delete the checkpoint file after a
// successful download.
if (downloadFileRequest.isEnableCheckpoint()) {
remove(downloadFileRequest.getCheckpointFile());
}
downloadFileResult.setObjectMetadata(downloadResult.getObjectMetadata());
return downloadFileResult;
}
这都是什么玩意?看不懂!
仔细阅读一下我们发现这一行代码有点意思downloadCheckPoint.load(downloadFileRequest.getCheckpointFile())
这是说加载检查点,那我们点进去看一下发现:
public synchronized void load(String cpFile) throws IOException, ClassNotFoundException {
FileInputStream fileIn = new FileInputStream(cpFile);
ObjectInputStream in = new ObjectInputStream(fileIn);
DownloadCheckPoint dcp = (DownloadCheckPoint) in.readObject();
assign(dcp);
in.close();
fileIn.close();
}
有点意思,这是从本地加载文件继续下载的位置,但是你加载位置就加载位置你怎么把我的bucketName也给加载进来了,问题找到了:当我们使用断点续传下载时,OSS会在我们本地文件夹生成.dcp文件,OSS以文件流的方式将该文件读取出来,
然后将dcp文件传入assign(dcp)中,代码如下:
可以看到这里将dcp.buketName赋值给了变量bucketName,所以我们在下载文件时传进来的bucketName被覆盖了,问题解决!将本地下载路径中的同名dcp文件删除,下载成功!
总的来说,我们在test环境切stage环境时,App端的文件下载路径是同一个,恰巧在test环境时文件未下载完,本地的dcp文件未被删除,切换至stage环境后下载相同名称的文件时,发现指定路径下已经存在了该文件的dcp文件,就将该dcp文件加载进来了,然后将dcp中的bucketName赋值给了当前bucketName;这时拿着stage环境获取到的sts令牌就被拒绝了。
好了,感谢大家能看到这里,文章结束了,如果上述内容有问题的话,欢迎大家指正!