android 集成阿里云oss 大文件上传分片模式 sdk遇到的坑

版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://blog.csdn.net/qq793318828/article/details/87778615

最近集成了阿里的oss 上传大文件sdk .不知道你们有没有遇到了 上传超大文件可能是个损坏的文件 ,我这边是遇到了这个坑。通过思考 可能是由于以下几个原因导致,以下为个人见解 如果不对欢迎指出 :

  1. token 过期
  2. 网络原因导致中断(分片模式下应该不会存在这个问题。因为最终会调用 CompleteMultipartUploadRequest
    方法合并分片 )
  3. 部分分片上传失败

个人感觉应该1跟3的可能性会大一些。所以针对这个我进行了如下修改 首先 应该实例个 OSSFederationCredentialProvider 对象。oss 文档上说明了OSSFederationCredentialProvider 会在token过期后重新调用url 获取参数 ,如下为初始化oss 代码

OSSCredentialProvider provider = new OSSFederationCredentialProvider() {
            @Override
            public OSSFederationToken getFederationToken() throws ClientException {

                OSSFederationToken authToken = null;
                String authData;
                try {
                    URL stsUrl = new URL(“获取授权url”);
                    HttpURLConnection conn = (HttpURLConnection) stsUrl.openConnection();
                    conn.setConnectTimeout(10000);
                    InputStream input = conn.getInputStream();
                    authData = IOUtils.readStreamAsString(input, OSSConstants.DEFAULT_CHARSET_NAME);
                    JSONObject jsonObj = new JSONObject(authData);
                    /**
                    解析服务端返回数据结构。然后赋值到OSSFederationToken 对象中返回
                    */
                    return authToken;
                } catch (Exception e) {
                    throw new ClientException(e);
                }
            }
        };

        //  OSSCredentialProvider provider = new OSSStsTokenCredentialProvider(oosBean.getCredentials().getAccessKeyId(), oosBean.getCredentials().getAccessKeySecret(), oosBean.getCredentials().getSecurityToken());
        //该配置类如果不设置,会有默认配置,具体可看该类
        ClientConfiguration conf = new ClientConfiguration();
        conf.setConnectionTimeout(15 * 1000); // 连接超时,默认15秒
        conf.setSocketTimeout(15 * 1000); // socket超时,默认15秒
        conf.setMaxConcurrentRequest(5); // 最大并发请求数,默认5个
        conf.setMaxErrorRetry(2); // 失败后最大重试次数,默认2次
        return new OSSClient(MyApplication.getInstance().getContext(), endpoint, provider);

getFederationToken 方法里面是调用授权url . 可以自行解析你们服务器返回的结构

oss对象使用OSSFederationCredentialProvider ,可以解决了token过期然后导致文件无法上传的问题
然后是第三个问题。部分分片上传失败的问题。首先我遇到这个问题的时候 在分片上传的时候增加了个判断

  partETags = new ArrayList<PartETag>();
        while (uploadedLength < mFileLen) {
            int partLength = (int) Math.min(partSize, mFileLen - uploadedLength);
            byte[] partData = IOUtils.readStreamAsBytesArray(input, partLength);
            UploadPartRequest uploadPart = new UploadPartRequest(BucketName, _objKey + ".mp4", uploadId, currentIndex);
            uploadPart.setPartContent(partData);
            UploadPartResult uploadPartResult = oss.uploadPart(uploadPart);
            if (uploadPartResult.getStatusCode() != 200) {
                if (mUploadEventListener != null) {
                    mUploadEventListener.onErr(0, mVid);
                    return;
                }
            }
            partETags.add(new PartETag(currentIndex, uploadPartResult.getETag()));
            uploadedLength += partLength;
            if (mUploadEventListener != null) {
                mUploadEventListener.onFilePtrChanged(uploadedLength, mFileLen, mVid);
            }
            currentIndex++;
        }
        input.close();

按照正常逻辑来讲 uploadPartResult 是当前分片上传结果的返回。如果uploadPartResult.getStatusCode() == 200 的话 应该是这个分片上传成功了。可是不知道是什么原因,我这边遇到了所有的uploadPartResult.getStatusCode() = 200 但是上传的文件大小跟源文件是不一样的。因此我增加了个最后的判断 获取oss 上当前obj 的所有分片。跟本地存储的分片tag 进行对比。判断下是否都一致,如果不一致将上传这个分片

private boolean listAllParts(String uploadId, String _objKey, int totalSize) throws ClientException, ServiceException, IOException {
        ListPartsRequest listPartsRequest = new ListPartsRequest(BucketName, _objKey, uploadId);
        ListPartsResult listPartsResult = oss.listParts(listPartsRequest);
        List<PartSummary> parts = listPartsResult.getParts();
        int count = parts.size();
        if (parts.size() == totalSize) {
            if (partETags == null ) partETags = new ArrayList<>();
            partETags.clear();
            for (int i = 0; i < parts.size(); i++) {
                partETags.add(new PartETag(parts.get(i).getPartNumber(), parts.get(i).getETag()));
            }
            return true;
        }
        InputStream input = new FileInputStream(mLocalFile);
        Collections.sort(parts, new Comparator<PartSummary>() {
            @Override
            public int compare(PartSummary p1, PartSummary p2) {
                return p1.getPartNumber() - p2.getPartNumber();
            }
        });

        for (int i = 0; i < count; i++) {
            if (i == count - 1) {
                if (parts.get(i).getPartNumber() != totalSize) {
                    int index = 0;
                    byte[] partData = null;
                    long uploadedLength = 0;
                    while (index <= i) {
                        int partLength = (int) Math.min(partSize, mFileLen - uploadedLength);
                        partData = IOUtils.readStreamAsBytesArray(input, partLength);
                        uploadedLength += partLength;
                        index++;
                    }
                    UploadPartRequest uploadPart = new UploadPartRequest(BucketName, _objKey, uploadId, parts.get(i).getPartNumber() + 1);
                    uploadPart.setPartContent(partData);
                    UploadPartResult uploadPartResult = oss.uploadPart(uploadPart);
                    input.close();
                    return listAllParts(uploadId, _objKey, totalSize);
                }
            } else if (parts.get(i).getPartNumber() - parts.get(i + 1).getPartNumber() != -1) {
                int index = 0;
                byte[] partData = null;
                long uploadedLength = 0;
                while (index <= i) {
                    int partLength = (int) Math.min(partSize, mFileLen - uploadedLength);
                    partData = IOUtils.readStreamAsBytesArray(input, partLength);
                    uploadedLength += partLength;
                    index++;
                }
                UploadPartRequest uploadPart = new UploadPartRequest(BucketName, _objKey, uploadId, parts.get(i).getPartNumber() + 1);
                uploadPart.setPartContent(partData);
                UploadPartResult uploadPartResult = oss.uploadPart(uploadPart);
                input.close();
                return listAllParts(uploadId, _objKey, totalSize);
            }

        }

        return true;
    }

这是一个递归的方法判断。如果一致了将会返回true .然后就可以调用CompleteMultipartUploadRequest 啦

以上为个人见解。如有看法欢迎留言

展开阅读全文

没有更多推荐了,返回首页