阿里云OSS单文件断点续传+前端 简单展示(springmvc架构)

业务没有需要多文件一起上传,所以这里只是单文件,多文件的话也是在获得File的地方变成List即可,多个循环,多一些线程,网上有代码

一、pom.xml


    <dependencys>
        <dependency>
           <groupId>com.aliyun</groupId>
            <artifactId>aliyun-java-sdk-core</artifactId>
            <version>3.0.8</version>
        </dependency>
        <dependency>
            <groupId>com.aliyun.oss</groupId>
            <artifactId>aliyun-sdk-oss</artifactId>
            <version>2.2.0</version>
        </dependency>
    </dependencys>

<!-- 阿里云sdk  -->
    <repositories>
        <repository>
            <id>sonatype-nexus-staging</id>
            <name>Sonatype Nexus Staging</name>
            <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>

二、前端(简单写,不是很严谨)

“`

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<script type="text/javascript" src="js/jquery-1.11.0.js"></script>
<script type="application/javascript">

    //标识,如果为true则每3秒访问一次进度
    var timeout = false;
    function progress(){
        if(!timeout) return;
        $.post("u_test.do",function(f){
            if(null == f){
                $("#jindu").val("上传错误");
            }else{
                $("#jindu").val(f+"%");
            }
        })
        setTimeout(progress,3000);
    }

    $(function(){
        $("#upload").click(function(){
            var uploadUrl = "upload.do";

            callback = function(res) {
                timeout = false;
                $("#d").append("上传结束" + res);
                $("#jindu").val("100%");
            };
            uploading = function(res) {
                $("#d").append(res+",");
                timeout = true;
                $("#jindu").val("1%");
                setTimeout(progress,3000);
            };
            beforeSend = function() {
                $("#d").append("开始");
            };

            var file = document.getElementById("myUpload-input").files[0];
            if(beforeSend instanceof Function){
                if(beforeSend(file) === false){
                    return false;
                }
            }

            var fd = new FormData();
            fd.append("file", file);
            var xhr = new XMLHttpRequest();
            xhr.onreadystatechange = function() {
                if (xhr.readyState == 4 && xhr.status == 200) {
                    if(callback instanceof Function){
                        callback(xhr.responseText);
                    }
                }
            }

            //侦查当前附件上传到TOMCAT的缓存情况,并不是真正传到了OSS中。如果是个大文件(2G+),这个过程也比较慢,有个初始化的进度条也可以
            xhr.upload.onprogress = function(evt) {
                var loaded = evt.loaded;
                var tot = evt.total;
                var per = Math.floor(100 * loaded / tot); //已经上传的百分比
                if(uploading instanceof Function){
                    uploading(per);
                }
            };

            xhr.open("post", uploadUrl);
            xhr.send(fd);
        })
    });

</script>
</head>
<body>
    //如果能得到文件的url,后端直接就使用该url即可,也方便,file不用转来转去
    <!--请输入url或文件路径:<input name="url" value="" id="url" width="200px"/><br>-->

    <input type="file" id="myUpload-input" name="file"/><br>
    <input type="button" value="点击上传" id="upload"/>

    <div id="d">存入缓存的进度:</div>
    上传的进度:<input id="jindu" value="0" />
</body>
</html>

前端效果

因为网络很慢,所以一片的大小尽管只有5M,上传还是挺慢


后端代码,注意我配置的springmvc是拦截.do路径,继承了一个创建OSSClient的工具类,在后面贴出

package xxxx

import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSSClient;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.*;
import com.market.web.controller.NewsControllerApi;
import org.apache.commons.fileupload.disk.DiskFileItem;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.multipart.commons.CommonsMultipartFile;

import java.io.*;
import java.util.*;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

/**
 * @author
 * @create 2016-10-31 19:13
 * @others 阿里云OSS测试
 */
@Controller
public class MultipartUpload extends OSSConfig {
    //日志记录
    private static final Logger LOG = LoggerFactory.getLogger(MultipartUpload.class);

    //获取配置文件
    ResourceBundle resourceBundle = ResourceBundle.getBundle("config");

    //默认5个线程
    private static ExecutorService executorService = Executors.newFixedThreadPool(5);

    //存储OSS返回来的partETags,是一个线程同步的arraylist,靠这个去查询
    private static List<PartETag> partETags = Collections.synchronizedList(new ArrayList<PartETag>());

    //如果上传完毕,uploadId则失效,这里将client设为null,则进度查询会显示完毕(不是必须)
    private static OSSClient client = null;

    // 创建一个要存放的Object的名称,使用文件名怕重复,直接改名+存入数据库
    private static String key;

    //默认一个静态的字段贮备,查询进度需要
    private static String uploadId;

    //默认一个静态的字段储备文件分割的数量,进度查询需要
    private static Integer partCount;

    //标识符,false情况下说名还没有开始,true则前端可以开始查询进度(不是必须)
    private boolean progress = false;

    // 每片最少5MB,这里写死了,最好用配置文件随时修改
    final long partSize = 5 * 1024 * 1024L;

    @ResponseBody
    @RequestMapping("upload")
    public void upload(@RequestParam(value = "file", required = false) MultipartFile mFile)
            throws IOException {
        //随便写一个文件名,这个是要和数据库关联的
        key = "time-1121";

        // MultipartFile转File,springmvc得到MultipartFile非常简单
        CommonsMultipartFile cf= (CommonsMultipartFile)mFile;
        DiskFileItem dFile = (DiskFileItem)cf.getFileItem();
        File file = dFile.getStoreLocation();

        // 构造OSSClient
        client = new OSSClient(endpoint, accessKeyId, accessKeySecret);

        try {
            // 1-获得uploadId
            uploadId = getUploadId(client, bucketName, key);

            // 2-断点续传分片
            partCount = getPartCount(file);

            // 3-开始配置子线程,开始上传多片文件到自己的bucket(oss的文件夹)中
            for (int i = 0; i < partCount; i++) {
                long startPos = i * partSize;
                long curPartSize = (i + 1 == partCount) ? (file.length() - startPos) : partSize;
                executorService.execute(new PartUploader(file, startPos, curPartSize, i + 1, uploadId));
            }

            // 4-方法执行中(只是步骤展示,这里没有代码)

            // 5-将标识符设为true,前端开始可以访问到上传进度
            progress = true;

            // 6-等待所有的线程上传完毕后关闭线程
            executorService.shutdown();
            while (!executorService.isTerminated()) {
                try {
                    //如果有线程没有完毕,等待5秒
                    executorService.awaitTermination(5, TimeUnit.SECONDS);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }

            // 7-判断是否所有的分片都上传完毕(不是必须)
            // judgePartCount();

            // 8-必须:在将所有数据Part都上传完成后对整个文件的分片验证。
            verification();

            // 9-成功,恢复初始值(恢复初始值是为了不让这次上传影响其他的文件上传)
            restoreDefaultValues();

            // 10-下载上传的文件(不是必须)
            //client.getObject(new GetObjectRequest(bucketName, key), new File(localFilePath));

        } catch (OSSException oe) {
            System.out.println("OSS错误原因:");
            System.out.println("Error Message: " + oe.getErrorCode());
            System.out.println("Error Code:       " + oe.getErrorCode());
            System.out.println("Request ID:      " + oe.getRequestId());
            System.out.println("Host ID:           " + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("OSSClient错误原因:");
            System.out.println("Error Message: " + ce.getMessage());
        } finally {
            //注意关闭资源
            if (client != null) {
                client.shutdown();
            }
        }
    }

    /**
     * 获得uploadId
     */
    public String getUploadId(OSSClient client, String bucketName, String key){
        InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(bucketName, key);
        InitiateMultipartUploadResult result = client.initiateMultipartUpload(request);
        return result.getUploadId();
    }

    /**
     * 断点续传分片数量判断
     */
    public int getPartCount(File file){
        long fileLength = file.length();
        int partCount = (int) (fileLength / partSize); // 分片个数,不能超过10000
        if (fileLength % partSize != 0) {
            partCount++;
        }
        if (partCount > 10000) {
            throw new RuntimeException("分片不能超过10000,请修改每片大小");
        } else {
            return partCount;
        }
    }

    /**
     * 子线程上传(多线程的上传效果提升明显,如果单线程则在for循环中一个个上传即可)
     */
    private static class PartUploader implements Runnable {
        private File localFile;
        private long startPos;
        private long partSize;
        private int partNumber;
        private String uploadId;
        public PartUploader(File localFile, long startPos, long partSize, int partNumber, String uploadId) {
            this.localFile = localFile;
            this.startPos = startPos;
            this.partSize = partSize;
            this.partNumber = partNumber;
            this.uploadId = uploadId;
        }
        @Override
        public void run() {
            InputStream instream = null;
            try {
                instream = new FileInputStream(this.localFile);
                instream.skip(this.startPos);

                UploadPartRequest uploadPartRequest = new UploadPartRequest();
                uploadPartRequest.setBucketName(bucketName);
                uploadPartRequest.setKey(key);
                uploadPartRequest.setUploadId(this.uploadId);
                uploadPartRequest.setInputStream(instream);
                uploadPartRequest.setPartSize(this.partSize);
                uploadPartRequest.setPartNumber(this.partNumber);

                UploadPartResult uploadPartResult = client.uploadPart(uploadPartRequest);
                synchronized (partETags) {
                    partETags.add(uploadPartResult.getPartETag());
                }
            } catch (Exception e) {
                e.printStackTrace();
            } finally {
                if (instream != null) {
                    try {
                        instream.close();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
            }
        }
    }

    /**
     * 判断是否上传完毕,这里起始需要得到哪个错了,从哪个开始重新上传,获得part#,然后计算5M的大小上传
     * 这里只是单纯的记录一下,以后再完善
     * @return
     */
    public void judgePartCount(){
        if (partETags.size() != partCount) {
            System.out.println("一部分没有上传成功,需要从ListMultipartUploads里面查看上传数量");
            throw new IllegalStateException("Upload multiparts fail due to some parts are not finished yet");
        } else {
            System.out.println("上传成功,文件名称:" + key + "\n");
        }
    }

    /**
     * 验证文件是否完全,OSS必须走这个方法
     * 最后返回CompleteMultipartUploadResult,这里不继续深化了
     * @return
     */
    public void verification(){
        // 修改顺序
        Collections.sort(partETags, new Comparator<PartETag>() {
            @Override
            public int compare(PartETag p1, PartETag p2) {
                return p1.getPartNumber() - p2.getPartNumber();
            }
        });
        System.out.println("开始验证\n");
        CompleteMultipartUploadRequest completeMultipartUploadRequest =
                new CompleteMultipartUploadRequest(bucketName, key, uploadId, partETags);
        client.completeMultipartUpload(completeMultipartUploadRequest);
    }

    /**
     * 恢复初始值
     * @return
     */
    public void restoreDefaultValues(){
        executorService = Executors.newFixedThreadPool(5);
        partETags = Collections.synchronizedList(new ArrayList<PartETag>());
        client = null;
        uploadId = null;
        partCount = 0;
    }

    /**
     * 页面上进度查询走这个方法
     * @return
     */
    @ResponseBody
    @RequestMapping("u_test")
    public Object u_test(){
        //获得OSS上面的已经好了的分片和总分片相除获得进度
        ListPartsRequest listPartsRequest = new ListPartsRequest(bucketName, key, uploadId);
        PartListing partListing;
        try {
            partListing = client.listParts(listPartsRequest);
            Integer nowPartCount = partListing.getParts().size();
            return (int)((double)nowPartCount/partCount * 100);
        }catch (Exception e){
            //如果标识符为true说明已经开始上传,设为100%;否则说明刚进入方法,还没有上传,设为1%
            if(progress)
                return "100";
            else
                return "1";
        }
    }
}

继承的父类

import com.aliyun.oss.OSSClient;
import com.aliyun.oss.model.Bucket;

/**
 * @author
 * @create 2016-10-31 15:05
 * @others 存放存储上传的信息
 */
public class OSSConfig {

    // endpoint为访问域名
    // 可以使用OSS域名、自定义域名(CNAME)、IP、STS作为endpoint
    // STS(介绍:https://help.aliyun.com/document_detail/31931.html?spm=5176.doc32010.2.6.u4i4ck)
    // 访问域名对照中心可参看;https://help.aliyun.com/document_detail/31837.html?spm=5176.doc32010.2.1.EwrB1F
    // 此处endpoint以杭州为例,其它region请按实际情况填写:http://oss-cn-hangzhou.aliyuncs.com

    protected static String endpoint = "oss-cn-hangzhou.aliyuncs.com";

    // accessKey请登录https://ak-console.aliyun.com/#/查看
    // accessKeyId和accessKeySecret为申请的秘钥

    protected static String accessKeyId = "";
    protected static String accessKeySecret = "";

    // 随机生成一个15位的BucketName,相当于命名空间(容器)的名称 "mybucket"+RandomUtils.getNumRandom(15)
    protected static String bucketName = "";

    protected static String localFilePath = "文件下载位置";

    // 创建OSSClient实例
    protected static OSSClient ossClient = new OSSClient(endpoint, accessKeyId, accessKeySecret);


    /**
     * 创建一个Bucket 存储空间
     * @param buckets 创建Bucket的名称
     * @return 创建的Bucket对象
     */
    public Bucket createBucket(String buckets){
        Bucket bucket = ossClient.createBucket(buckets);
        // 关闭client
        ossClient.shutdown();
        return bucket;
    }

}

然后跑起来试试看吧

  • 3
    点赞
  • 11
    收藏
    觉得还不错? 一键收藏
  • 8
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 8
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值