vue+springboot实现大文件分片上传、断点续传

一、前言

前一时间没事自己做了个个人网盘小项目，中间遇到大文件分片上传的问题，第一次解决还是比较坎坷，这里记录下我的实现原理及过程。

效果图：

二、思路

VUE前端

1. 选择要上传的文件
2. 计算选择文件的md5信息
3. 调用文件创建接口将文件名、大小、md5等信息传给后台，用来校验文件是否已经上传，如果之前已经上传完成过，这里就停止了
4. 如果没上传将选择的文件按提前设定好大小分片
5. 得到分反的文件，循环依次调用 文件上传接口上传，片段文件上传成功后，接口会返回已经上传的大小
6. 根据已经传文件的大小来判断是否上传成功，并跳出循环结束上传

springboot服务端

1. 文件创建接口：

• 根据md5判断文件记录是否创建，如果已经创建状态文件信息返回给前端，其中包含是否已经上传成功，如果没有创建，在表中创建一第文件记录，记下件名、大小、md5等信息，然后再返回。

2. 文件上传接口：

• 查询文件是否创建记录，如果已创建并记录状态是上传完成，结束。
• 查询文件上传的缓存文件是否存在，如果存在判断缓存文件的大小，并直接返回，前端接口这个大小再改为这个长度开始上传（实现断点续传的功能）
• 每次上传的分文件都追加到缓存的文件中，当缓存文件的长度和表中记录的总上度一样且md5值相等，证明文件上传完成了，这里将缓存文件重命名就算完成了

三、代码

前端我用的是vue+elementui组件做的

<template>
    <div style="width: 680px; padding:0 10px 0 10px;">
        <el-upload
            v-if="uploadIf"
            class="upload-demo"
            drag
            multiple
            action="123"
            :limit="50"
            :show-file-list="false"
            :http-request="myUpload"
            :on-exceed="handleExceed"
        >
            <i class="el-icon-upload"></i>
            <div class="el-upload__text">
                将文件拖到此处，或
                <em>点击上传</em>
            </div>
            <div class="el-upload__tip" slot="tip" style="font-size:12px;font-weight:400;color:red">
                提示：<br/>
                1.支持断点续传，最多一次上传50个文件<br/>
                2.中断的任务可在缓存列表里查看, 成功的任务在资源列表里查看<br/>
                3.同一资源不同名称只能上传一次
            </div>
        </el-upload>

        <el-table class="table" :data="uploadData" style="width: 100%;" :max-height="tableHeight" v-if="!uploadIf">
            <el-table-column prop="name" label="名称" show-overflow-tooltip></el-table-column>
            <el-table-column prop="size" label="大小" width="100">
                <template slot-scope="scope">{{ scope.row.size | getSize }} </template>
            </el-table-column>
            <el-table-column label="进度" width="180">
                <template slot-scope="scope">
                    <el-progress :percentage="scope.row.progress" >{{scope.row.progress}}</el-progress>
                </template>
            </el-table-column>
            <el-table-column label="速度" width="100">
                <template slot-scope="scope">{{scope.row.progress==100?'完成':scope.row.speed}}</template>
            </el-table-column>
        </el-table>
    </div>
</template>

<script>
import {formatFileSize} from '../util/common-util'
import SparkMD5 from "spark-md5";

export default {
    name: "ResourceUpload",
    props: {
        pid: {
            type: Number,
            required: true
        }
    },
    data() {
        return {
            uploadData: [],
            eachSize: 1 * 1024 * 1024,
            maxSize: 2048 * 1024 * 1024,
            uploadIf:true,
            tableHeight:500,
            uploadStop:false,
        };
    },
    created(){
        this.computeHeight();
    },
    methods: {
        uploadStore(){
            this.$store.commit('resource/setUploadList', this.uploadData);
        },
        handleExceed(){
            this.$notify.error({title: '操作失败',message: '文件个数超出最大限制50'});
        },
        async myUpload(params) {
            this.uploadIf = false;
            //console.log("开始上传...");
            const file = params.file;
            
            const { eachSize,uploadData,maxSize } = this;
            if(file.size > maxSize){
                this.$message.error('文件《'+file.name+'》已超2GB，禁止上传');
                return;
            }
            var selectFile = {
                id: 0,
                pid: this.pid,
                state:0,
                name:file.name,
                size:file.size,
                progress: 0,
                speed: '计算md5',
                speedStart: 0,
                speedEnd: 0
            }
            uploadData.push(selectFile);
            this.uploadStore();
            let fileMd5 = await this.calculateMd5(file);
            selectFile['md5'] = fileMd5;
            //计算速度
            setInterval(()=>{
                var speed = formatFileSize(selectFile.speedEnd - selectFile.speedStart);
                selectFile.speed = speed + '/秒';
                selectFile.speedStart = selectFile.speedEnd;
                if(selectFile.progress == 100 || this.uploadStop){
                    return;
                }
            }, 1000);
            //看之前有没有上传过
            await this.$api.post('resource/create', selectFile).then( data => {
                selectFile.id = data.id;  
                selectFile.state = data.state;
            })
            if(selectFile.state == 1){
                selectFile.progress = 100;
                return;
            }
            //开始分片上传
            for (let startSize = 0; ; ) {
                if(this.uploadStop){
                    break;
                }
                const chunkFile = file.slice(startSize, startSize + eachSize);
                const formData = new FormData();
                formData.append("file", chunkFile);
                formData.append("id", selectFile.id);
                formData.append("startSize", startSize);
                startSize = await this.$api({
                    url: "resource/upload",
                    method: "post",
                    data: formData,
                    onUploadProgress: e => {
                        let num = (((startSize + e.loaded) / file.size) * 100) | 0;
                        if(num > 100)num = 100;
                        selectFile.progress = num;
                        //计算速度用
                        selectFile.speedEnd = startSize + e.loaded;
                    }
                }).then(res => {
                    return res;
                });
                if (startSize >= file.size) {
                    selectFile.progress = 100;
                    this.uploadStore();
                    break;
                }
            }
        },
        calculateMd5(file) {
            return new Promise((resolve, reject) => {
                var fileReader = new FileReader(),
                    blobSlice =
                        File.prototype.mozSlice ||
                        File.prototype.webkitSlice ||
                        File.prototype.slice,
                    chunkSize = 5242880, //5MB
                    chunks = Math.ceil(file.size / chunkSize),
                    currentChunk = 0,
                    spark = new SparkMD5();
                fileReader.onload = function(e) {
                    spark.appendBinary(e.target.result); // append binary string
                    currentChunk++;
                    if (currentChunk < chunks) {
                        loadNext();
                    } else {
                        resolve(spark.end());
                    }
                };
                function loadNext() {
                    var start = currentChunk * chunkSize,
                        end =
                            start + chunkSize >= file.size
                                ? file.size
                                : start + chunkSize;

                    fileReader.readAsBinaryString(
                        blobSlice.call(file, start, end)
                    );
                }
                loadNext();
            });
        },
        computeHeight(){
            this.tableHeight = document.body.clientHeight*0.80;
        }
    },
    mounted() {
        window.onresize = () => {
            return (() => {
                this.computeHeight();
            })();
        }
    },
    beforeDestroy(){
        this.uploadStop = true;  
    },
    filters:{
        getSize:function (size) {
             return formatFileSize(size);
        },
    }
};
</script>

<style scoped>
.el-table {
    font-size: 12px;
}
</style>

接口

表

@Getter
@Setter
@Entity(name = "resource")
public class Resource {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String name;
    private String md5;
    private Long size;
    private String path;
    private String url;
    private String type;//音频|视频|软件|文件|图片|压缩包|文件夹
    private Integer state;//0未完成1已完成
    private Date createTime;
    private Long pid;
    private Integer deleted;
    private Double seq;

}

创建文件接口

@Getter
@Setter
public class ResourceCreateDTO {

    @ApiModelProperty("有md5就认为是文件")
    private String md5;
    private Long size;
    @NotBlank
    private String name;
    @NotNull
    private Long pid = 0L;

    public String getFileSuffix(){
        if(name.indexOf(".")>-1){
            return name.substring(name.lastIndexOf(".")+1);
        }
        return "";
    }

}

@Transactional
public Resource create(ResourceCreateDTO dto){
    if(StringUtils.isEmpty(dto.getMd5())){
        Resource resource = new Resource();
        resource.setName(dto.getName());
        resource.setState(1);
        resource.setCreateTime(new Date());
        resource.setPid(dto.getPid());
        resource.setType("0");
        resource.setSize(0L);
        resource.setDeleted(0);
        resource.setSeq(9999d);
        resourceDAO.save(resource);
        return resource;
    }else {
        Resource resource = resourceDAO.findByMd5(dto.getMd5());
        if(resource == null){
            //判断文件有没有超最大值
            validSize(dto.getSize());
            //判断有没有超容量
            validTotalSize(dto.getSize());
            String fileSuffix = dto.getFileSuffix();
            resource = new Resource();
            resource.setMd5(dto.getMd5());
            resource.setName(dto.getName());
            resource.setPath(dto.getName());
            resource.setSize(dto.getSize());
            resource.setUrl(ConfigService.getResourceRootUrl()+dto.getName());
            resource.setState(0);
            resource.setCreateTime(new Date());
            resource.setPid(dto.getPid());
            resource.setType(fileSuffix.toLowerCase());
            resource.setDeleted(0);
            resource.setSeq(0d);
            resourceDAO.save(resource);
            return resource;
        }else{
            if(resource.getDeleted()==1){ //如果之前删除，但还没有清除数据，可以再启用
                resource.setDeleted(0);
                resource.setPid(dto.getPid());
                resource.setName(dto.getName());
                resource.setCreateTime(new Date());
            }
            return resource;
        }
    }
}

分片上传接口

@Getter
@Setter
public class ResourceUploadDTO {

    @NotNull
    private Long startSize;
    private MultipartFile file;
    @NotNull
    private Long id;

}


@Transactional
public Long upload(ResourceUploadDTO dto) {
    Resource resource = getById(dto.getId());
    if(resource.getState()==1){
        return resource.getSize();
    }
    String resourceUploadPath = SystemConfigService.getResourceDiskPath();
    File tempFolder = new File(resourceUploadPath);
    //如果文件存在
    File file = new File(tempFolder,resource.getPath());
    if(file.exists()){
        return file.length();
    }
    //如果缓存不存在，或缓存=开始上传的大小，否则就给当前缓存的大小让他重新上传
    File tempFile = new File(tempFolder,"temp/"+resource.getPath()+".temp");
    Long tempLen = tempFile.length(); // 缓存文件的大小
    if(!tempFile.exists() || tempLen.longValue() == dto.getStartSize().longValue()){
        FileUtil.append(tempFile,dto.getFile());
    }else {
        return tempLen;
    }
    //上传后的大小更新
    tempLen = tempFile.length();
    //如果上传完成，重名命名该文件，且更新记录
    if(resource.getSize().longValue() == tempLen.longValue()){
        tempFile.renameTo(new File(tempFolder,resource.getPath()));
        resource.setState(1);
    }
    return tempLen;
}

FileUtil工具类方法

public static void append(File file, MultipartFile multipartFile) {
 	try {
        append(file,multipartFile.getInputStream());
    } catch (IOException e) {
        throw new APIException(APICode.INTERNAL_SERVER_ERROR,"追加文件出错"+e.getMessage());
    }
}

public static void append(File file, InputStream inputStream) {
    BufferedOutputStream outputStream = null;
    FileOutputStream fileOutputStream = null;
    try {
        int bufSize = 1024;
        fileOutputStream = new FileOutputStream(file,true);
        outputStream = new BufferedOutputStream(fileOutputStream);
        byte[] buffer = new byte[bufSize];
        int temp;
        while ((temp = inputStream.read(buffer)) > 0) {
            outputStream.write(buffer, 0, temp);
        }
        if(inputStream!=null)inputStream.close();
        outputStream.flush();
    } catch (Exception e) {
        e.printStackTrace();
        throw new APIException(APICode.INTERNAL_SERVER_ERROR,"追加文件出错"+e.getMessage());
    }finally {
        try{
            if(fileOutputStream!=null)fileOutputStream.close();
            if(inputStream!=null)inputStream.close();
            if(outputStream!=null)outputStream.close();
        }catch (IOException e){

        }
    }
}

原创文章未经本人许可，不得用于商业用途及传统媒体。转载请注明出处，否则属于侵权行为，谢谢合作！