七牛云实现文件上传下载（响应压缩包zip）

奥库甘道夫

已于 2022-11-12 17:26:24 修改

阅读量1.4k

点赞数 3

分类专栏：文件处理文章标签：云计算数据库运维 java

于 2022-11-10 17:54:14 首次发布

本文链接：https://blog.csdn.net/xzx18822942899/article/details/127793347

版权

文件处理专栏收录该内容

3 篇文章 0 订阅

订阅专栏

七牛云实现文件上传下载（响应压缩包zip）

1.介绍

在实际开发中，我们会有很多处理不同功能的服务器。例如：

应用服务器：负责部署我们的应用

数据库服务器：运行我们的数据库

文件服务器：负责存储用户上传文件的服务器

分服务器处理的目的是让服务器各司其职，从而提高我们项目的运行效率。

常见的图片存储方案：

方案一：使用nginx搭建图片服务器

方案二：使用开源的分布式文件存储系统，例如Fastdfs、HDFS等

方案三：使用云存储，例如阿里云、七牛云等

2 七牛云存储

七牛云（隶属于上海七牛信息技术有限公司）是国内领先的以视觉智能和数据智能为核心的企业级云计算服务商，同时也是国内知名智能视频云服务商，累计为 70 多万家企业提供服务，覆盖了国内80%网民。围绕富媒体场景推出了对象存储、融合 CDN 加速、容器云、大数据平台、深度学习平台等产品、并提供一站式智能视频云解决方案。为各行业及应用提供可持续发展的智能视频云生态，帮助企业快速上云，创造更广阔的商业价值。

官网：https://www.qiniu.com/

通过七牛云官网介绍我们可以知道其提供了多种服务，我们主要使用的是七牛云提供的对象存储服务来存储图片。

2.1 注册、登录

要使用七牛云的服务，首先需要注册成为会员。地址：https://portal.qiniu.com/signup

注册完成后就可以使用刚刚注册的邮箱和密码登录到七牛云：

登录成功后点击页面右上角管理控制台：

注意：登录成功后还需要进行实名认证才能进行相关操作。

2.2 新建存储空间

要进行图片存储，我们需要在七牛云管理控制台新建存储空间。点击管理控制台首页对象存储下的立即添加按钮，页面跳转到新建存储空间页面：

可以创建多个存储空间，各个存储空间是相互独立的。

2.3 查看存储空间信息

存储空间创建后，会在左侧的存储空间列表菜单中展示创建的存储空间名称，点击存储空间名称可以查看当前存储空间的相关信息

2.4 开发者中心

可以通过七牛云提供的开发者中心学习如何操作七牛云服务，地址：https://developer.qiniu.com/

点击对象存储，跳转到对象存储开发页面，地址：https://developer.qiniu.com/kodo

七牛云提供了多种方式操作对象存储服务，本项目采用Java SDK方式，地址：https://developer.qiniu.com/kodo/sdk/1239/java

使用Java SDK操作七牛云需要导入如下maven坐标：

<dependency>
  <groupId>com.qiniu</groupId>
  <artifactId>qiniu-java-sdk</artifactId>
  <version>[7.7.0, 7.10.99]</version>
</dependency>

2.5 鉴权

Java SDK的所有的功能，都需要合法的授权。授权凭证的签算需要七牛账号下的一对有效的Access Key和Secret Key，这对密钥可以在七牛云管理控制台的个人中心（https://portal.qiniu.com/user/key）获得，如下图：

2.6 Java SDK操作七牛云

本章节我们就需要使用七牛云提供的Java SDK完成图片上传和删除，我们可以参考官方提供的例子。

如果报error region 错误可以修改 new Configuration(Region.autoRegin());

//构造一个带指定 Region 对象的配置类
//如果报error region 错误 可以将 new Configuration(Region.autoRegin());
Configuration cfg = new Configuration(Region.region0());
cfg.resumableUploadAPIVersion = Configuration.ResumableUploadAPIVersion.V2;// 指定分片上传版本
//...其他参数参考类注释

UploadManager uploadManager = new UploadManager(cfg);
//...生成上传凭证，然后准备上传
String accessKey = "your access key";
String secretKey = "your secret key";
String bucket = "your bucket name";
//如果是Windows情况下，格式是 D:\\qiniu\\test.png
String localFilePath = "/home/qiniu/test.png";
//默认不指定key的情况下，以文件内容的hash值作为文件名
String key = null;

Auth auth = Auth.create(accessKey, secretKey);
String upToken = auth.uploadToken(bucket);

try {
    Response response = uploadManager.put(localFilePath, key, upToken);
    //解析上传成功的结果
    DefaultPutRet putRet = new Gson().fromJson(response.bodyString(), DefaultPutRet.class);
    System.out.println(putRet.key);
    System.out.println(putRet.hash);
} catch (QiniuException ex) {
    Response r = ex.response;
    System.err.println(r.toString());
    try {
        System.err.println(r.bodyString());
    } catch (QiniuException ex2) {
        //ignore
    }
}

//构造一个带指定Zone对象的配置类
Configuration cfg = new Configuration(Region.autoRegin());
//...其他参数参考类注释

String accessKey = "your access key";
String secretKey = "your secret key";

String bucket = "your bucket name";
String key = "your file key";

Auth auth = Auth.create(accessKey, secretKey);
BucketManager bucketManager = new BucketManager(auth, cfg);
try {
    bucketManager.delete(bucket, key);
} catch (QiniuException ex) {
    //如果遇到异常，说明删除失败
    System.err.println(ex.code());
    System.err.println(ex.response.toString());
}

2.7 封装工具类

为了方便操作七牛云存储服务，我们可以将官方提供的案例简单改造成工具类，在我们的项目中直接使用此工具类来操作就可以：

1.config 配置类

首先添加字段属性到配置文件（application.yml）中

# 七牛云
qiniu:
  accessKey: your accsessKey
  secretKey: your secretKey
  bucket: your namespace
  # 外联域名
  domain: your domain

/**
 * 七牛云配置类
 *
 * @author xzx
 * @date 2022/11/10
 */
@Configuration
public class QiniuConfig {

    @Value("${qiniu.accessKey}")
    private String accessKey;
    @Value("${qiniu.secretKey}")
    private String secretKey;

    @Bean(name = "uploadManager")
    public UploadManager uploadManager() {
        return new UploadManager(configuration());
    }

    @Bean(name="configuration")
    public com.qiniu.storage.Configuration configuration() {
        return new com.qiniu.storage.Configuration(Region.autoRegion());
    }

    @Bean(name = "auth")
    public Auth authConfiguration() {
        return Auth.create(accessKey, secretKey);
    }
}

2.service（业务层）

实现类的MyFileUtil工具类的实现在后文

package com.sztu.server.service;

import com.sztu.common.model.ResponseResult;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.List;

/**
 * 七牛云文件存储
 *
 * @author xzx
 * @date 2022/11/10
 */
public interface IQiniuService {

    /**
     * 上传文件返回外链
     *
     * @param file
     * @return
     */
    ResponseResult uploadFile(MultipartFile file);

    /**
     * 上传多文件，返回文件名和外链映射
     *
     * @param files
     * @return
     */
    ResponseResult uploadFiles(List<MultipartFile> files);

    /**
     * 下载压缩包
     * @param targetUrls 上传后存储在数据库的url
     * @param request
     * @param response
     */
    void getZip(String[] targetUrls, HttpServletRequest request, HttpServletResponse response);

    /**
     * 下载单个文件
     * @param targetUrl 上传后存储在数据库的url
     * @param request
     * @param response
     */
    void downloadFile(String targetUrl, HttpServletRequest request, HttpServletResponse response);

}

package com.sztu.server.service.impl;

import cn.hutool.core.date.DateUtil;
import cn.hutool.core.io.file.FileWriter;
import cn.hutool.core.util.StrUtil;
import cn.hutool.http.HttpRequest;
import cn.hutool.http.HttpResponse;
import com.qiniu.http.Response;
import com.qiniu.storage.UploadManager;
import com.qiniu.util.Auth;
import com.sztu.common.enums.HttpCodeEnum;
import com.sztu.common.model.ResponseResult;
import com.sztu.common.util.file.MyFileUtil;
import com.sztu.server.service.IQiniuService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.util.*;

import static com.sztu.common.constant.Constants.HTTP;

/**
 * 七牛云文件存储
 *
 * @author xzx
 * @date 2022/11/10
 */
@Service
@Slf4j
public class QiniuServiceImpl implements IQiniuService {

    @Autowired
    private UploadManager uploadManager;
    @Autowired
    private Auth auth;

    @Value("${qiniu.bucket}")
    private String bucket;
    @Value("${qiniu.domain}")
    private String domain;

    @Override
    public ResponseResult uploadFile(MultipartFile file) {
        try {
            String targetUrl = upload(file);
            if (StrUtil.isEmpty(targetUrl)) {
                return ResponseResult.errorResult(HttpCodeEnum.SERVER_ERROR);
            }
            return ResponseResult.okResult(targetUrl);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return ResponseResult.errorResult(HttpCodeEnum.SERVER_ERROR);
    }

    @Override
    public ResponseResult uploadFiles(List<MultipartFile> files) {
        HashMap<String, String> resMap = new HashMap<>();
        for (MultipartFile file : files) {
            try {
                // 1. get targetUrl
                String targetUrl = upload(file);
                // 2. Map[fileName, url]
                resMap.put(file.getOriginalFilename(), StrUtil.isNotEmpty(targetUrl) ? targetUrl : "upload error!");
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        return ResponseResult.okResult(resMap);
    }

    @Override
    public void getZip(String[] targetUrls, HttpServletRequest request, HttpServletResponse response) {
        List<Map<String, String>> mapList = new ArrayList<>();
        for (String targetUrl : targetUrls) {
            HashMap<String, String> map = new HashMap<>();
            // downloadUrl 下载地址
            map.put("path", getDownloadUrl(targetUrl));
            // fileName 下载后文件名
            map.put("name", getFileName(targetUrl));
            mapList.add(map);
        }
        MyFileUtil.zipUrlToFile(mapList, request, response);
    }

    @Override
    public void downloadFile(String targetUrl, HttpServletRequest request, HttpServletResponse response) {
        MyFileUtil.downloadHttpFile(getDownloadUrl(targetUrl), request, response, getFileName(targetUrl));
    }


    /**
     * 下载到本地
     *
     * @param downloadUrl
     * @param filePath
     */
    public void download(String downloadUrl, String filePath) {
        HttpResponse response = HttpRequest.get(downloadUrl).execute();
        if (response.isOk()) {
            // 2.获取输入流
            InputStream is = response.bodyStream();
            // 3.写如本地
            File file = new File(filePath);
            FileWriter fileWriter = new FileWriter(file);
            fileWriter.writeFromStream(is);
        }
    }

    /**
     * 上传文件返回链接
     * 失败返回null
     *
     * @param file
     * @return
     * @throws IOException
     */
    private String upload(MultipartFile file) throws IOException {
        String key = generateRemoteFileName(file);
        Response res = uploadManager.put(file.getBytes(), key, getUpToken());
        // 打印返回的信息
        if (res.isOK() && res.isJson()) {
            // 返回这张存储照片的地址
            // TODO: key应该是response的key,需要修改一下
            return getTargetUrl(key);
        } else {
            log.error("七牛云异常：{}", res.bodyString());
            return null;
        }
    }

    /**
     * 简单上传，使用默认策略，只需要设置上传的空间名就可以了
     *
     * @return
     */
    private String getUpToken() {
        return auth.uploadToken(bucket);
    }

    /**
     * 获取文件名
     *
     * @param targetUrl
     * @return
     */
    public String getFileName(String targetUrl) {
        return targetUrl.substring(targetUrl.lastIndexOf("/") + 1);
    }

    /**
     * 拼接url
     *
     * @param key
     * @return
     */
    private String getTargetUrl(String key) {
        // HTTP 是 指 "http://"
        return StrUtil.isNotBlank(key) ? (HTTP + domain + "/" + key) : null;
    }

    /**
     * TODO: 有待完善
     * 目前：yyyy-MM-dd HH:mm:ss_xxx.xxx
     *
     * @param file 上传文件
     * @return
     */
    private String generateRemoteFileName(MultipartFile file) {
        return DateUtil.now() + "_" + file.getOriginalFilename();
    }

    /**
     * 获取下载地址
     *
     * @param targetUrl
     * @return
     */
    private String getDownloadUrl(String targetUrl) {
        return auth.privateDownloadUrl(targetUrl);
    }

}

3.controller（接口）

package com.sztu.web.controller;

import com.sztu.common.model.ResponseResult;
import com.sztu.server.service.IQiniuService;
import org.apache.ibatis.annotations.Param;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.List;

/**
 * 七牛云文件存储
 *
 * @author xzx
 * @date 2022/11/10
 */
@RestController
@RequestMapping("/qiniu")
public class QiniuController {

    @Autowired
    private IQiniuService qiniuService;

    /**
     * 上传单个文件
     * @param file
     * @return
     */
    @PostMapping("/upload")
    public ResponseResult uploadFile(@Param("file") MultipartFile file) {
        return qiniuService.uploadFile(file);
    }

    /**
     * 上传多个文件
     * @param files
     * @return
     */
    @PostMapping("/uploads")
    public ResponseResult uploadFiles(@Param("files") List<MultipartFile> files) {
        return qiniuService.uploadFiles(files);
    }

    /**
     * 下载压缩包
     * @param targetUrls 上传后存储在数据库的url
     * @param request
     * @param response
     */
    @GetMapping("/zip")
    public void getZip(String[] targetUrls, HttpServletRequest request, HttpServletResponse response) {
        qiniuService.getZip(targetUrls, request, response);
    }

    /**
     * 下载单个文件
     * @param targetUrl 上传后存储在数据库的url
     * @param request
     * @param response
     */
    @GetMapping("/download")
    public void downloadFile(String targetUrl, HttpServletRequest request, HttpServletResponse response) {
        qiniuService.downloadFile(targetUrl, request, response);
    }

}

3.实现文件下载和压缩核心逻辑

MyFileUtil工具类

添加Maven依赖

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-compress</artifactId>
    <version>1.20</version>
</dependency>

添加工具类

使用到了hutool-http的请求响应

HttpRequest和HttpResponse

官方链接

hutool-http

package com.sztu.common.util.file;

import cn.hutool.core.io.IoUtil;
import cn.hutool.http.HttpRequest;
import org.apache.commons.compress.archivers.ArchiveEntry;
import org.apache.commons.compress.archivers.zip.Zip64Mode;
import org.apache.commons.compress.archivers.zip.ZipArchiveEntry;

import org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.activation.MimetypesFileTypeMap;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.*;
import java.net.URLEncoder;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;
import java.util.Map;

/**
 * 文件管理相关工具类
 *
 * @author xzx
 * @date 2022/11/10
 **/
public class MyFileUtil {

    private static final Logger logger = LoggerFactory.getLogger(MyFileUtil.class);

    private static MimetypesFileTypeMap mimetypesFileTypeMap;

    /**
     * 下载http文件流
     *
     * @param urlStr   下载链接
     * @param request  请求
     * @param response 响应
     * @param fileName 文件名
     * @return
     */
    public static void downloadHttpFile(String urlStr, HttpServletRequest request, HttpServletResponse response, String fileName) {
        ServletOutputStream out = null;
        try {
            // 1.发送请求获取bytes
            byte[] getData = HttpRequest.get(urlStr).execute().bodyBytes();
            // 2.写入响应输出流
            out = response.getOutputStream();
            long contentLength = getData.length;
            MyFileUtil.setResponse(fileName, contentLength, request, response);
            out.write(getData);
            out.flush();
        } catch (Exception e) {
            throw new RuntimeException("下载失败!");
        } finally {
            IoUtil.close(out);
        }
    }


    /**
     * 下载文件流
     *
     * @param file
     * @param request
     * @param response
     * @param fileName
     * @return
     */
    public static void downloadFile(File file, HttpServletRequest request, HttpServletResponse response, String fileName) {
        if (file != null && file.exists() && file.length() > 0L) {
            try {
                RandomAccessFile randomFile = new RandomAccessFile(file, "r");
                Throwable var5 = null;

                Object var54;
                try {
                    ServletOutputStream out = response.getOutputStream();
                    Throwable var7 = null;

                    try {
                        long contentLength = randomFile.length();
                        String range = request.getHeader("Range");
                        long start = 0L;
                        long end = 0L;
                        if (range != null && range.startsWith("bytes=")) {
                            String[] values = range.split("=")[1].split("-");
                            start = Long.parseLong(values[0]);
                            if (values.length > 1) {
                                end = Long.parseLong(values[1]);
                            }
                        }

                        int requestSize;
                        if (end != 0L && end > start) {
                            requestSize = Long.valueOf(end - start + 1L).intValue();
                        } else {
                            requestSize = 2147483647;
                        }

                        MyFileUtil.setResponse(fileName, contentLength, request, response);

                        randomFile.seek(start);

                        byte[] buffer;
                        for (int needSize = requestSize; needSize > 0; needSize -= buffer.length) {
                            buffer = new byte[1024];
                            int len = randomFile.read(buffer);
                            if (needSize < buffer.length) {
                                out.write(buffer, 0, needSize);
                            } else {
                                out.write(buffer, 0, len);
                                if (len < buffer.length) {
                                    break;
                                }
                            }
                        }

                        out.flush();
                        var54 = null;
                    } catch (Throwable var47) {
                        var7 = var47;
                        throw var47;
                    } finally {
                        if (out != null) {
                            if (var7 != null) {
                                try {
                                    out.close();
                                } catch (Throwable var46) {
                                    var7.addSuppressed(var46);
                                }
                            } else {
                                out.close();
                            }
                        }

                    }
                } catch (Throwable var49) {
                    var5 = var49;
                    throw var49;
                } finally {
                    if (randomFile != null) {
                        if (var5 != null) {
                            try {
                                randomFile.close();
                            } catch (Throwable var45) {
                                var5.addSuppressed(var45);
                            }
                        } else {
                            randomFile.close();
                        }
                    }

                }
            } catch (IOException var51) {
                logger.debug(var51.getMessage(), var51);
                throw new RuntimeException(var51.getMessage());
            }
        } else {
            throw new RuntimeException("文件为空或不存在！");
        }
    }

    /**
     * @param fileName
     * @param contentLength
     * @param request
     * @param response
     * @return
     */
    public static void setResponse(String fileName, long contentLength, HttpServletRequest request, HttpServletResponse response) {
        try {
            response.setContentType(MyFileUtil.getContentType("0.jpg"));
            boolean isPreview = "preview".equalsIgnoreCase(request.getParameter("source"));
            response.addHeader("Content-Disposition", (!isPreview ? "attachment; " : "") + "filename*=utf-8'zh_cn'" + URLEncoder.encode(fileName, "UTF-8"));
            response.setHeader("Accept-Ranges", "bytes");

            String range = request.getHeader("Range");
            if (range == null) {
                response.setHeader("Content-Length", String.valueOf(contentLength));
            } else {
                response.setStatus(206);
                long requestStart = 0L;
                long requestEnd = 0L;
                String[] ranges = range.split("=");
                if (ranges.length > 1) {
                    String[] rangeDatas = ranges[1].split("-");
                    requestStart = Long.parseLong(rangeDatas[0]);
                    if (rangeDatas.length > 1) {
                        requestEnd = Long.parseLong(rangeDatas[1]);
                    }
                }

                long length = 0L;
                if (requestEnd > 0L) {
                    length = requestEnd - requestStart + 1L;
                    response.setHeader("Content-Length", String.valueOf(length));
                    response.setHeader("Content-Range", "bytes " + requestStart + "-" + requestEnd + "/" + contentLength);
                } else {
                    length = contentLength - requestStart;
                    response.setHeader("Content-Length", String.valueOf(length));
                    response.setHeader("Content-Range", "bytes " + requestStart + "-" + (contentLength - 1L) + "/" + contentLength);
                }
            }
        } catch (Exception e) {
            throw new RuntimeException("response响应失败!");
        }
    }

    public static String getContentType(String fileName) {
        if (mimetypesFileTypeMap == null) {
            mimetypesFileTypeMap = new MimetypesFileTypeMap();
        }

        return mimetypesFileTypeMap.getContentType(fileName);
    }

    /**
     * 将磁盘的多个文件打包成压缩包并输出流下载
     *
     * @param pathList
     * @param request
     * @param response
     */
    public static void zipDirFileToFile(List<Map<String, String>> pathList, HttpServletRequest request, HttpServletResponse response) {
        try {
            // 设置response参数并且获取ServletOutputStream
            ZipArchiveOutputStream zous = getServletOutputStream(response);

            for (Map<String, String> map : pathList) {
                String fileName = map.get("name");
                File file = new File(map.get("path"));
                InputStream inputStream = new FileInputStream(file);
                setByteArrayOutputStream(fileName, inputStream, zous);
            }
            zous.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    /**
     * 将网络url资源文件的多个文件打包成压缩包并输出流下载
     *
     * @param pathList
     * @param request
     * @param response
     */
    public static void zipUrlToFile(List<Map<String, String>> pathList, HttpServletRequest request, HttpServletResponse response) {
        try {
            // 设置response参数并且获取ServletOutputStream
            ZipArchiveOutputStream zous = getServletOutputStream(response);

            for (Map<String, String> map : pathList) {
                String fileName = map.get("name");
                InputStream inputStream = getInputStreamFromUrl(map.get("path"));
                setByteArrayOutputStream(fileName, inputStream, zous);
            }
            zous.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private static ZipArchiveOutputStream getServletOutputStream(HttpServletResponse response) throws Exception {

        String outputFileName = "文件" + new SimpleDateFormat("yyyyMMddHHmmss").format(new Date()) + ".zip";
        response.reset();
        response.setHeader("Content-Type", "application/octet-stream");
        response.setHeader("Content-Disposition", "attachment;filename=" + URLEncoder.encode(outputFileName, "UTF-8"));
        response.setHeader("Pragma", "no-cache");
        response.setHeader("Cache-Control", "no-cache");
        ServletOutputStream out = response.getOutputStream();

        ZipArchiveOutputStream zous = new ZipArchiveOutputStream(out);
        zous.setUseZip64(Zip64Mode.AsNeeded);
        return zous;
    }

    private static void setByteArrayOutputStream(String fileName, InputStream inputStream, ZipArchiveOutputStream zous) throws Exception {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        byte[] buffer = new byte[1024];
        int len;
        while ((len = inputStream.read(buffer)) != -1) {
            baos.write(buffer, 0, len);
        }
        baos.flush();
        byte[] bytes = baos.toByteArray();

        //设置文件名
        ArchiveEntry entry = new ZipArchiveEntry(fileName);
        zous.putArchiveEntry(entry);
        zous.write(bytes);
        zous.closeArchiveEntry();
        baos.close();
    }

    /**
     * 通过网络地址获取文件InputStream
     *
     * @param path 地址
     * @return
     */
    private static InputStream getInputStreamFromUrl(String path) {
        return HttpRequest.get(path).execute().bodyStream();
    }
}