使用Spring进行文件的上传和下载

使用Spring进行文件的上传和下载

本文主要介绍在Spring框架下面调用微服务的dubbo rpc接口进行文件的上传和下载，以及记录在实现过程中遇到的一些容易出错的地方。

Spring上传文件接口设计

contoller层的代码实现如下所示：

    @PostMapping("/submitEvidence")
    public BaseResponse<?> submitEvidence(@RequestParam("id") Long id, @RequestParam("label") String label,
                                          @RequestParam(value = "file") MultipartFile file) {
           uploadEvidence(id, label, file);
           return BaseResponse.success().errorMsg("操作成功").build();
        }
    }

使用postman请求上传文件接口，具体参数如下图所示：
Service层代码实现如下所示：

public void uploadEvidence(Long takeDownId, String label, MultipartFile multipartFile) {
        if(Objects.isNull(multipartFile)) {
            throw new RunTimeException("上传的文件不能为空");
        }
        String fileName = multipartFile.getOriginalFilename();
        InputStream file = null;
        try {
            file = multipartFile.getInputStream();
        } catch (IOException e) {}
        byte[] fileBytes = new byte[20 * 1024 * 1024];
        InputStream inputStream = null;
        ByteArrayOutputStream outputStream = null;
        try {
            outputStream =  new ByteArrayOutputStream();
            inputStream = multipartFile.getInputStream();
            try {
                byte[] buffer = new byte[1024];
                int read = inputStream.read(buffer);
                while (read != -1) {
                    outputStream.write(buffer, 0, read);
                    read = inputStream.read(buffer);
                }
            } catch (Exception e) {
                log.error("处理返回值失败" + e.getMessage());
                throw new RunTimeException("上传文件失败");
            } finally {
                try {
                    inputStream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            outputStream.flush();
            fileBytes = outputStream.toByteArray();
        } catch (IOException e) {
        }finally {
            if(outputStream != null) {
                try {
                    outputStream.close();
                } catch (IOException e1) {
                    e1.printStackTrace();
                    throw new RuntimeException("上传文件失败");
                }
            }
        }
        UploadEvidenceNetCraftReq req = UploadEvidenceNetCraftReq.builder()
                        .takeDownId(takeDownId.intValue()).label(label).fileName(fileName).fileBytes(fileBytes).build();
        // outVendor是一个dubbo框架下的rpc服务，用于上传文件
        BaseResponse baseResponse = outerVendorsService.uploadEvidence(req);
        ...
    }

dubbo接口设计

上传文件流的RPC的接口设计

我们构建的RPC接口采用的是dubbo框架，最初始的接口设计，将HttpServletResponse作为接口的参数类型传参，结果报错
io.netty.handler.codec.EncoderException: java.lang.IllegalStateException: Serialized class

Dubbo报错：io.netty.handler.codec.EncoderException: java.lang.IllegalStateException: Serialized class

HttpServletResponse不能被dubbo作为接口参数序列化，于是转而求其次，将可序列化的类型byte[]作为传输流的参数
outerVendorsService服务提供的上传文件的rpc接口：uploadEvidence接口，具体代码设计如下所示：

BaseResponse<UploadEvidenceRsp> uploadEvidence(UploadEvidenceReq req);
@Data
@Builder
@Jacksonized
public class UploadEvidenceReq implements Serializable {
    private Integer takeDownId;
   //使用可序列化的byte数组作为入参
    private byte[] fileBytes;

    private String fileName;

    private String label;
}

@Data
@Setter
@Getter
public class UploadEvidenceRsp implements Serializable {
    private Integer file_id;

    @JsonProperty("error_code")
    private String errorCode;

    @JsonProperty("error_message")
    private String errorMessage;
}

Spring文件下载接口设计

文件下载的相关接口有两种实现：一种是将HttpServletResponse作为controller层的传参引入，将流写入到HttpServletResponse中，然后返回前端，但是在使用过程中，直接在HttpServletResponse的示例中setHeader失败，于是转而选择构ResponseEntity的方式来进行http返回值的构造，具体实现如下所示：

@GetMapping("/searchForEvidence")
public ResponseEntity searchForEvidence(@RequestParam String id) {
    return searchForEvidence(id);
}

使用postman请求下载文件接口，具体参数如下图所示：

Service层代码实现如下所示：

public ResponseEntity fetchEvidence(Long takeDownId){
    FetchEvidenceNetCraftReq req = FetchEvidenceNetCraftReq.builder().takeDownId(takeDownId.intValue()).build();
    BaseResponse<QueryEvidenceRsp> rsp = outerVendorsService
            .fetchEvidence(req);
    if(Objects.isNull(rsp)) {
        throw new RunTimeException("拉取文件失败");
    }
    byte[] outPutBytes = null;
    if(Objects.nonNull(rsp) && rsp.isSuccess() == true) {
        QueryEvidenceRsp evidenceRsp = rsp.getResult();
        if(Objects.nonNull(evidenceRsp.getErrorCode())){
            String message = "errorCode:" + evidenceRsp.getErrorCode() + ",errorMessage:" + evidenceRsp.getErrorMessage();
            throw new RunTimeException(message);
        }
        outPutBytes = evidenceRsp.getFileBytes();
        String fileName = evidenceRsp.getFileName();
        HttpHeaders responseHeaders = new HttpHeaders();
        responseHeaders.setContentType(MediaType.valueOf(MediaType.APPLICATION_OCTET_STREAM_VALUE));// 设置文件格式
        responseHeaders.setContentLength(outPutBytes.length);
        responseHeaders.set("Content-Disposition", "attachment;filename=" + fileName);// 设置文件名
        return new ResponseEntity<>(outPutBytes, responseHeaders, HttpStatus.OK);
    }
    throw new RunTimeException("拉取文件失败");
}

public class QueryEvidenceRsp implements Serializable {
    byte[] fileBytes;
    String fileName;
    String errorCode;
    String errorMessage;
}

dubbo接口设计

下载文件流的RPC的接口设计

参照之前的上传文件的设计，服务提供的下载文件的rpc接口设计：

@Data
@Setter
@Getter
public class QueryEvidenceRsp implements Serializable {
    byte[] fileBytes;
    String fileName;
    String errorCode;
    String errorMessage;
}

@Data
@Builder
@Jacksonized
public class FetchEvidenceNetCraftReq implements Serializable {
    private Integer takeDownId;
}

    @Override
    public BaseResponse fetchEvidence(FetchEvidenceNetCraftReq req) {
        if(Objects.isNull(netCraftConfig) || Objects.isNull(netCraftConfig.getAccessNetCraftDomain())) {
            log.error("netCraft config is not set");
            return BaseResponse.fail().errorCode(ErrorCode.BUSINESS_EXCEPTION.getCode())
                    .errorMsg("netCraft config is not set").build();
        }
        String fetch_evidence = "https://" + netCraftConfig.getAccessNetCraftDomain()
                + netCraftConfig.getFETCH_EVIDENCE();
        Map<String, String> headers = new HashMap<>();
        headers.put("content-type", "application/json");
        headers.put("Authorization", netCraftConfig.getAccessNetCraftAuthToken());
        ByteArrayOutputStream  outputStream = new ByteArrayOutputStream();
        byte[] fileBytes;
        String fileName;
        try {
            outputStream =  new ByteArrayOutputStream();
            fileName = getFromOctetStream(fetch_evidence + "?takedown_id="
                    + req.getTakeDownId(), headers, outputStream);
            if(Objects.isNull(fileName)) {
                return BaseResponse.fail().errorCode(ErrorCode.BUSINESS_EXCEPTION.getCode())
                        .errorMsg("获取netcraft证据文件失败").build();
            }
            outputStream.flush();
            fileBytes = outputStream.toByteArray();
        } catch (Exception e) {
            log.error("获取证据文件失败:" + e.getMessage());
            return BaseResponse.fail().errorCode(ErrorCode.BUSINESS_EXCEPTION.getCode())
                    .errorMsg("获取netcraft证据文件失败").build();
        } finally {
            if(outputStream != null) {
                try {
                    outputStream.close();
                } catch (IOException e1) {
                    e1.printStackTrace();
                }
            }
        }
        NetCraftQueryEvidenceRsp rsp = new NetCraftQueryEvidenceRsp();
        if(Objects.nonNull(fileName)) {
            NetCraftErrorMessageRsp errorMessageRsp = JsonUtils.fromCamelJson(fileName, NetCraftErrorMessageRsp.class);
            if(Objects.nonNull(errorMessageRsp)
                    && Objects.nonNull(errorMessageRsp.getErrorCode())
                    && Objects.nonNull(errorMessageRsp.getErrorMessage())) {
                rsp.setErrorCode(errorMessageRsp.getErrorCode());
                rsp.setErrorMessage(errorMessageRsp.getErrorMessage());
                return BaseResponse.success(rsp).build();
            }
            rsp.setFileBytes(fileBytes);
            rsp.setFileName(fileName);
            return BaseResponse.success(rsp).build();
        }
        return BaseResponse.fail().errorCode(ErrorCode.BUSINESS_EXCEPTION.getCode())
                .errorMsg("获取netcraft证据文件失败").build();
    }

    public static String getFromOctetStream(String url, Map<String, String> headers, OutputStream outputStream) {
        return getFromOctetStream(url, headers, OK_HTTP_CLIENT_30s, outputStream);
    }
  public static String getFromOctetStream(String url, Map<String, String> headers, String client, OutputStream outputStream) {
        Request.Builder requestBuilder = new Request.Builder();
        requestBuilder.url(url);
        if (headers != null && headers.size() > 0) {
            for (String s : headers.keySet()) {
                requestBuilder.addHeader(s, headers.get(s));
            }
        }
        requestBuilder.get();
        Request req = requestBuilder.build();
        try (Response response = okHttpClientMap.get(client).newCall(req).execute()) {
            log.info("okhttp send get,resp:{}", JsonUtils.toJson(response));
            if (null != response.body()) {
                InputStream inputStream = response.body().byteStream();
                try {
                    byte[] buffer = new byte[1024];
                    int read = inputStream.read(buffer);
                    while (read != -1) {
                        outputStream.write(buffer, 0, read);
                        read = inputStream.read(buffer);
                    }
                } catch (Exception e) {
                    log.error("处理返回值失败" + e.getMessage());
                    return null;
                } finally {
                    try {
                        inputStream.close();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
                String contentDisposition = response.header("Content-Disposition");
                if (Objects.isNull(contentDisposition)) {
                    log.error("调用netcraft获取证据接口, 获取文件名失败");
                    return outputStream.toString();
                }
                // 解析文件名
                return contentDisposition.substring(contentDisposition.indexOf("filename=") + 9);
            }

spring上传文件大小控制

在spring配置文件application.properties中，通过配置下面两个参数的值来限制文件的大小

spring.servlet.multipart.max-file-size=-1
spring.servlet.multipart.max-request-size=-1

spring.servlet.multipart.max-file-size配置限制上传单个文件的大小，为-1代表不限制
spring.servlet.multipart.max-request-size配置限制http中上传总文件的大小，为-1代表不限制