背景:使用hutool工具进行文件上传。
直接上代码:
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.1.0</version>
</dependency>
import cn.hutool.http.HttpRequest;
import java.io.File;
import java.util.*;
//……
String urlString = "……/upload";
Map<String, Object> paramMap = new HashMap<>();
File file = new File("D:\\soft\\Anaconda3-2021.05-Windows-x86_64.exe");
paramMap.put("file", file);
HttpRequest.post(urlString).form(paramMap).timeout(60*1000).execute().body();
当文件比较大的时候,就引发了OOM异常。
经过查阅HttpRequest源码发现它有一个字段:blockSize(Chuncked块大小,0或小于0表示不设置Chuncked模式),最终是设置到java.net.HttpURLConnection的chunkLength字段中。
Chuncked模式,即分块传输模式,
因此设置该值大于0即可解决该OOM问题。
附测试demo:
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>cn.sky</groupId>
<artifactId>mytest</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.3.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>2.2.2.RELEASE</version>
</dependency>
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.1.0</version>
</dependency>
<dependency>
<groupId>com.konghq</groupId>
<artifactId>unirest-java</artifactId>
<version>3.11.12</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.78</version>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>3.13.1</version>
</dependency>
</dependencies>
</project>
MyController:
package cn.sky.mytest.controller;
import cn.hutool.http.HttpRequest;
import com.alibaba.fastjson.JSON;
import kong.unirest.HttpResponse;
import kong.unirest.Unirest;
import okhttp3.*;
import okhttp3.RequestBody;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import java.io.File;
import java.io.IOException;
import java.util.*;
@RequestMapping("/myController")
@RestController
public class MyController {
@PostMapping("/upload")
public Object upload(@RequestParam(value = "file") MultipartFile file){
System.out.println(file.getOriginalFilename()+":"+(file.getSize()/1024/1024)+"M");
return "ok";
}
@GetMapping("/test")
public Object test(int type) throws IOException {
String urlString = "http://localhost:8080/myController/upload";
Map<String, Object> paramMap = new HashMap<>();
File file = new File("D:\\soft\\Anaconda3-2021.05-Windows-x86_64.exe");
paramMap.put("file", file);
if(type==1) {
int blockSize = 1024*10;
String body = HttpRequest.post(urlString)
/**
* 采用流方式上传数据,无需本地缓存数据。
* HttpUrlConnection默认是将所有数据读到本地缓存,然后再发送给服务器,这样上传大文件时就会导致内存溢出。
* blockSize – 块大小(bytes数),0或小于0表示不设置Chuncked模式
*/
.setChunkedStreamingMode(blockSize)
.form(paramMap).timeout(60*1000).execute().body();
System.out.println(body);
}else if(type==2){
String body = HttpRequest.post(urlString).form(paramMap).timeout(60*1000).execute().body();
System.out.println(body);
}else if(type==3){
HttpResponse<String> stringHttpResponse = Unirest.post(urlString).fields(paramMap).asString();
System.out.println(stringHttpResponse.getStatus()+":"+stringHttpResponse.getBody());
}else if(type==4){
OkHttpClient client = new OkHttpClient().newBuilder()
.build();
MediaType mediaType = MediaType.parse("text/plain");
RequestBody body = new MultipartBody.Builder().setType(MultipartBody.FORM)
.addFormDataPart("file",file.getName(),RequestBody.create(MediaType.parse("application/octet-stream"),file))
.build();
Request request = new Request.Builder()
.url(urlString)
.method("POST", body)
.build();
Response response = client.newCall(request).execute();
System.out.println(JSON.toJSONString(response));
}
return "ok";
}
}
application.properties:
server.port=8080
#需要设置这个,否则文件太大会上传失败
spring.servlet.multipart.max-file-size = 1073741824
spring.servlet.multipart.max-request-size = 1073741824
启动脚本(限制内存为100M):
mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Xmx100M -Xms100M"
访问地址:http://localhost:8080/myController/test?type=
type可为1,2,3,4,堆内存使用情况如下(文件为477M):
type=1:
type=2:
type=3:
type=4: