1. Ways to convert an InputStream to a String:
1. Using IOUtils.toString (Apache Utils)
String result = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
2. Using CharStreams (Guava)
String result = CharStreams.toString(new InputStreamReader(inputStream, Charsets.UTF_8));
3. Using Scanner (JDK)
Scanner s = new Scanner(inputStream).useDelimiter("\\A");
String result = s.hasNext() ? s.next() : "";
4. Using Stream API (Java 8). Warning: This solution converts different line breaks (like \r\n) to \n.
String result = new BufferedReader(new InputStreamReader(inputStream))
.lines().collect(Collectors.joining("\n"));
5. Using parallel Stream API (Java 8). Warning: This solution converts different line breaks (like \r\n) to \n.
String result = new BufferedReader(new InputStreamReader(inputStream))
.lines().parallel().collect(Collectors.joining("\n"));
6. Using InputStreamReader and StringBuilder (JDK)
int bufferSize = 1024;
char[] buffer = new char[bufferSize];
StringBuilder out = new StringBuilder();
Reader in = new InputStreamReader(stream, StandardCharsets.UTF_8);
for (int numRead; (numRead = in.read(buffer, 0, buffer.length)) > 0; ) {
out.append(buffer, 0, numRead);
}
return out.toString();
7. Using StringWriter and IOUtils.copy (Apache Commons)
StringWriter writer = new StringWriter();
IOUtils.copy(inputStream, writer, "UTF-8");
return writer.toString();
8. Using ByteArrayOutputStream and inputStream.read (JDK)
ByteArrayOutputStream result = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
for (int length; (length = inputStream.read(buffer)) != -1; ) {
result.write(buffer, 0, length);
}
// StandardCharsets.UTF_8.name() > JDK 7
return result.toString("UTF-8");
9. Using BufferedReader (JDK). Warning: This solution converts different line breaks (like \n\r) to line.separator system property (for example, in Windows to “\r\n”).
String newLine = System.getProperty("line.separator");
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
StringBuilder result = new StringBuilder();
for (String line; (line = reader.readLine()) != null; ) {
if (result.length() > 0) {
result.append(newLine);
}
result.append(line);
}
return result.toString();
10. Using BufferedInputStream and ByteArrayOutputStream (JDK)
BufferedInputStream bis = new BufferedInputStream(inputStream);
ByteArrayOutputStream buf = new ByteArrayOutputStream();
for (int result = bis.read(); result != -1; result = bis.read()) {
buf.write((byte) result);
}
// StandardCharsets.UTF_8.name() > JDK 7
return buf.toString("UTF-8");
11. Using inputStream.read() and StringBuilder (JDK). Warning: This solution has problems with Unicode, for example with Russian text (works correctly only with non-Unicode text)
StringBuilder sb = new StringBuilder();
for (int ch; (ch = inputStream.read()) != -1; ) {
sb.append((char) ch);
}
return sb.toString();
Warning:
-
Solutions 4, 5 and 9 convert different line breaks to one.
-
Solution 11 can’t work correctly with Unicode text
Performance tests
Performance tests for small String (length = 175), url in github (mode = Average Time, system = Linux, score 1,343 is the best):
Benchmark Mode Cnt Score Error Units
8. ByteArrayOutputStream and read (JDK) avgt 10 1,343 ± 0,028 us/op
6. InputStreamReader and StringBuilder (JDK) avgt 10 6,980 ± 0,404 us/op
10. BufferedInputStream, ByteArrayOutputStream avgt 10 7,437 ± 0,735 us/op
11. InputStream.read() and StringBuilder (JDK) avgt 10 8,977 ± 0,328 us/op
7. StringWriter and IOUtils.copy (Apache) avgt 10 10,613 ± 0,599 us/op
1. IOUtils.toString (Apache Utils) avgt 10 10,605 ± 0,527 us/op
3. Scanner (JDK) avgt 10 12,083 ± 0,293 us/op
2. CharStreams (guava) avgt 10 12,999 ± 0,514 us/op
4. Stream Api (Java 8) avgt 10 15,811 ± 0,605 us/op
9. BufferedReader (JDK) avgt 10 16,038 ± 0,711 us/op
5. parallel Stream Api (Java 8) avgt 10 21,544 ± 0,583 us/op
Performance tests for big String (length = 50100), url in github (mode = Average Time, system = Linux, score 200,715 is the best):
Benchmark Mode Cnt Score Error Units
8. ByteArrayOutputStream and read (JDK) avgt 10 200,715 ± 18,103 us/op
1. IOUtils.toString (Apache Utils) avgt 10 300,019 ± 8,751 us/op
6. InputStreamReader and StringBuilder (JDK) avgt 10 347,616 ± 130,348 us/op
7. StringWriter and IOUtils.copy (Apache) avgt 10 352,791 ± 105,337 us/op
2. CharStreams (guava) avgt 10 420,137 ± 59,877 us/op
9. BufferedReader (JDK) avgt 10 632,028 ± 17,002 us/op
5. parallel Stream Api (Java 8) avgt 10 662,999 ± 46,199 us/op
4. Stream Api (Java 8) avgt 10 701,269 ± 82,296 us/op
12. BufferedInputStream, ByteArrayOutputStream avgt 10 740,837 ± 5,613 us/op
3. Scanner (JDK) avgt 10 751,417 ± 62,026 us/op
11. InputStream.read() and StringBuilder (JDK) avgt 10 2919,350 ± 1101,942 us/op
Graphs (performance tests depending on Input Stream length in Windows 7 system)
Performance test (Average Time) depending on Input Stream length in Windows 7 system:
length 182 546 1092 3276 9828 29484 58968
test8 0.38 0.938 1.868 4.448 13.412 36.459 72.708
test4 2.362 3.609 5.573 12.769 40.74 81.415 159.864
test5 3.881 5.075 6.904 14.123 50.258 129.937 166.162
test9 2.237 3.493 5.422 11.977 45.98 89.336 177.39
test6 1.261 2.12 4.38 10.698 31.821 86.106 186.636
test7 1.601 2.391 3.646 8.367 38.196 110.221 211.016
test1 1.529 2.381 3.527 8.411 40.551 105.16 212.573
test3 3.035 3.934 8.606 20.858 61.571 118.744 235.428
test2 3.136 6.238 10.508 33.48 43.532 118.044 239.481
test10 1.593 4.736 7.527 20.557 59.856 162.907 323.147
test11 3.913 11.506 23.26 68.644 207.591 600.444 1211.545
2. nio 使用 Paths.get 的 java.nio.file.FileSystemNotFoundException 分析
Question:
I have a Maven project and inside a method I want to create a path for a directory in my resources folder. This is done like this:
try {
final URI uri = getClass().getResource("/my-folder").toURI();
Path myFolderPath = Paths.get(uri);
} catch (final URISyntaxException e) {
...
}
The generated URI looks like jar:file:/C:/path/to/my/project.jar!/my-folder.
The stacktrace is as following:
Exception in thread "pool-4-thread-1" java.nio.file.FileSystemNotFoundException
at com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
at com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
at java.nio.file.Paths.get(Paths.java:143)
The URI seems to be valid. The part before ! points to the generated jar-file and the part after it to my-folder in the root of the archive. I have used this instructions before to create paths to my resources. Why am I getting an exception now?
Answer:
You need to create the file system before you can access the path within the zip like
final URI uri = getClass().getResource("/my-folder").toURI();
Map<String, String> env = new HashMap<>();
env.put("create", "true");
FileSystem zipfs = FileSystems.newFileSystem(uri, env);
Path myFolderPath = Paths.get(uri);
This is not done automatically.
See http://docs.oracle.com/javase/7/docs/technotes/guides/io/fsp/zipfilesystemprovider.html
or
private FileSystem initFileSystem(URI uri) throws IOException {
try {
return FileSystems.newFileSystem(uri, Collections.emptyMap());
}catch(IllegalArgumentException e) {
return FileSystems.getDefault();
}
}
or
private FileSystem initFileSystem(URI uri) throws IOException {
try {
return FileSystems.getFileSystem(uri);
} catch( FileSystemNotFoundException e ) {
Map<String, String> env = new HashMap<>();
env.put("create", "true");
return FileSystems.newFileSystem(uri, env);
}
}
Calling this with the URI you are about to load will ensure the filesystem is in working condition. I always call FileSystem.close() after using it:
FileSystem zipfs = initFileSystem(fileURI);
filePath = Paths.get(fileURI);
// Do whatever you need and then close the filesystem
zipfs.close();
Careful, a ZipFileSystem can be closed, but a WindowsFileSystem will complain.
3. 在使用nio加载文件时，在idea中运行没有问题，但打成jar包后在windows和linux下都有问题
public void test() throws Exception {
URI uri = getClass().getClassLoader().getResource("conf/sh.txt").toURI();
FileSystem aDefault = FileSystems.getDefault();
System.out.println(aDefault.getClass());
FileSystemProvider provider = FileSystems.getDefault().provider();
System.out.println(provider.getClass());
System.out.println("====================" + uri.getScheme());
List<FileSystemProvider> fileSystemProviders = FileSystemProvider.installedProviders();
fileSystemProviders.forEach(p -> System.out.println(p.getClass()));
Path path = Paths.get(uri);
}
这种情况下在idea中没有问题:
class sun.nio.fs.WindowsFileSystemclass sun.nio.fs.WindowsFileSystemProvider
====================fileclass sun.nio.fs.WindowsFileSystemProviderclass
com.sun.nio.zipfs.ZipFileSystemProvider
但是在打成jar包运行时Path path = Paths.get(uri)这一行会抛出异常：
Exception in thread "pool-4-thread-1" java.nio.file.FileSystemNotFoundException
at com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
at com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
at java.nio.file.Paths.get(Paths.java:143)
究其原因，是FileSystemProvider的使用问题，先看java.nio.file.Paths#get(java.net.URI)：
public static Path get(URI uri) {
String scheme = uri.getScheme();
if (scheme == null) throw new IllegalArgumentException("Missing scheme");
// check for default provider to avoid loading of installed providers
if (scheme.equalsIgnoreCase("file"))
return FileSystems.getDefault().provider().getPath(uri);
// try to find provider
for (FileSystemProvider provider : FileSystemProvider.installedProviders()) {
if (provider.getScheme().equalsIgnoreCase(scheme)) {
return provider.getPath(uri);
}
}
throw new FileSystemNotFoundException("Provider " + scheme + " not installed");
}
- uri.getScheme()在idea中是file，在打成jar包后变成了jar。
- 当前缀以file开头时，会使用FileSystems.getDefault().provider()来处理，这个provider在windows环境下是WindowsFileSystemProvider， 在linux环境下是LinuxFileSystemProvider。
- FileSystemProvider.installedProviders()对应windows中的WindowsFileSystemProvider和ZipFileSystemProvider，对应linux中的LinuxFileSystemProvider和ZipFileSystemProvider。
- 当前缀不以file开头时，会使用FileSystemProvider.installedProviders()中与uri.getScheme()匹配的provider来处理，对应的就是ZipFileSystemProvider。
- ZipFileSystemProvider对应的FileSystem需要自己创建，使用和创建方式参考：https://docs.oracle.com/javase/8/docs/technotes/guides/io/fsp/zipfilesystemprovider.html
解决办法：
在 Path path = Paths.get(uri) 中进行处理
Path path = null;
try {
path = Paths.get(uri);
} catch (Exception e) {
// @see https://stackoverflow.com/questions/25032716/getting-filesystemnotfoundexception-from-zipfilesystemprovider-when-creating-a-p
// @see http://docs.oracle.com/javase/7/docs/technotes/guides/io/fsp/zipfilesystemprovider.html
Map<String, String> env = new HashMap<>();
env.put("create", "true");
FileSystem zipfs = FileSystems.newFileSystem(uri, env);
path = Paths.get(uri);
}
或者使用其他办法加载资源文件:
byte[] data;
try (InputStream in = getClass().getResourceAsStream("/elasticsearch/segmentsIndex.json")) {
data = IOUtils.toByteArray(in);
}
4. 读取 inputStream 的方式
/**
* 直接通过文件名+getPath()来获取路径
*
* @param fileName
* @throws IOException
*/
public void func00(String fileName) throws IOException {
String path = this.getClass().getClassLoader().getResource(fileName).getPath(); // 注意getResource("")里面是空字符串
System.out.println(path);
String filePath = URLDecoder.decode(path, "UTF-8"); // 如果路径中带有中文会被URLEncoder,因此这里需要解码
System.out.println(filePath);
getFileContent(filePath);
}
/**
* 直接通过文件名+getFile()来获取
*
* url.getFile()=/pub/files/foobar.txt?id=123456
* url.getPath()=/pub/files/foobar.txt
*
* @param fileName
* @throws IOException
*/
public void func01(String fileName) throws IOException {
String path = this.getClass().getClassLoader().getResource(fileName).getFile(); // 注意getResource("")里面是空字符串
System.out.println(path);
String filePath = URLDecoder.decode(path, "UTF-8"); // 如果路径中带有中文会被URLEncoder,因此这里需要解码
System.out.println(filePath);
getFileContent(filePath);
}
/**
* 直接使用getResourceAsStream方法获取流
* springboot项目中需要使用此种方法，因为jar包中没有一个实际的路径存放文件
*
* @param fileName
* @throws IOException
*/
public void func02(String fileName) throws IOException {
String filePath = URLDecoder.decode(fileName, "UTF-8"); // 如果路径中带有中文会被URLEncoder,因此这里需要解码
InputStream in = this.getClass().getClassLoader().getResourceAsStream(filePath);
getFileContent(in);
}
/**
* 通过ClassPathResource类获取，建议SpringBoot中使用
* springboot项目中需要使用此种方法，因为jar包中没有一个实际的路径存放文件
*
* @param fileName
* @throws IOException
*/
public void func03(String fileName) throws IOException {
ClassPathResource classPathResource = new ClassPathResource(fileName);
InputStream inputStream = classPathResource.getInputStream();
getFileContent(inputStream);
}
/**
* 通过绝对路径获取项目中文件的位置（通过new File("")获取当前的绝对路径，只是本地绝对路径，不能用于服务器）
*
* @param fileName
* @throws IOException
*/
public void func04(String fileName) throws IOException {
// 参数为空
File directory = new File("");
// 规范路径：getCanonicalPath() 方法返回绝对路径，会把 ..\ 、.\ 这样的符号解析掉
String rootCanonicalPath = directory.getCanonicalPath();
// 绝对路径：getAbsolutePath() 方法返回文件的绝对路径，如果构造的时候是全路径就直接返回全路径，如果构造时是相对路径，就返回当前目录的路径 + 构造 File 对象时的路径
String rootAbsolutePath =directory.getAbsolutePath();
System.out.println(rootCanonicalPath);
System.out.println(rootAbsolutePath);
String filePath = rootCanonicalPath + "\\chapter-2-springmvc-quickstart\\src\\main\\resources\\"+fileName;
getFileContent(filePath);
}
4838
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
