对io stream使用缓存数组大小的探索与思考

实验用src

测试类

public class TestIOBuffer {
	
	public static void main(String[] args) {
		
		File srcFile = new File("E:\\np-eclipse-workspaces\\learn_poi\\test\\file\\telephones.txt");
		File targetFile = new File("E:\\np-eclipse-workspaces\\learn_poi\\test\\file\\telephones_stream.txt");
		
		FileCopyUtils.copyByStream1(srcFile, targetFile);
		
		FileCopyUtils.copyByStream2(srcFile, targetFile);
		
		FileCopyUtils.copyByStream3(srcFile, targetFile);
		
	}

}

文件复制工具类，用文件复制的方式测试io性能

public class FileCopyUtils {
	
	/**
	 * 通过Stream实现文件复制
	 * 不使用缓存数组
	 * @param srcFile
	 * @param targetFile
	 */
	public static void copyByStream1(File srcFile, File targetFile) {
		Instant startTime = Instant.now();
		
		InputStream in = null;
		OutputStream out = null;
		
		try {
			// 校验
			if(!srcFile.exists()) {
				throw new IllegalArgumentException("源文件不存在！");
			}
			if(targetFile.exists()) {
				targetFile.delete();
			}
			
			// 执行
			in = new FileInputStream(srcFile);
			out = new FileOutputStream(targetFile);
			
			int value = -1;
			while((value = in.read()) != -1) {
				out.write(value);
			}
			
		} catch(IOException e) {
			e.printStackTrace();
		} finally {
			// 结束
			if(in != null) {
				try {
					in.close();
				} catch (IOException e) {
					e.printStackTrace();
				}
			}
			if(out!= null) {
				try {
					out.close();
				} catch (IOException e) {
					e.printStackTrace();
				}
			}
		}
		
		Instant entTime = Instant.now();
		System.out.println("copy file by stream without buffer, total time spent : " + Duration.between(startTime, entTime).toMillis() + "ms");
	}
	
	/**
	 * 通过Stream实现文件复制
	 * 使用缓存数组，但较小
	 * @param srcFile
	 * @param targetFile
	 */
	public static void copyByStream2(File srcFile, File targetFile) {
		Instant startTime = Instant.now();
		
		InputStream in = null;
		OutputStream out = null;
		
		try {
			// 校验
			if(!srcFile.exists()) {
				throw new IllegalArgumentException("源文件不存在！");
			}
			if(targetFile.exists()) {
				targetFile.delete();
			}
			
			// 执行
			in = new FileInputStream(srcFile);
			out = new FileOutputStream(targetFile);
			
			byte[] buffer = new byte[1024];
			int readLenth = -1;
			while((readLenth = in.read(buffer)) != -1) {
				out.write(buffer, 0, readLenth);
			}
			
		} catch(IOException e) {
			e.printStackTrace();
		} finally {
			// 结束
			if(in != null) {
				try {
					in.close();
				} catch (IOException e) {
					e.printStackTrace();
				}
			}
			if(out!= null) {
				try {
					out.close();
				} catch (IOException e) {
					e.printStackTrace();
				}
			}
		}
		
		Instant entTime = Instant.now();
		System.out.println("copy file by stream with small buffer, total time spent : " + Duration.between(startTime, entTime).toMillis() + "ms");
	}
	
	/**
	 * 通过Stream实现文件复制
	 * 使用缓存数组，但较大
	 * @param srcFile
	 * @param targetFile
	 */
	public static void copyByStream3(File srcFile, File targetFile) {
		Instant startTime = Instant.now();
		
		InputStream in = null;
		OutputStream out = null;
		
		try {
			// 校验
			if(!srcFile.exists()) {
				throw new IllegalArgumentException("源文件不存在！");
			}
			if(targetFile.exists()) {
				targetFile.delete();
			}
			
			// 执行
			in = new FileInputStream(srcFile);
			out = new FileOutputStream(targetFile);
			
			byte[] buffer = new byte[1024 * 2014 * 100];
			int readLenth = -1;
			while((readLenth = in.read(buffer)) != -1) {
				out.write(buffer, 0, readLenth);
			}
			
		} catch(IOException e) {
			e.printStackTrace();
		} finally {
			// 结束
			if(in != null) {
				try {
					in.close();
				} catch (IOException e) {
					e.printStackTrace();
				}
			}
			if(out!= null) {
				try {
					out.close();
				} catch (IOException e) {
					e.printStackTrace();
				}
			}
		}
		
		Instant entTime = Instant.now();
		System.out.println("copy file by stream with large buffer, total time spent : " + Duration.between(startTime, entTime).toMillis() + "ms");
	}


}

实验变量

通过查看源码可知，三个复制文件的方法中，只有缓存数组的大小不一样，所以，实验变量是缓存数组大小。

实验现象

程序运行时，查看服务器的cpu使用率和内存使用情况，可以看到，
当不使用缓存数组时，cpu会有明显增长，且持续时间长，内存几乎不增长。
当使用较小的缓存数组（1024 byte）时，cup会有明显增长，但持续时间不长，内存只有些微增长。
当使用较大的缓存数组（1024 * 1024 * 1024 byte）时，cup会有明显增长，但持续时间不长，内存有明显增长。

实验结果 console output

1、源文件src.txt文件大小为743KB时

copy file by stream without buffer, total time spent : 2344ms
copy file by stream with small buffer, total time spent : 60ms
copy file by stream with large buffer, total time spent : 1023ms

2、源文件src.txt文件大小为7430KB时

copy file by stream without buffer, total time spent : 22135ms
copy file by stream with small buffer, total time spent : 95ms
copy file by stream with large buffer, total time spent : 1058ms

3、源文件src.txt文件大小为74300KB时

copy file by stream without buffer, total time spent : 222104ms
copy file by stream with small buffer, total time spent : 429ms
copy file by stream with large buffer, total time spent : 1767ms

实验结论

不用缓存的时候，程序运行完，耗时最长，随着缓存的增大，耗时逐渐变短，但会遇到极限值，之后，随着缓存的增大，耗时又开始变长。