大文件上传及NIO

最新推荐文章于 2022-06-29 16:24:14 发布

chenxuezhou

最新推荐文章于 2022-06-29 16:24:14 发布

阅读量1.3k

点赞数

分类专栏： java基础文章标签：大文件文件分片线程池定时任务

本文链接：https://blog.csdn.net/u012557298/article/details/78983171

版权

java基础专栏收录该内容

7 篇文章 0 订阅

订阅专栏

IO流

编码

UTF-8 编码与 GBK 和 GB2312 不同，不用查码表，所以在编码效率上 UTF-8 的效率会更好，所以在存储中文字符时 UTF-8 编码比较理想
Java 的内存编码就是采用 UTF-16 编码。但是它不适合在网络之间传输，因为网络传输容易损坏字节流，一旦字节流损坏将很难恢复，想比较而言 UTF-8 更适合网络传输，在编码效率上介于 GBK 和 UTF-16 之间，所以 UTF-8 在编码效率上和编码安全性上做了平衡，是理想的中文编码方式
UTF-16编码效率高
char转byte造成字节丢失，因此StringInputStream少用，也是被废弃的

文件追加

文件追加【字节流也是支持的，在构造输出流时，设定append是否为追加】

简要代码【面试总有很多问题，现在稍加回顾】


package javax.servlet.http;

import java.io.File;
import java.io.FileOutputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.OutputStream;
import java.io.RandomAccessFile;


public class AppendContentToFile {

    final static String path = "D:\\vue\\npm-debug.log";

    final static String appendText = "追加内容";

    public static void main(String[] args) throws IOException {
        OutputStream outputStream=new FileOutputStream(path,true);
        outputStream.write(appendText.getBytes());
        outputStream.close();
        AppendContentToFile a = new AppendContentToFile();
        a.method1();
        //a.method2("D:\\vue\\npm-debug.log", "222222222222222");
        a.method3("D:\\vue\\npm-debug.log", "33333333333");
    }

    public void method1() {
        FileWriter fw = null;
        try {
            //如果文件存在，则追加内容；如果文件不存在，则创建文件
            File f = new File(path);
            fw = new FileWriter(f, true);
            fw.write(appendText);
            //fw.write(appendText,0,appendText.length());
        } catch (IOException e) {
            e.printStackTrace();
        }
        //PrintWriter pw = new PrintWriter(fw);
        //pw.println("追加内容");
        //pw.flush();
        try {
            fw.flush();
            //pw.close();
            fw.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    //
    //public static void method2(String file, String conent) {
    //    BufferedWriter out = null;
    //    try {
    //        out = new BufferedWriter(new OutputStreamWriter(
    //                new FileOutputStream(file, true)));
    //        out.write(conent+"\r\n");
    //    } catch (Exception e) {
    //        e.printStackTrace();
    //    } finally {
    //        try {
    //            out.close();
    //        } catch (IOException e) {
    //            e.printStackTrace();
    //        }
    //    }
    //}

    public static void method3(String fileName, String content) {
        try {
            // 打开一个随机访问文件流，按读写方式
            RandomAccessFile randomFile = new RandomAccessFile(fileName, "rw");
            // 文件长度，字节数
            long fileLength = randomFile.length();
            // 将写文件指针移到文件尾。
            randomFile.seek(fileLength);
            randomFile.writeBytes(content + "\r\n");
            randomFile.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

IO实践应用

csv文件生成

带BOM标志头

new byte[] { (byte) 0xEF, (byte) 0xBB,(byte) 0xBF }

StringBufferInputStream、ByteArrayInputStream使用在oss上有区别

断点续传

文件API上传逻辑

暂停操作的日志逻辑


DEBUG 2018-12-06 16:03:18,345 ProgressCalculator pool-4-thread-1 Calculated progress for file 84ac57fc-33a4-4842-b2e5-b91a1d0efa36: Uploaded 22466560/219124184 Bytes(10.25289%)at rate: 100.0KB/s. Finishing in 32m. 
DEBUG 2018-12-06 16:03:18,718 UploadServlet qtp1403649277-18 Processing action getProgress 
DEBUG 2018-12-06 16:03:19,345 ProgressCalculator pool-4-thread-1 Calculated progress for file 84ac57fc-33a4-4842-b2e5-b91a1d0efa36: Uploaded 22568960/219124184 Bytes(10.299621%)at rate: 100.0KB/s. Finishing in 32m. 
DEBUG 2018-12-06 16:03:19,729 UploadServlet qtp1403649277-37 Processing action getProgress 
DEBUG 2018-12-06 16:03:20,345 ProgressCalculator pool-4-thread-1 Calculated progress for file 84ac57fc-33a4-4842-b2e5-b91a1d0efa36: Uploaded 22664815/219124184 Bytes(10.343366%)at rate: 100.0KB/s. Finishing in 32m. 
DEBUG 2018-12-06 16:03:20,699 UploadServlet qtp1403649277-18 Processing action pauseFile
//关闭文件输出流 
DEBUG 2018-12-06 16:03:20,748 UploadServletAsyncProcessor$WriteChunkToFileTask pool-3-thread-3 User cancellation detected. 
DEBUG 2018-12-06 16:03:20,748 UploadServletAsyncProcessor$WriteChunkToFileTask pool-3-thread-3 completion for 84ac57fc-33a4-4842-b2e5-b91a1d0efa36. closing file stream 
DEBUG 2018-12-06 16:03:20,748 UploadServletAsyncProcessor$WriteChunkToFileTask pool-3-thread-3 Closing FileOutputStream of 84ac57fc-33a4-4842-b2e5-b91a1d0efa36 
DEBUG 2018-12-06 16:03:20,771 UploadServletAsyncListenerAdapter qtp1403649277-37 Done: (84ac57fc-33a4-4842-b2e5-b91a1d0efa36)
//上下文监听器asyncContext.addListener 
DEBUG 2018-12-06 16:03:20,771 UploadServletAsync$1 qtp1403649277-37 request [POST /javaLargeFileUploaderAsyncServlet?action=upload&fileId=84ac57fc-33a4-4842-b2e5-b91a1d0efa36&crc=c1ac55d1]@530800963 org.eclipse.jetty.server.Request@1fa36143 completed. 
DEBUG 2018-12-06 16:03:20,771 UploadServletAsyncProcessor qtp1403649277-37 resetting token bucket for 84ac57fc-33a4-4842-b2e5-b91a1d0efa36 
DEBUG 2018-12-06 16:03:20,771 UploadProcessingOperationManager qtp1403649277-37 stopping operation for client a877f7e5-a46a-4109-8798-aae8f77a4ebd and file 84ac57fc-33a4-4842-b2e5-b91a1d0efa36
//PostConstruct 类似Servlet中的init，为啥打出日志 
DEBUG 2018-12-06 16:05:24,116 RateLimiterConfigurationManager$2 pool-4-thread-1 removal from requestconfig of a877f7e5-a46a-4109-8798-aae8f77a4ebd because of EXPIRED 
DEBUG 2018-12-06 16:05:24,117 RateLimiterConfigurationManager pool-4-thread-1 inactivity detected for client a877f7e5-a46a-4109-8798-aae8f77a4ebd 
DEBUG 2018-12-06 16:05:26,706 RateLimiterConfigurationManager$2 pool-4-thread-1 removal from requestconfig of 84ac57fc-33a4-4842-b2e5-b91a1d0efa36 because of EXPIRED

123123

JavaLargeFileUploaderService#updateEntity更新缓存并生成这个分析文件

文件系统上包含有关上传文件的信息。扩展它。必须调用StaticStateManager.init(Class)，使用您定义的类类型扩展这个类。

<com.am.jlfu.staticstate.entities.StaticStatePersistedOnFileSystemEntity>

 <fileStates>

   <entry>

     <java.util.UUID>

       <mostSigBits>458929293533072433</mostSigBits>

       <leastSigBits>-5771342430987235874</leastSigBits>

     </java.util.UUID>

     <com.am.jlfu.staticstate.entities.StaticFileState>

       <absoluteFullPathOfUploadedFile>C:\Users\WB-YZL~1\AppData\Local\Temp\lala2733719466718018790test\3bc6ed03-a431-46eb-9636-1fa56790e903\zenameofzefile.owf</absoluteFullPathOfUploadedFile>

       <staticFileStateJson>

         <originalFileName>zenameofzefile.owf</originalFileName>

         <originalFileSizeInBytes>121123456</originalFileSizeInBytes>

         <creationDate>2018-01-05 15:36:27.646 CST</creationDate>

         <crcedBytes>0</crcedBytes>

         <firstChunkCrc>lala</firstChunkCrc>

       </staticFileStateJson>

     </com.am.jlfu.staticstate.entities.StaticFileState>

   </entry>

 </fileStates>

</com.am.jlfu.staticstate.entities.StaticStatePersistedOnFileSystemEntity>

SequenceInputStream

多个inputStream集合没有什么用

https://blog.csdn.net/liuyancainiao/article/details/80957809

大文件拷贝处理——》普通IO，BufferIo，ChannelIO流

需要性能测试

NIO详细学习参考并发编程网

前端知识点设计：

window.sleep = function sleep(delay) {
            var start = (new Date()).getTime();
            while ((new Date()).getTime() - start < delay) {
                continue;
            }
        }
        var a={}
        function test(a) {
            console.log(123)
            // if (a.xhr){
            //     a.xhr.abort()
            // }
        }

        function uploadComplete() {
            console.log("uploadComplete")
        }

        function uploadFailed() {
            console.log("uploadFailed")
        }

        function reqListener () {
            console.log(this.responseText);
        }

        function updateProgress (oEvent) {
            if (oEvent.lengthComputable) {
                var percentComplete = oEvent.loaded / oEvent.total * 100;
                console.log(percentComplete)
            } else {
                // 总大小未知时不能计算进程信息
            }
        }

        function go() {
            var request = new XMLHttpRequest();
            a.xhr=request
            request.open('GET', 'http://localhost:8080/javaLargeFileUploaderServlet?action=getConfig');
            request.addEventListener('abort', function (event) {
                console.log(event)
            }, false)//abort触发是执行abort()
            request.addEventListener("load", uploadComplete, false);
            request.addEventListener("error", uploadFailed, false);
            request.addEventListener("load", reqListener);
            request.addEventListener("progress", updateProgress);

            request.onreadystatechange = function () {
                if (request.readyState === 4 && request.status === 200) {
                    setTimeout('go', 1000)
                }
            }
            request.send(null);
        }

        go()
        test(a)

参考文献：

https://developer.mozilla.org/zh-CN/docs/Web/API/XMLHttpRequest/Using_XMLHttpRequest

https://developer.mozilla.org/zh-CN/docs/Web/API/XMLHttpRequest

编码：https://blog.csdn.net/qq_35038153/article/details/79690608

咨询：QQ1134133509