记一次文件下载乱码排查

记一次文件下载乱码排查

场景

从服务端读取一个zip文件(里面可能有多个文件)的内容，然后转成Properties。

现象

本地转，乱码问题不出现，只有发布预生产的时候发生这样的问题。

代码

 public static Properties readProperties(String uri) throws IOException{
        return readProperties(new ByteArrayInputStream(IOUtils.toByteArray(new URL(uri))));
    }

    public static Properties readProperties(InputStream inputStream){
        Properties properties=new Properties();
        ZipInputStream zipInputStream=new ZipInputStream(inputStream);
        try {
            while(zipInputStream.getNextEntry()!=null){
                properties.load(new ByteArrayInputStream(IOUtils.toByteArray(zipInputStream)));
            }
        } catch (IOException e) {
            logger.error("解析文件失败",e);
        } finally {
            IOUtils.closeQuietly(zipInputStream);
        }
        return properties;
    }

排查

• 手动下载远程的zip文件打开，发现没有乱码，且其编码方式是UTF-8。
• ZipInputStream入参的charset跟内容无关。

    /* @param charset
     *        The {@linkplain java.nio.charset.Charset charset} to be
     *        used to decode the ZIP entry name (ignored if the
     *        <a href="package-summary.html#lang_encoding"> language
     *        encoding bit</a> of the ZIP entry's general purpose bit
     *        flag is set).
     */
public ZipInputStream(InputStream in, Charset charset){}

• 预生产环境，System.getProperty(“file.encoding”) = UTF-8 。

• 使用public static String toString(URI uri, Charset encoding){}发现没有乱码，其中encoding=utf-8。

因为缺乏流相关的知识，所以查阅了很多资料。其中给以我启发的是这句话

想起了java.io包中Reader/Writer和Stream的区别
大概是：Reader/Write是处理编码文本的，而InputStream/OutputStream只把数据当作2进制流

Difference between Reader and InputStream classes
InputStreams are used to read bytes from a stream . It grabs the data byte by byte without performing any kind of translation. So they are useful for binary data such as images, video and serialized objects.

Readers on the other hand are character streams so they are best used to read character data. If the information you are reading is all text, then the Reader will take care of the character decoding for you and give you unicode characters from the raw input stream. If you are reading any type of text,

this is the stream to use.

InputStreamReader 是字节流通向字符流的桥梁：它使用指定的 charset 读取字节并将其解码为字符。它使用的字符集可以由名称指定或显式给定，或者可以接受平台默认的字符集。 

解决

  properties.load(new ByteArrayInputStream(IOUtils.toByteArray(zipInputStream)));
   改成
  properties.load(new InputStreamReader(zipInputStream,"UTF-8"));

参考

软件开发者必须要知道Unicode和字符集，这是最起码的要求(别找借口)

Difference between Reader and InputStream classes

InputStream和Reader的区别

于java.util.Properties读取中文乱码的正确解决方案（不要再用native2ascii.exe了）
Java IO包装流如何关闭？