有时需要InputStreamReader(InputStream in, Charset cs)这个构造来处理字符流。然而Charset不一定知道。这个时候就需要检测编码方式了。jchardet是firefox使用的字节流编码检测算法的java开源实现,协议为MPL(Mozilla Public License),对商业友好。下载源代码后发现示例并不怎么好使用,于是封装了一下。下面就封装类和使用Demo。
CharsetDetector 这个封装了内部实现,用户直接new这个类就可以检测字节流编码
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import org.mozilla.intl.chardet.nsDetector;
import org.mozilla.intl.chardet.nsICharsetDetectionObserver;
import org.mozilla.intl.chardet.nsPSMDetector;
public class CharsetDetector
{
private boolean found = false;
private String result;
private int lang;
public String[] detectChineseCharset(InputStream in) throws IOException
{
lang = nsPSMDetector.CHINESE;
String[] prob;
// Initalize the nsDetector() ;
nsDetector det = new nsDetector(lang);
// Set an observer...
// The Notify() will be called when a matching charset is found.
det.Init(new nsICharsetDetectionObserver()
{
public void Notify(String charset)
{
found = true;
result = charset;
}
});
BufferedInputStream imp = new BufferedInputStream(in);
byte[] buf = new byte[1024];
int len;
boolean isAscii = true;
while ((len = imp.read(buf, 0, buf.length)) != -1)
{
// Check if the stream is only ascii.
if (isAscii)
isAscii = det.isAscii(buf, len);
// DoIt if non-ascii and not done yet.
if (!isAscii)
{
if (det.DoIt(buf, len, false))
break;
}
}
imp.close();
in.close();
det.DataEnd();
if (isAscii)
{
found = true;
prob = new String[]
{
"ASCII"
};
} else if (found)
{
prob = new String[]
{
result
};
} else
{
prob = det.getProbableCharsets();
}
return prob;
}
public String[] detectAllCharset(InputStream in) throws IOException
{
try
{
lang = nsPSMDetector.ALL;
return detectChineseCharset(in);
} catch (IOException e)
{
throw e;
}
}
}
Demo:这个演示CharsetDetector用法示例
import java.io.IOException;
import java.net.URL;
public class Demo
{
public static void main(String[] args)throws IOException
{
CharsetDetector charDect = new CharsetDetector();
URL url = new URL("http://www.qq.com/");
String[] probableSet = charDect.detectChineseCharset(url.openStream());
for (String charset : probableSet)
{
System.out.println(charset);
}
}
}
附件为封装后的jar包
如果编码选择是 Unicode ,则检测的结果是 windows-1252