hadoop文件IO

InputStreamReader 是字节流通向字符流的桥梁：它使用指定的 charset 读取字节并将其解码为字符。它使用的字符集可以由名称指定或显式给定，或者可以接受平台默认的字符集。
InputStreamReader(InputStream in)
InputStreamReader(InputStream in, String charsetName)
 
OutputStreamWriter 是字符流通向字节流的桥梁：可使用指定的 charset 将要写入流中的字符编码成字节。它使用的字符集可以由名称指定或显式给定，否则将接受平台默认的字符集。
OutputStreamWriter(OutputStream out)
OutputStreamWriter(OutputStream out, String charsetName)


//普通文件的字符转码IO，标准IO
//读
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream),"charsetName");
//写
BufferedWriter bw = new BufferedWriter(new OutputStreamwWriter(new FileOutputStream),"charsetName");


//hadoop下IO加转码
//读
FSDataInputStream fsDataInputStream = FileSystem.get(conf).open(path);
BufferedReader br = new BufferedReader(new InputStreamReader(fsDataInputStream,"charsetName"));
//写
FSDataOutputStream fsDataOutputStream = FileSystem.get(conf).create(path);
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(fsDataOutputStream, "charsetName"));


//使用hadoop自带LineReader工具
LineReader lineReader = new LineReader(FileSystem.get(conf).open(new Path(file)));
Text text = new Text();//注意此处应该实例化Text，见下图
while (lineReader.readLine(text) > 0){
//自己实现

}

参考文献：http://blog.csdn.net/waldmer/article/details/13503867