从控制台输入数据,然后对其进行压缩输出到控制台:
package Demo5;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.io.compress.CompressionCodec;
import org.apache.hadoop.io.compress.CompressionOutputStream;
import org.apache.hadoop.util.ReflectionUtils;
public class StreamCompressor {
public static void main(String[] args) throws Exception {
String codecCalssname="org.apache.hadoop.io.compress.GzipCodec";
//加载类
Class<?> codecClass=Class.forName(codecCalssname);
Configuration conf=new Configuration();
//创建实例
CompressionCodec codec=(CompressionCodec)ReflectionUtils.newInstance(codecClass, conf);
//输出
CompressionOutputStream out=codec.createOutputStream(System.out);
IOUtils.copyBytes(System.in, out, 4096,false);
out.finish();
}
}
打包放到集群运行:
输入命令:
echo "Test" |hadoop jar StreamCompressor.jar | gunzip