Hadoop 压缩文件命令

Hadoop 压缩文件命令

hadoop jar /usr/bin/hadoop/software/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.2.4.jar 
    -Dmapred.reduce.tasks=5 
    -Dmapred.output.compress=true 
    -Dmapred.compress.map.output=true 
    -Dmapred.output.compression.codec=org.apache.hadoop.io.compress.BZip2Codec 
    -input /home/wechat_log_2020_02_18.har 
    -output /home/wechat_log_2020_02_18.har.bz2 
    -mapper /bin/cat 
    -reducer /bin/cat 
    -inputformat org.apache.hadoop.mapred.TextInputFormat 
    -outputformat org.apache.hadoop.mapred.TextOutputFormat