lzo文件默认不支持split,创建索引后支持split,这样作为map输入时就可以将文件分割成多个map,否则只能有一个map


创建索引,执行:


hadoop jar /usr/hdp/2.2.6.0-2800/hadoop/lib/hadoop-lzo-0.6.0.2.2.6.0-2800.jar  com.hadoop.compression.lzo.DistributedLzoIndexer  hdfs://hadoop001:8020/apps/hive/warehouse/hly.db/lzo_aa/1.txt.lzo


说明:可以只写到hdfs://hadoop001:8020/apps/hive/warehouse/hly.db/lzo_aa/ 目录层


索引文件:


 hdfs://hadoop001:8020/apps/hive/warehouse/hly.db/lzo_aa/1.txt.lzo.index