SPARK报错分析
问题表现:
User class threw exception: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
问题排查:
- 确认spark-env.sh中是否配置有/usr/hdp/current/hadoop-client/lib/hadoop-lzo-*jar
- 若有,则确认文件是否存在(ll /usr/hdp/current/hadoop-client/lib/hadoop-lzo-*jar)
- 没有,配置