SqoopHDFS到MySQL报错：缺少lzo压缩包

最新推荐文章于 2023-04-27 14:37:07 发布

拉普达男孩

最新推荐文章于 2023-04-27 14:37:07 发布

阅读量645

点赞数 1

分类专栏：大数据文章标签： sqoop

本文链接：https://blog.csdn.net/ITwangnengjie/article/details/106633522

版权

Sqoop在从HDFS导入MySQL时遇到错误，日志显示缺少lzo压缩包。尽管在集群配置的xml文件中已包含所有压缩和解压缩格式，但问题依然存在。通过在Sqoop命令行中添加特定配置参数或在shell中添加lib Path，最终成功解决了这个问题。

摘要由CSDN通过智能技术生成

错误日志：

20/06/05 16:27:07 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
	at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
	at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:180)
	at org.apache.sqoop.mapreduce.CombineFileInputFormat.isSplitable(CombineFileInputFormat.java:157)
	at org.apache.sqoop.mapreduce.CombineFileInputFormat.getMoreSplits(CombineFileInputFormat.java:296)
	at org.apache.sqoop.mapreduce.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:256)
	at org.apache.sqoop.mapreduce.ExportInputFormat.getSplits(ExportInputFormat.java:73)
	at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:310)
	at org.apache.hadoop