环境:
sqoop版本:1.4.7.
hadoop2.7.6.
hive2.3.1.
导入参数:
bin/sqoop import --connect jdbc:mysql://192.168.0.55:3306/test?tinyInt1isBit=false --username root --password '123456' --table test_table --fields-terminated-by '\001' --hive-import --hive-table ods.test_table --hive-overwrite --hive-drop-import-delims --delete-target-dir --m 1 --null-string '\\N' --null-non-string '\\N'
运行后报错,报错内容:
20/08/31 17:54:24 ERROR hive.HiveConfig: Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.
20/08/31 17:54:24 ERROR tool.ImportTool: Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50)
at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:392)
at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:379)
at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:337)
at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:44)
... 12 more
网上查了很多资料,说是要配置hive conf,进行如下配置:
vim /etc/profile
export HIVE_HOME=/home/office/hive-2.3.1
export HIVE_CONF_DIR=$HIVE_HOME/conf
export PATH=:$HIVE_HOME/bin:$PATH
配置后,再进行导入,还是报上面的错.
网上再仔细查找,有网友提到,可能是sqoop的lib中缺少hive的依赖包.
进入到sqoop的目录下,果然没有hive的依赖包.
进入到hive中.将hive-exec-2.3.1.jar
拷到sqoop的lib目录下.
将hive的依赖包拷过来以后,成功导入数据.解决.