Sqoop java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf

问题描述

[root@hadoop01 profile.d]# sqoop create-hive-table --connect jdbc:mysql://hadoop01:3306/userdb --table emp_add --username root --password 123456 --hive-table test.emp_add_sp 
Warning: /export/servers/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /export/servers/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /export/servers/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /export/servers/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
19/11/29 11:01:57 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
19/11/29 11:01:57 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/11/29 11:01:57 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
19/11/29 11:01:57 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
19/11/29 11:01:57 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/11/29 11:01:57 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `emp_add` AS t LIMIT 1
19/11/29 11:01:57 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `emp_add` AS t LIMIT 1
19/11/29 11:01:59 INFO hive.HiveImport: Loading uploaded data into Hive
19/11/29 11:01:59 ERROR hive.HiveConfig: Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.
19/11/29 11:01:59 ERROR tool.CreateHiveTableTool: Encountered IOException running create table job: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
        at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50)
        at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:392)
        at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:379)
        at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:337)
        at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
        at org.apache.sqoop.tool.CreateHiveTableTool.run(CreateHiveTableTool.java:57)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:44)
        ... 11 more

[root@hadoop01 profile.d]# 

解决方案

百度搜到的原因：缺少了hive-common-2.3.3.jar包，在hive的lib目录下，拷贝到sqoop的lib目录下即可。

cp /export/servers/hive-1.1.0-cdh5.14.0/lib/hive-common-1.1.0-cdh5.14.0.jar /export/servers/sqoop-1.4.7.bin__hadoop-2.6.0/lib/

最终结果

问题解决