sqoop import --connect jdbc:mysql://ip:3306/库名 --table 表名 --username 用户名 --password 密码
异常:
14/03/26 13:31:34 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0
14/03/26 13:31:34 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/03/26 13:31:35 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/03/26 13:31:35 INFO tool.CodeGenTool: Beginning code generation
14/03/26 13:31:35 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:782)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:670)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:693)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:242)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:225)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1298)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1110)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
解决办法:
网上各种说将mysql驱动放到$SQOOP_HOME/lib或者$HADOOP_HOME/lib下,足够幸运的话,你能成功,可是下次出现其他问题了,又各种找网上资料还不一定有用。言归正传。
先看执行脚本sqoop-import,里面啥也没有,除了调用了sqoop脚本,并且将输入命令$@一成不变全部传给了sqoop脚本;
好,看sqoop脚本,一堆环境的设置,最后调用了hadoop脚本,并且将输入命令$@一成不变全部传给了hadoop脚本;
好,看hadoop家目录下的bin/hadoop脚本,发现里面有CLASSPATH,echo打印CLASSPATH,发现里面根本就没有$HADOOP_HOME/lib目录,这是为啥?明明在/etc/profile中设置了啊??
往下看,在$HADOOP_HOME/etc/hadoop/hadoop-env.sh中里面有个HADOOP_CLASSPATH,将HADOOP_CLASSPATH设置为$HADOOP_HOME/lib/*,并且export一下;
次奥~hadoop脚本中的CLASSPATH变了,指到了$HADOOP_HOME/lib/*目录。
真相大白,hadoop2.0中CLASSPATH默认使用的是HADOOP_CLASSPATH等变量,而且没有将$HADOOP_HOME/lib/*加到HADOOP_CLASSPATH中,更不是/etc/profile中的CLASSPATH。
异常:
14/03/26 13:31:34 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0
14/03/26 13:31:34 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/03/26 13:31:35 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/03/26 13:31:35 INFO tool.CodeGenTool: Beginning code generation
14/03/26 13:31:35 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:782)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:670)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:693)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:242)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:225)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1298)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1110)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
解决办法:
网上各种说将mysql驱动放到$SQOOP_HOME/lib或者$HADOOP_HOME/lib下,足够幸运的话,你能成功,可是下次出现其他问题了,又各种找网上资料还不一定有用。言归正传。
先看执行脚本sqoop-import,里面啥也没有,除了调用了sqoop脚本,并且将输入命令$@一成不变全部传给了sqoop脚本;
好,看sqoop脚本,一堆环境的设置,最后调用了hadoop脚本,并且将输入命令$@一成不变全部传给了hadoop脚本;
好,看hadoop家目录下的bin/hadoop脚本,发现里面有CLASSPATH,echo打印CLASSPATH,发现里面根本就没有$HADOOP_HOME/lib目录,这是为啥?明明在/etc/profile中设置了啊??
往下看,在$HADOOP_HOME/etc/hadoop/hadoop-env.sh中里面有个HADOOP_CLASSPATH,将HADOOP_CLASSPATH设置为$HADOOP_HOME/lib/*,并且export一下;
次奥~hadoop脚本中的CLASSPATH变了,指到了$HADOOP_HOME/lib/*目录。
真相大白,hadoop2.0中CLASSPATH默认使用的是HADOOP_CLASSPATH等变量,而且没有将$HADOOP_HOME/lib/*加到HADOOP_CLASSPATH中,更不是/etc/profile中的CLASSPATH。
将mysql驱动放到$HADOOP_HOME/lib/目录下,并且将目录加到HADOOP_CLASSPATH中。搞定!