Sqoop 数据导入问题

最新推荐文章于 2022-12-02 14:26:49 发布

philadelphiazt

最新推荐文章于 2022-12-02 14:26:49 发布

阅读量1.2k

点赞数

分类专栏： Hadoop 文章标签： sqoop hive jdbc mysql

本文链接：https://blog.csdn.net/philadelphiazt/article/details/51219166

版权

Hadoop 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

今天在使用sqoop将数据导入到hive中时一直遇到这个问题

sqoop import --connect jdbc:mysql://172.22.35.145:3306/test --username hive --P  --table  student --fields-terminated-by '\t' -m 1 --target-dir=/hive/warehouse/

16/04/22 12:45:47 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
Enter password: 
16/04/22 12:45:49 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
16/04/22 12:45:49 INFO tool.CodeGenTool: Beginning code generation
16/04/22 12:45:50 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student` AS t LIMIT 1
16/04/22 12:45:50 ERROR manager.SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@61220ed4 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@61220ed4 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931)
    at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569)
    at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1521)
    at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003)
    at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602)
    at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445)
    at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:286)
    at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
    at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
    at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
    at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
    at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
    at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
16/04/22 12:45:50 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
    at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
    at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

经过google得知原来是mysql的jdbc驱动包不匹配导致的。
http://stackoverflow.com/questions/29162447/sqoop-import-issue-with-mysql

Try with mysql-connector-java-5.1.31.jar, it is compatable with sqoop
1.4.5.

mysql-connector-java-5.1.17.jar driver does not work with sqoop 1.4.5.

refer :

https://issues.apache.org/jira/browse/SQOOP-1400