今天在运行sqoop的时候出现了mysql的bug导致数据不能导入
具体的异常如下所示:
java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@5b239d7d is still active.
No statements may be issued when any streaming result sets are open and in use on a given connection.
Ensure that you have called .close() on any active streaming result sets before attempting more queries.
ERROR sqoop.Sqoop: Got exception running Sqoop:
java.lang.NullPointerException
19/09/20 15:14:55 ERROR manager.SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@5b239d7d is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@5b239d7d is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:930)
at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2622)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1846)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2060)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2536)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2465)
at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1383)
at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:2931)
at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:582)
at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:441)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:286)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
at org.apache.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:126)
at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:188)
at org.apache.sqoop.tool.CreateHiveTableTool.run(CreateHiveTableTool.java:58)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
19/09/20 15:14:55 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:175)
at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:188)
at org.apache.sqoop.tool.CreateHiveTableTool.run(CreateHiveTableTool.java:58)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
[root@hadoop01 ~]#
执行语句为:
sqoop create-hive-table \
--connect jdbc:mysql://hadoop01:3306/test \
--username root --password root \
--table testlast \
--hive-database qf24 \
--hive-table testlast
原因没有指定驱动
加上这句话就可以
–driver com.mysql.jdbc.Driver \
具体原因应该是
连接Jar包版本问题:
mysql-connector-java-5.1.31-bin ,替换原有jar包
就是下载新的jar包 然后就可以了
但是不知道用哪个版本啊
so 我就每次执行用上面的那句话 指定驱动即可