sqoop import No Java type for SQL type -151 for column __28ArchiveTime

博客讨论了在使用sqoop从SQLServer导入数据时遇到的问题,即sqoop无法处理时间字段,因为没有匹配的Java类型。提出了两种解决方案:1) 创建视图将时间字段转换为字符串;2) 修改sqoop源码,但这需要复杂的开发、测试和部署流程。
摘要由CSDN通过智能技术生成
[root@node11 ~]# sqoop import --connect 'jdbc:sqlserver://192.xxx.xxx.xxx:1433;username=xxxx;password=xxxx;database=ssws' --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --columns "id,pump_type,_01SystemPressure,_28ArchiveTime,rowkey"  -table 'xxx'  --hbase-table 'xxx'  --column-family content --hbase-row-key rowkey -m 1
Warning: /root/TDH-Client/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /root/TDH-Client/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
2020-07-17 11:36:06,267 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-transwarp-6.0.0
2020-07-17 11:36:06,368 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apa
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
root@zhaosai conf]# sqoop import --connect jdbc:mysql://192.168.160.130:3306/mydb --username root -P --table news --hive-import --hive-table mydb.news --incremental append --check-column id --last-value 0 --split-by id --target-dir /hdfs://zhaosai:9000/user/hive/warehouse/news --num-mappers 1 23/06/07 17:23:56 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 Enter password: 23/06/07 17:24:04 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override 23/06/07 17:24:04 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc. 23/06/07 17:24:04 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 23/06/07 17:24:04 INFO tool.CodeGenTool: Beginning code generation Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 23/06/07 17:24:04 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `news` AS t LIMIT 1 23/06/07 17:24:04 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `news` AS t LIMIT 1 23/06/07 17:24:04 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/programs/hadoop-2.7.6 注: /tmp/sqoop-root/compile/b07035b094b6ac39b87f2ef11261c934/news.java使用或覆盖了已过时的 API。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 23/06/07 17:24:05 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/b07035b094b6ac39b87f2ef11261c934/news.jar 23/06/07 17:24:05 INFO tool.ImportTool: Maximal id query for free form incremental import: SELECT MAX(`id`) FROM `news` 23/06/07 17:24:05 ERROR tool.ImportTool: Import failed: java.io.IOException: java.sql.SQLSyntaxErrorException: Unknown column 'id' in 'field list' at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:322) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:511) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) Caused by: java.sql.SQLSyntaxErrorException: Unknown column 'id' in 'field list' at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120) at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1200) at org.apache.sqoop.tool.ImportTool.getMaxColumnId(ImportTool.java:238) at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:309)
最新发布
06-08

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值