做一个SQL Server to PostgreSQL的ETL,之前偷懒,表结构都是用Navicat导入导出的,然后数据用Navicat导入没问题,但是用工具就死活报错,如下:
2022-03-17 00:06:54.698 [0-0-0-writer-targetInsert] ERROR StdoutPluginCollector -
org.postgresql.util.PSQLException: 错误: 无效的类型 integer 输入语法: "true"
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198) ~[postgresql-9.3-1102-jdbc4.jar:na]
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927) ~[postgresql-9.3-1102-jdbc4.jar:na]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) ~[postgresql-9.3-1102-jdbc4.jar:na]
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561) ~[postgresql-9.3-1102-jdbc4.jar:na]
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419) ~[postgresql-9.3-1102-jdbc4.jar:na]
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412) ~[postgresql-9.3-1102-jdbc4.jar:na]
at com.xjgreat.di.plugin.rdbms.writer.CommonRdbmsWriter$Task.doOneInsert(CommonRdbmsWriter.java:824) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.xjgreat.di.plugin.rdbms.writer.CommonRdbmsWriter$Task.doBatchInsert(CommonRdbmsWriter.java:803) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.xjgreat.di.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWriterWithType(CommonRdbmsWriter.java:745) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.xjgreat.di.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:697) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.xjgreat.di.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:473) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.xjgreat.di.plugin.writer.postgresqlwriter.PostgresqlWriter$Task.startWrite(PostgresqlWriter.java:95) [postgresqlwriter-0.0.1-SNAPSHOT.jar:na]
at com.xjgreat.di.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:59) [di-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
2022-03-17 00:06:54.706 [0-0-0-writer-targetInsert] ERROR StdoutPluginCollector - 脏数据:
最后发现是因为某个字段有问题造成的。
这个字段在SQL Server里是bit,如下:
Navicat导入到PostgreSQL就变成了smallint ,我一开始以为是smallint问题,改成int还是报上面的错。
然后搜索一番,发现原来SQL Server的bit对应的其实就是PostgreSQL的布尔值:
PostgreSQL vs. SQL Server (MSSQL) - Extremely Detailed Comparison
将字段改为Boolean就行了。ps,直接改不行,需要删掉这个字段重新添加。