datax 从oracle增量抽取数据到pg，解决0x00问题

1、datax官方版本不支持pg的insert on conflict模式，需要改造代码如下：

•   将postgresqlwriter.java中的init改为如下代码

	@Override
		public void init() {
			this.originalConfig = super.getPluginJobConf();

			this.commonRdbmsWriterMaster = new CommonRdbmsWriter.Job(DATABASE_TYPE);
			this.commonRdbmsWriterMaster.init(this.originalConfig);
		}

• 修改plugin-rdbms-util目录下的writerUtil.java，找到getWriteTemplate并在类型为PostgreSQL判断条件下修改为如下内容：

 if(null!=writeMode&&(writeMode.trim().toLowerCase().startsWith("update")||writeMode.trim().toLowerCase().startsWith("replace"))){
                      writeDataSqlTemplate = new StringBuilder().append("INSERT INTO %s (")
                        .append(StringUtils.join(columnHolders, ","))
                        .append(") VALUES(").append(StringUtils.join(valueHolders, ","))
                        .append(")").append(onConFlictDoString(writeMode, columnHolders)).toString();
                    }else{
                          writeDataSqlTemplate = new StringBuilder().append("INSERT INTO %s (")
                        .append(StringUtils.join(columnHolders, ","))
                        .append(") VALUES(").append(StringUtils.join(valueHolders, ","))
                        .append(")").toString();
                    }

• 在writerUtil.java增加onConFlictDoString方法如下：

 public static String onConFlictDoString(String conflict, List<String> columnHolders) {
        conflict = conflict.replace("update", "");
        StringBuilder sb = new StringBuilder();
        sb.append(" ON CONFLICT ");
        sb.append(conflict);
        sb.append(" DO ");
        if (columnHolders == null || columnHolders.size() < 1) {
            sb.append("NOTHING");
            return sb.toString();
        }
        sb.append(" UPDATE SET ");
        boolean first = true;
        for (String column : columnHolders) {
            if (!first) {
                sb.append(",");
            } else {
                first = false;
            }
            sb.append(column);
            sb.append("=excluded.");
            sb.append(column);
        }
        return sb.toString();
    }

2、解决oracle中的零字符串,0x00

在plugin-rdbms-util下commonRdbmsWriter.java中找到fillPreparedStatementColumnType方法，使用replaceAll将0x00即\u0000 替换为空字符串修改内容如下：

case Types.CHAR:
                case Types.NCHAR:
                case Types.CLOB:
                case Types.NCLOB:
                case Types.VARCHAR:
                case Types.ARRAY:
                case Types.LONGVARCHAR:
                case Types.NVARCHAR:
                case Types.LONGNVARCHAR:
                    preparedStatement.setString(columnIndex + 1, null!=column
                            .asString()?column.asString().replaceAll("\\u0000",""):column
                            .asString());
                    break;

                case Types.SMALLINT:
                case Types.INTEGER:
                case Types.BIGINT:
                case Types.NUMERIC:
                case Types.DECIMAL:
                case Types.FLOAT:
                case Types.REAL:
                case Types.DOUBLE:
                    String strValue = column.asString();
                    if (emptyAsNull && "".equals(strValue)) {
                        preparedStatement.setString(columnIndex + 1, null);
                    } else {
                        preparedStatement.setString(columnIndex + 1, null!=strValue?strValue.replaceAll("\\u0000",""):strValue);
                    }
                    break;

3、打包postgresqlwriter，plugin-rdbms-util

    mvn clean install -DskipTests

    打包完成后，分别：

  将postgresqlwriter-0.0.1-SNAPSHOT.jar拷贝至datax/plugin/writer/postgresqlwriter目录下 

  将plugin-rdbms-util-0.0.1-SNAPSHOT.jar拷贝至datax/plugin/writer/postgresqlwriter/libs目录下