datax同步数据，从Oracle到Phoenix

Oracle数据源端文档说明
https://github.com/alibaba/DataX/blob/master/oraclereader/doc/oraclereader.md
Phoenix4.x数据目标端文档说明
https://github.com/alibaba/DataX/blob/master/hbase11xsqlwriter/doc/hbase11xsqlwriter.md

Oracle：oraclereader
Phoenix：hbase11xsqlwriter

实例json如下：
$符号开头的均为命令行参数，在执行的时候从服务器环境变量或者外部变量中获取，避免不必要的重复配置和难以改变（虽然一般情况下也不会变）；
其中，writer的部分有两个变化：
1. hbase.zookeeper.quorum的配置，zk的端口号不是默认的2181，这里对源码做了小小的改动，支持自定义的端口号（源码是拼好的2181）

private static void parseClusterConfig(HbaseSQLWriterConfig cfg, Configuration dataxCfg) {
  ...
  // zk_quorum配置中包含端口号，比如 zoo1,zoo2,zoo3:12181
  // 生成sql使用的连接字符串， 格式： jdbc:phoenix:zk_quorum:2181:/znode_parent
  cfg.connectionString = "jdbc:phoenix:" + zkQuorum + ":" + znode;
}

2. 多了phoenixConfig的配置，是因为“报错了！”
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[Hbasewriter-02], Description:[获取Hbase连接时出错.]. - 无法连接hbase集群，配置不正确或目标集群不可用，请检查配置和集群状态 或者 联系 HBase 管理员. - java.sql.SQLException: ERROR 726 (43M10):  Inconsistent namespace mapping properites.. Cannot initiate connection as SYSTEM:CATALOG is found but client does not have phoenix.schema.isNamespaceMappingEnabled enabled

这里增加了phoenix.schema.isNamespaceMappingEnabled和phoenix.schema.mapSystemTablesToNamespace的配置，默认是false。
具体源码见地址：
https://github.com/shaozhipeng/DataX

{
  "job": {
    "content": [
      {
        "reader": {
          "name": "oraclereader",
          "parameter": {
            "column": [],
            "connection": [
              {
                "fetchSize": "1024",
                "jdbcUrl": [
                  "$DW_ORCL_ZLXX_JDBCURL"
                ],
                "querySql": [
                  "select id, trade_id, amount, status, TO_CHAR(modify_time,'YYYYMMDDhh24miss') AS modify_time from zlxx.trade_refund where id in ('1215529','1215528','1205528','1195531','1195530')"
                ]
              }
            ],
            "password": "$DW_ORCL_ZLXX_PASSWORD",
            "username": "$DW_ORCL_ZLXX_USERNAME"
          }
        },
        "writer": {
          "name": "hbase11xsqlwriter",
          "parameter": {
            "batchSize": "256",
            "hbaseConfig": {
              "hbase.zookeeper.quorum": "192.168.10.11,192.168.10.12,192.168.10.13:12181",
              "zookeeper.znode.parent": "/hbase"
            },
            "phoenixConfig": {
                "phoenix.schema.isNamespaceMappingEnabled": "true",
                "phoenix.schema.mapSystemTablesToNamespace": "true"
            },
            "table": "ZLXX:WT_TRADE_REFUND",
            "nullMode": "empty",
            "column": [
              "ID",
              "TRADE_ID",
              "AMOUNT",
              "STATUS",
              "MODIFY_TIME"
            ],
            "encoding": "utf-8"
          }
        }
      }
    ],
    "setting": {
      "speed": {
        "channel": "10"
      }
    }
  }
}

更多干货文章，可扫码关注公众号，有惊喜！！