Centos7 Datax-web安装与使用

环境
MySQL (5.5+) 必选,

JDK (1.8.0_xxx) 必选

DataX 必选

Python (2.x,3.x??)

datax-admin

/home/cws/program/datax-web-2.1.2/modules/datax-admin/bin/env.properties
# environment variables

JAVA_HOME=/home/cws/program/jdk1.8

WEB_LOG_PATH=${BIN}/../logs
WEB_CONF_PATH=${BIN}/../conf

DATA_PATH=${BIN}/../data

SERVER_PORT=9527

#PID_FILE_PATH=${BIN}/dataxadmin.pid


# mail account
#MAIL_USERNAME=""
#MAIL_PASSWORD=""


#debug
#REMOTE_DEBUG_SWITCH=true
#REMOTE_DEBUG_PORT=7003

#PYTHON_PATH=/home/cws/program/datax/bin/datax.py
/home/cws/program/datax-web-2.1.2/modules/datax-admin/conf/bootstrap.properties
#Database
DB_HOST=192.168.0.88
DB_PORT=3306
DB_USERNAME=root
DB_PASSWORD=root
DB_DATABASE=dataxweb

datax-executor

/home/cws/program/datax-web-2.1.2/modules/datax-executor/bin/env.properties 

# environment variables

JAVA_HOME=/home/cws/program/jdk1.8

SERVICE_LOG_PATH=${BIN}/../logs
SERVICE_CONF_PATH=${BIN}/../conf
DATA_PATH=${BIN}/../data


## datax json文件存放位置
JSON_PATH=${BIN}/../json


## executor_port
EXECUTOR_PORT=9999


## 保持和datax-admin端口一致
DATAX_ADMIN_PORT=

## PYTHON脚本执行位置
#PYTHON_PATH=/home/hadoop/install/datax/bin/datax.py
PYTHON_PATH=/home/cws/program/datax/bin/datax.py


## dataxweb 服务端口
SERVER_PORT=9504

#PID_FILE_PATH=${BIN}/service.pid


#debug 远程调试端口
#REMOTE_DEBUG_SWITCH=true
#REMOTE_DEBUG_PORT=7004
"writer": {
          "name": "mysqlwriter",
          "parameter": {
            "writeMode": "replace",
            "username": "yRjwDFuoPKlqya9h9H2Amg==",
            "password": "yRjwDFuoPKlqya9h9H2Amg==",
            "column": [
              "`id`",
              "`name`"
            ],
            "connection": [
              {
                "table": [
                  "BSS_Business"
                ],
                "jdbcUrl": "jdbc:mysql://192.168.0.88:3306/test?characterEncoding=utf-8&useSSL=false"
              }
            ]
          }

writeMode模式选择
总共三种:insert/replace/update

对于导入端是 mysql的话,

insert 就是单纯的插入。不判重。如果遇到主键冲突,就报错提示。

replace,原理类似 mysql 的 replace into 的原理,

update,原理类似 mysql 的on duplicate key update的原理。

  "job": {
    "setting": {
      "speed": {
        "channel": 3,
        "byte": 1048576  ##报错请删除byte
      },
  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值