环境
MySQL (5.5+) 必选,
JDK (1.8.0_xxx) 必选
DataX 必选
Python (2.x,3.x??)
datax-admin
/home/cws/program/datax-web-2.1.2/modules/datax-admin/bin/env.properties
# environment variables
JAVA_HOME=/home/cws/program/jdk1.8
WEB_LOG_PATH=${BIN}/../logs
WEB_CONF_PATH=${BIN}/../conf
DATA_PATH=${BIN}/../data
SERVER_PORT=9527
#PID_FILE_PATH=${BIN}/dataxadmin.pid
# mail account
#MAIL_USERNAME=""
#MAIL_PASSWORD=""
#debug
#REMOTE_DEBUG_SWITCH=true
#REMOTE_DEBUG_PORT=7003
#PYTHON_PATH=/home/cws/program/datax/bin/datax.py
/home/cws/program/datax-web-2.1.2/modules/datax-admin/conf/bootstrap.properties
#Database
DB_HOST=192.168.0.88
DB_PORT=3306
DB_USERNAME=root
DB_PASSWORD=root
DB_DATABASE=dataxweb
datax-executor
/home/cws/program/datax-web-2.1.2/modules/datax-executor/bin/env.properties
# environment variables
JAVA_HOME=/home/cws/program/jdk1.8
SERVICE_LOG_PATH=${BIN}/../logs
SERVICE_CONF_PATH=${BIN}/../conf
DATA_PATH=${BIN}/../data
## datax json文件存放位置
JSON_PATH=${BIN}/../json
## executor_port
EXECUTOR_PORT=9999
## 保持和datax-admin端口一致
DATAX_ADMIN_PORT=
## PYTHON脚本执行位置
#PYTHON_PATH=/home/hadoop/install/datax/bin/datax.py
PYTHON_PATH=/home/cws/program/datax/bin/datax.py
## dataxweb 服务端口
SERVER_PORT=9504
#PID_FILE_PATH=${BIN}/service.pid
#debug 远程调试端口
#REMOTE_DEBUG_SWITCH=true
#REMOTE_DEBUG_PORT=7004
"writer": {
"name": "mysqlwriter",
"parameter": {
"writeMode": "replace",
"username": "yRjwDFuoPKlqya9h9H2Amg==",
"password": "yRjwDFuoPKlqya9h9H2Amg==",
"column": [
"`id`",
"`name`"
],
"connection": [
{
"table": [
"BSS_Business"
],
"jdbcUrl": "jdbc:mysql://192.168.0.88:3306/test?characterEncoding=utf-8&useSSL=false"
}
]
}
writeMode模式选择
总共三种:insert/replace/update
对于导入端是 mysql的话,
insert 就是单纯的插入。不判重。如果遇到主键冲突,就报错提示。
replace,原理类似 mysql 的 replace into 的原理,
update,原理类似 mysql 的on duplicate key update的原理。
"job": {
"setting": {
"speed": {
"channel": 3,
"byte": 1048576 ##报错请删除byte
},