DataX(一)：编译运行

一、下载代码datax

二、编译

本人开发环境Macbook M1，一切顺利～

三、运行datax

1.拷贝plugin，这里先使用streamReader和streamWriter测试，plugin在对应插件target目录下，拷贝过来即可。

 2.测试生成job文件

注意datax.py目录在core目录下，这里以stream进行测试

/opt/homebrew/bin/python3 datax.py -r streamreader -w streamwriter

{
    "job": {
        "content": [
            {
                "reader": {
                    "name": "streamreader",
                    "parameter": {
                        "column": [],
                        "sliceRecordCount": ""
                    }
                },
                "writer": {
                    "name": "streamwriter",
                    "parameter": {
                        "encoding": "",
                        "print": true
                    }
                }
            }
        ],
        "setting": {
            "speed": {
                "channel": ""
            }
        }
    }
}

3.运行

配置文件core/target/datax/bin/stream.json文件

{
  "job": {
    "content": [{
      "reader": {
        "name": "streamreader",
        "parameter": {
          "sliceRecordCount": 10,
          "column": [{
            "type": "long",
            "value": "10"
          },
            {
              "type": "string",
              "value": "hello，你好，世界-DataX"
            }
          ]
        }
      },
      "writer": {
        "name": "streamwriter",
        "parameter": {
          "encoding": "UTF-8",
          "print": true
        }
      }
    }],
    "setting": {
      "speed": {
        "channel": 5
      }
    }
  }
}

 运行：/opt/homebrew/bin/python3 datax.py stream.json

结果示例

四、本地debug

4.1 打印启动命令

4.2 根据启动命令配置IDEA参数

程序参数

-mode standalone -jobid -1 -job /Users/shaobing/Desktop/learn/openSource/DataX/core/target/datax/bin/stream.json

JVM参数

-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/Users/shaobing/Desktop/learn/openSource/DataX/core/target/datax/log
-Ddatax.home=/Users/shaobing/Desktop/learn/openSource/DataX/core/target/datax
-Dlogback.configurationFile=/Users/shaobing/Desktop/learn/openSource/DataX/core/target/datax/conf/logback.xml

接下来可以愉快的进行Debug啦