Java通过python命令执行DataX任务

最新推荐文章于 2024-10-10 08:39:27 发布

王绍桦

最新推荐文章于 2024-10-10 08:39:27 发布

阅读量4.4k

点赞数 2

分类专栏： java工具类文章标签： python java odps

本文链接：https://blog.csdn.net/rexueqingchun/article/details/78843535

版权

java工具类专栏收录该内容

29 篇文章 12 订阅

订阅专栏

1.安装datax
2.安装python并配置环境变量
3.把mysql2odps.json文件放在datax安装目录的job文件夹下
4.运行Test.java测试

mysql2odps.json文件：

{
    "job": {
        "content": [
            {
                "reader": {
                    "name": "mysqlreader",
                    "parameter": {
                        "username": "$username",
                        "password": "$password",
                        "where": "optime>='$startDate $startTime' and optime<'$endDate $endTime'",
                        "column": ["id","xm"],
                        "connection": [ { "table": [ "test" ], "jdbcUrl": [ "$jdbcUrl" ] } ] }
                },
                "writer": {
                    "name": "odpswriter",
                    "parameter": {
                        "accessId": "****************",
                        "accessKey": "******************************",
                        "column": ["id","xm"],
                        "odpsServer": "http://service.odps.xxx.com/api",
                        "partition": "bt=$bt,region=$region",
                        "project": "dst_project_name",
                        "table": "test" }
                }
            }
        ],
        "setting": {
            "speed": {
                "channel": 10
            }
        }
    }
}

测试程序（我的datax安装路径为F盘根目录）：

import java.io.BufferedReader;
import java.io.InputStreamReader;

public class Test {

    public static void main(String[] args) {
        try {
            System.out.println("start");
            String windowcmd = "cmd /c python F:/datax/bin/datax.py -p \" -Dusername=root -Dpassword=1234 -DjdbcUrl=jdbc:mysql://192.168.1.122:3306/center -Dwhere=left(optime,10)=CURDATE() -Dbt=20171214 -Dregion=beijing \" F:/datax/job/mysql2odps.json";
            System.out.println(windowcmd);
            Process pr = Runtime.getRuntime().exec(windowcmd);
            BufferedReader in = new BufferedReader(new InputStreamReader(pr.getInputStream()));
            String line;
            while ((line = in.readLine()) != null) {
                System.out.println(line);
            }
            in.close();
            pr.waitFor();
            System.out.println("end");
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

}