dataxweb执行脚本核心代码
ScriptJobHandler
@Override
public ReturnT<String> execute(TriggerParam tgParam) throws Exception {
if (!glueType.isScript()) {
return new ReturnT<>(IJobHandler.FAIL.getCode(), "glueType[" + glueType + "] invalid.");
}
// cmd
String cmd = glueType.getCmd();
// make script file
String scriptFileName = JobFileAppender.getGlueSrcPath()
.concat(File.separator)
.concat(String.valueOf(jobId))
.concat("_")
.concat(String.valueOf(glueUpdatetime))
.concat(glueType.getSuffix());
File scriptFile = new File(scriptFileName);
if (!scriptFile.exists()) {
ScriptUtil.markScriptFile(scriptFileName, gluesource);
}
// log file
String logFileName = JobFileAppender.contextHolder.get();
// script params:0=param、1=分片序号、2=分片总数
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
String[] scriptParams = new String[3];
scriptParams[0] = tgParam.getExecutorParams();
scriptParams[1] = String.valueOf(shardingVO.getIndex());
scriptParams[2] = String.valueOf(shardingVO.getTotal());
// invoke
JobLogger.log("----------- script file:"+ scriptFileName +" -----------");
int exitValue = ScriptUtil.execToFile(cmd, scriptFileName, logFileName,tgParam.getLogId(),tgParam.getLogDateTime(), scriptParams);
if (exitValue == 0) {
return IJobHandler.SUCCESS;
} else {
return new ReturnT<>(IJobHandler.FAIL.getCode(), "script exit value(" + exitValue + ") is failed");
}
}
关键位置
GlueTypeEnum
BEAN("BEAN", false, null, null),
GLUE_GROOVY("GLUE(Java)", false, null, null),
GLUE_SHELL("GLUE(Shell)", true, "bash", ".sh"),
GLUE_PYTHON("GLUE(Python)", true, "python3", ".py"),
GLUE_PHP("GLUE(PHP)", true, "php", ".php"),
GLUE_NODEJS("GLUE(Nodejs)", true, "node", ".js"),
GLUE_POWERSHELL("GLUE(PowerShell)", true, "powershell", ".ps1");
源码是: GLUE_PYTHON(“GLUE(Python)”, true, “python”, “.py”)
修改为: GLUE_PYTHON(“GLUE(Python)”, true, “python3”, “.py”)
因为操作系统默认也会使用python,所以不可以修改python的链接指向。重新打包替换项目中的Jar包,重启即可。
路径:datax-web-2.1.2/modules/datax-executor/lib
Jar包名:datax-core-2.1.2.jar
编写脚本测试
#!/opt/python-3.6.8/bin/python3
# -*- coding: utf-8 -*-
import platform
print(platform.python_version())
结果图