Oozie 集成 Ssh

最新推荐文章于 2023-02-05 15:19:40 发布

sun_xo

最新推荐文章于 2023-02-05 15:19:40 发布

阅读量299

点赞数

文章标签： hadoop ssh oozie

本文链接：https://blog.csdn.net/sun_xo/article/details/125771590

版权

1) prepare ssh case
$ cd ~/work/oozie-5.2.1
$ tree oozie/apps/ssh

├── job.properties
├── mushishi.sh
└── workflow.xml

$ cat oozie/apps/ssh/mushishi.sh

#!/bin/sh

rm /tmp/mushishi_*.txt
echo "$1 $JAVA_HOME" > /tmp/mushishi_$$.txt
# env >> /tmp/mushishi_$$.txt
echo "Mushishi=$1"

$ cat oozie/apps/ssh/job.properties

nameNode=hdfs://localhost:9000
resourceManager=localhost:8032
queueName=default
oozieRoot=user/${user.name}/oozie

oozie.wf.application.path=${nameNode}/${oozieRoot}/apps/ssh

$ cat oozie/apps/ssh/workflow.xml

<workflow-app xmlns="uri:oozie:workflow:1.0" name="ssh-wf">
    <start to="ssh-node"/>
    <action name="ssh-node">
        <ssh xmlns="uri:oozie:ssh-action:0.1">
            <host>localhost</host>
            <command>~/work/oozie-5.2.1/oozie/apps/ssh/mushishi.sh</command>
            <args>"Ginko"</args>
            <capture-output/>
        </ssh>
        <ok to="check-output"/>
        <error to="fail"/>
    </action>
    <decision name="check-output">
        <switch>
            <case to="end">
                ${wf:actionData('ssh-node')['Mushishi'] eq 'Ginko'}
            </case>
            <default to="fail-output"/>
        </switch>
    </decision>
    <kill name="fail-output">
        <message>Incorrect output, expected [Ginko] but was [${wf:actionData('ssh-node')['Mushishi']}]</message>
    </kill>
    <kill name="fail">
        <message>SSH action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

2) test the script
## run locally
$ ~/work/oozie-5.2.1/oozie/apps/ssh/mushishi.sh anonymous
Mushishi=anonymous
$ cat /tmp/mushishi_*.txt
anonymous /Library/Java/JavaVirtualMachines/jdk1.8.0_321.jdk/Contents/Home
## run remotely
$ ssh localhost ~/work/oozie-5.2.1/oozie/apps/ssh/mushishi.sh anonymous
Mushishi=anonymous
$ cat /tmp/mushishi_*.txt
anonymous
## Note: environment variables are missing on remote

3) run by oozie
## Note: it doesn't need to upload to hdfs
## submit and run
$ bin/oozie job -config oozie/apps/ssh/job.properties -run
job: 0000000-220713195559163-oozie-sun_-W
## check result
$ bin/oozie job -info 0000000-220713195559163-oozie-sun_-W

Job ID : 0000000-220713195559163-oozie-sun_-W
------------------------------------------------------------------------------------------------------------------------------------
Workflow Name : ssh-wf
App Path      : hdfs://localhost:9000/user/sun_xo/oozie/apps/ssh
Status        : SUCCEEDED
Run           : 0
User          : sun_xo
Group         : -
Created       : 2022-07-13 12:37 GMT
Started       : 2022-07-13 12:37 GMT
Last Modified : 2022-07-13 12:37 GMT
Ended         : 2022-07-13 12:37 GMT
CoordAction ID: -

Actions
------------------------------------------------------------------------------------------------------------------------------------
ID                                                                            Status    Ext ID                 Ext Status Err Code  
------------------------------------------------------------------------------------------------------------------------------------
0000000-220713195559163-oozie-sun_-W@:start:                                  OK        -                      OK         -         
------------------------------------------------------------------------------------------------------------------------------------
0000000-220713195559163-oozie-sun_-W@ssh-node                                 OK        59990                  OK         -         
------------------------------------------------------------------------------------------------------------------------------------
0000000-220713195559163-oozie-sun_-W@check-output                             OK        -                      end        -         
------------------------------------------------------------------------------------------------------------------------------------
0000000-220713195559163-oozie-sun_-W@end                                      OK        -                      OK         -         
------------------------------------------------------------------------------------------------------------------------------------

$ cat /tmp/mushishi_*.txt
Ginko