设置oozie工作流调度
workflow
Coordinator
bundle
workflow.xml
job.properties内容:
nameNode=hdfs://hadoop01:9000 --客户端连接hdfs集群
jobTracker=hadoop01:8032 --客户端连接yarn集群
queueName=default --调度队列
filePath=/gp1819/oozie --oozie的根目录
oozie.use.system.libpath=true –
oozie.libpath=${nameNode}/gp1819/oozielib --第三方依赖路径
oozie.wf.application.path={nameNode}nameNode{filePath}/sqoop/ --工作流应用程序的目录
Coordinator.xml
hdfs dfs -mkdir /gp1919
hdfs dfs -mkdir -p /gp1919/oozie /gp1919/oozielib
hdfs dfs -put $HIVE_HOME/lib/mysql-connector-java-5.1.32.jar /gp1919/oozielib/
1、创建sqoop的job
vi gp1919_sqoop_desc.sh
2、检查oozie任务的配置是否正确
启动oozie服务
oozied.sh start
启动历史服务器
mr-jobhistory-daemon.sh start historyserver
检查配置
oozie validate sqoop/workflow.xml
报错:
java.lang.IllegalArgumentException: Oozie URL is not available neither in command option or in the environment
解决:
export OOZIE_URL=http://hadoop01:11000/oozie
提交oozie的任务:
oozie job --oozie http://hadoop01:11000/oozie -config sqoop/job.properties -run
2、执行sqoop的job
oozie job --oozie http://hadoop01:11000/oozie -config sqoop-exec/job.properties -run
3、工作流调度
oozie job --oozie http://hadoop01:11000/oozie -config coor/job.properties -run