Oozie安装

安装基础环境

Maven-3.9.0
Hadoop-2.7.1

Server安装

解压oozie-4.3.0.tar.gz

将oozie的安装包解压到/usr/local目录下:
/usr/local/oozie-4.3.0

修改pom.xml

<targetJavaVersion>1.8</targetJavaVersion>
<sourceJavaVersion>1.8</sourceJavaVersion>
<minJavaVersion>1.7</minJavaVersion>

<hadoop.version>2.7.1</hadoop.version>

mvn编译

/usr/local/oozie-4.3.0/bin/mkdistro.sh -DskipTests -Dhadoop.version=2.7.1
因为要下载很多东西,需要一点时间
最后会在/usr/local/oozie-4.3.0/distro/target目录生成编译好的oozie-4.3.0-distro.tar.gz压缩包。将其解压到/data目录下:
/data/oozie-4.3.0
并配置好环境变量:
export OOZIE_HOME=/data/oozie-4.3.0

添加相关jar包

oozie server需要用到一个js库,需要下载ext-2.2.zip这个文件放的libext文件夹里。否则就看不了oozie的web控制台。(打war包的时候会报:INFO: Oozie webconsole disabled, ExtJS library not specified)
mkdir libext
下载ext-2.2.zip放到libext目录下面

然后,把hadoop的一些jar把也放到这个libext文件夹内:
cp ${HADOOP_HOME}/share/hadoop/*.jar libext/
cp ${HADOOP_HOME}/share/hadoop/*/lib/*.jar libext/
注意注意:这里有个大坑,oozie server默认使用tomcat 6.0.41,而hadoop也有内置的server,如果按照上面两个命令把hadoop依赖的jar包都拷贝过去,有可能出现冲突,这两个server使用的servlet、jsp版本很可能不一样。
这里需要把这几个jar包删除,不要放到libext中
jsp-api-2.1.jar

oozie server还需要依赖个数据库,常用的是mysql,所以需要把mysql的驱动jar包也放的libext中。
mysql-connector-java-5.1.36.jar

core-site.xml

修改/data/oozie-4.3.0/conf/hadoop-conf/core-site.xml
注意:因为Namenode配置了HA,所以需要配置fs.defaultFS,并且把hdfs-site.xml拷贝到/data/oozie-4.3.0/conf/hadoop-conf目录下

<!– OOZIE –>
<property>
<name>fs.defaultFS</name>
<value>hdfs://appcluster</value>
</property>

<property>
<name>oozie.service.JPAService.create.db.schema</name>
<value>true</value>
</property>

<property>
<name>oozie.service.JPAService.jdbc.driver</name>
<value>com.mysql.jdbc.Driver</value>
</property>

<property>
<name>oozie.service.JPAService.jdbc.url</name>
<value>jdbc:mysql://10.25.23.160:3306/db_oozie?createDatabaseIfNotExist=true</value>
</property>

<property>
<name>oozie.service.JPAService.jdbc.username</name>
<value>root</value>
</property>

<property>
<name>oozie.service.JPAService.jdbc.password</name>
<value>root</value>
</property>

<property>
<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
<value>*=/usr/local/hadoop-2.7.1/etc/hadoop</value>
</property> 

修改/usr/local/hadoop-2.7.1/etc/hadoop/core-site.xml
修改该配置是为oozie添加伪装功能(impersonation),关于伪装功能,我这里不在细说,大家可以看本篇文章最后参考文章中的第三篇。添加伪装功能后,不需要重启hdfs与yarn,执行下面的命令即可:
hdfs dfsadmin -refreshSuperUserGroupsConfiguration
yarn rmadmin -refreshSuperUserGroupsConfiguration

<property>
   <name>hadoop.proxyuser.oozie.hosts</name>
   <value>*</value>
</property>

<property>
   <name>hadoop.proxyuser.oozie.groups</name>
   <value>*</value>
</property>

打war包

好,上面的步骤完成后,就可以生产server的war包了:
oozie-setup.sh prepare-war
打印日志:
setting CATALINA_OPTS=”$CATALINA_OPTS -Xmx1024m”

INFO: Adding extension: /data/oozie-4.3.0/libext/activation-1.1.jar
…这里省略很多jar包
INFO: Adding extension: /data/oozie-4.3.0/libext/commons-codec-1.4.jar
New Oozie WAR file with added ‘ExtJS library, JARs’ at /data/oozie-4.3.0/oozie-server/webapps/oozie.war
war包在/data/oozie-4.3.0/oozie-server/webapps/下

共享HDFS配置

在启动之前,我们还需要把执行下面的命令:
/data/oozie-4.3.0/bin/oozie-setup.sh sharelib create -fs hdfs://appcluster:8020
这把hadoop的一些jar包传到hdfs上,后面会用到。
成功后提示:the destination path for sharelib is: /user/root/share/lib/lib_20161214170752

启动与停止

/data/oozie-4.3.0/bin/oozied.sh start
/data/oozie-4.3.0/bin/oozied.sh stop
启动会显示:

setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data/oozie-4.3.0/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data/oozie-4.3.0/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data/oozie-4.3.0/libext/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
the destination path for sharelib is: /user/root/share/lib/lib_20161214170752
[root@SSSVL0315 hadoop-conf]# /data/oozie-4.3.0/bin/oozied.sh start

Setting OOZIE_HOME:          /data/oozie-4.3.0
Setting OOZIE_CONFIG:        /data/oozie-4.3.0/conf
Sourcing:                    /data/oozie-4.3.0/conf/oozie-env.sh
  setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
Setting OOZIE_CONFIG_FILE:   oozie-site.xml
Setting OOZIE_DATA:          /data/oozie-4.3.0/data
Setting OOZIE_LOG:           /data/oozie-4.3.0/logs
Setting OOZIE_LOG4J_FILE:    oozie-log4j.properties
Setting OOZIE_LOG4J_RELOAD:  10
Setting OOZIE_HTTP_HOSTNAME: SSSVL0315
Setting OOZIE_HTTP_PORT:     11000
Setting OOZIE_ADMIN_PORT:     11001
Setting OOZIE_HTTPS_PORT:     11443
Setting OOZIE_BASE_URL:      http://SSSVL0315:11000/oozie
Setting CATALINA_BASE:       /data/oozie-4.3.0/oozie-server
Setting OOZIE_HTTPS_KEYSTORE_FILE:     /root/.keystore
Setting OOZIE_HTTPS_KEYSTORE_PASS:     password
Setting OOZIE_INSTANCE_ID:       SSSVL0315
Setting CATALINA_OUT:        /data/oozie-4.3.0/logs/catalina.out
Setting CATALINA_PID:        /data/oozie-4.3.0/oozie-server/temp/oozie.pid

Using   CATALINA_OPTS:        -Xmx1024m -Dderby.stream.error.file=/data/oozie-4.3.0/logs/derby.log
Adding to CATALINA_OPTS:     -Doozie.home.dir=/data/oozie-4.3.0 -Doozie.config.dir=/data/oozie-4.3.0/conf -Doozie.log.dir=/data/oozie-4.3.0/logs -Doozie.data.dir=/data/oozie-4.3.0/data -Doozie.instance.id=SSSVL0315 -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10 -Doozie.http.hostname=SSSVL0315 -Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.https.port=11443 -Doozie.base.url=http://SSSVL0315:11000/oozie -Doozie.https.keystore.file=/root/.keystore -Doozie.https.keystore.pass=password -Djava.library.path=

Setting up oozie DB
  setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"

Validate DB Connection
DONE
DB schema does not exist
Check OOZIE_SYS table does not exist
DONE
Create SQL schema
DONE
Create OOZIE_SYS table
DONE

Oozie DB has been created for Oozie version '4.3.0'


The SQL commands have been written to: /tmp/ooziedb-6860768248946039879.sql

Using CATALINA_BASE:   /data/oozie-4.3.0/oozie-server
Using CATALINA_HOME:   /data/oozie-4.3.0/oozie-server
Using CATALINA_TMPDIR: /data/oozie-4.3.0/oozie-server/temp
Using JRE_HOME:        /usr/java/jdk1.8.0_111
Using CLASSPATH:       /data/oozie-4.3.0/oozie-server/bin/bootstrap.jar
Using CATALINA_PID:    /data/oozie-4.3.0/oozie-server/temp/oozie.pid

参 考

http://www.cnblogs.com/tovin/p/3885162.html
https://segmentfault.com/a/1190000002738484

  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 2
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值