Oozie安装配置手册

安装 
准备： 
oozie-3.3.2,oozie-4.0.0在执行oozie-setup.sh时失败，所以选取3.3.2版本 
mysql 
hadoop-1.0.3 
apache-6 
mavn3（必须是3或者以上版本） 
ext-2.2.zip拷贝至oozie-3.3.2目录下 

创建 oozie 组和用户 
/usr/sbin/groupadd oozie 
/usr/sbin/useradd   oozie   -g  oozie 

环境变量： 
export JAVA_HOME=/usr/local/jrockit-jdk1.6.0_29 
export JRE_HOME=$JAVA_HOME/jre 
export HADOOP_HOME=/usr/local/hadoop-1.0.3 
export HIVE_HOME=/usr/local/hive-0.9.0 
export CATALINA_HOME=/home/qiulp/apache-tomcat-6.0.37 
export OOZIE_HOME=/home/qiulp/oozie-3.3.2 
export M2_HOME=/home/qiulp/apache-maven-3.1.0 
export MAVEN_HOME=/home/qiulp/apache-maven-3.1.0 
及相应的path 


编译： 
bin/mkdistro.sh -DskipTests 
跳过测试，不跳过时build failed 


编译好的目录： 
oozie-3.3.2_bak/distro/target 


拷贝出oozie-3.3.2-distro.tar.gz，原oozie-3.3.2就可以删除掉了，解压oozie-3.3.2-distro.tar.gz 


修改bin/addtowar.sh 
elif [ "${version}" = "0.20.200" ]; then 
成 
elif [ "${version}" = "1.0.3" ]; then 


创建libext目录 
将mysql-connector-java-5.0.3-bin.jar分别放路lib、libext和CATALINA_HOME/lib下 

执行：./oozie-setup.sh -hadoop 1.0.3 ${HADOOP_HOME} -extjs ../ext-2.2.zip 
INFO: Adding extension: /home/hadoop/oozie-3.3.2/libext/mysql-connector-java-5.0.3-bin.jar 
Injecting following Hadoop JARs 
/home/hadoop/hadoop-1.0.3/hadoop-core-1.0.3.jar 
/home/hadoop/hadoop-1.0.3/lib/jackson-core-asl-1.8.8.jar 
/home/hadoop/hadoop-1.0.3/lib/jackson-mapper-asl-1.8.8.jar 
/home/hadoop/hadoop-1.0.3/lib/commons-configuration-1.6.jar 
New Oozie WAR file with added 'Hadoop JARs, ExtJS library, JARs' at /home/hadoop/oozie-3.3.2/oozie-server/webapps/oozie.war 
INFO: Oozie is ready to be started 



mysql库创建(摘抄)： 
mysql> create database oozie; 
mysql> create user 'oozie' identifiedby 'oozie';  
//下面这条命令十分重要，‘%’代表可以从任意host访问，即使oozie用户具有远程访问权限。 
mysql>grant all privileges on  oozie.* to 'oozie'@'%' with grant option; 
mysql> grant all privileges on oozie.* to 'oozie'@'localhost' with grant option; 
//下面这条命令是这是oozie用户，从任意host访问的密码皆是'123456' 
mysql> update mysql.user set password=password('123456') where User="oozie" and Host="%"; 
mysql> update mysql.user set password=password('123456') where User="oozie" and Host="localhost"; 
mysql> flush privileges; 


修改conf/oozie-site.xml 
<property> 
        <name>oozie.service.JPAService.create.db.schema</name> 
        <value>true</value> 
        <description> 
            Creates Oozie DB. 


            If set to true, it creates the DB schema if it does not exist. If the DB schema exists is a NOP. 
            If set to false, it does not create the DB schema. If the DB schema does not exist it fails start up. 
        </description> 
    </property> 


    <property> 
        <name>oozie.service.JPAService.jdbc.driver</name> 
        <value>com.mysql.jdbc.Driver</value> 
        <description> 
            JDBC driver class. 
        </description> 
    </property> 


    <property> 
        <name>oozie.service.JPAService.jdbc.url</name> 
        <value>jdbc:mysql://10.200.187.77:3306/oozie</value> 
        <description> 
            JDBC URL. 
        </description> 
    </property> 


    <property> 
        <name>oozie.service.JPAService.jdbc.username</name> 
        <value>oozie</value> 
        <description> 
            DB user name. 
        </description> 
    </property> 


    <property> 
        <name>oozie.service.JPAService.jdbc.password</name> 
        <value>123456</value> 
        <description> 
            DB user password. 


            IMPORTANT: if password is emtpy leave a 1 space string, the service trims the value, 
                       if empty Configuration assumes it is NULL. 
        </description> 
    </property> 


执行: 
./ooziedb.sh create -sqlfile oozie.sql -run  
日志 
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m" 


Validate DB Connection 
DONE 
Check DB schema does not exist 
DONE 
Check OOZIE_SYS table does not exist 
DONE 
Create SQL schema 
DONE 
Create OOZIE_SYS table 
DONE 


Oozie DB has been created for Oozie version '3.3.2' 
同时形成了oozie.sql执行过得语句，此时mysql已经创建好了所需的基础表。 

最后修改hadoop的core-site.xml，增加配置，并重启 
<property> 
    <name>hadoop.proxyuser.[OOZIE_SERVER_USER].hosts</name> 
    <value>[OOZIE_SERVER_HOSTNAME]</value> 
  </property> 
  <property> 
    <name>hadoop.proxyuser.[OOZIE_SERVER_USER].groups</name> 
    <value>[USER_GROUPS_THAT_ALLOW_IMPERSONATION]</value> 
  </property> 
这个配置很是麻烦，这里解释一下： 
OOZIE_SERVER_USER是oozie server的user 
OOZIE_SERVER_HOSTNAME是oozie server的hostname 
USER_GROUPS_THAT_ALLOW_IMPERSONATION是client端提交任务的usergroup 


我安装的hadoop oozie都在同一台机器上，user非别是hadoop和oozie，client在hadoop用户组里，所以USER_GROUPS_THAT_ALLOW_IMPERSONATION是hadoop。 
如下： 
<property> 
   <name>hadoop.proxyuser.oozie.hosts</name> 
   <value>master</value> 
</property> 
<property> 
    <name>hadoop.proxyuser.oozie.groups</name> 
    <value>hadoop</value> 
</property> 
</configuration> 
这里我测试了多种情况，hadoop、oozieserver、oozieclient在不同的机器上机会报： 
Error: E0902 : E0902: Exception occured: [User: XXX is not allowed to impersonate XXX] 


我的理解：如上的hadoop core配置中，OOZIE_SERVER_USER是oozie server的user，因为配置在hadoop master机器上所以此用户必须是同一台机器的相同或者不同user上。 
同样USER_GROUPS_THAT_ALLOW_IMPERSONATION必须是同一台机器的相同或者不同usergroup里。 


不知道是否正确，但试下来同一台机器是可以的，机器不同就example跑不起来。 

​ 

运行 
./oozied.sh run 


打开url 
http://localhost:11000/oozie/ 



oozie client部署和测试： 
解压后的oozie-3.3.2内部含有oozie-client-3.3.2.tar.gz，oozie-examples.tar.gz 

分别解压。 
上传测试数据至hdfs，hadoop fs -put hadoop fs -put examples examples 
(至hdfs当前用户下，即.../usr/user/example,注意测试client的用户需要与数据上传的用户相同，或者上传时指定数据的绝对路径，不然测试job获取不到数据) 
examples/apps/map-reduce/job.properties修改hadoop的相关内容 
提示：examples/apps下有各种测试样例，一个目录一个样例，如hive sqoop pig 
运行测试样例： 
oozie-client-3.3.2/bin/oozie job -oozie http://10.200.187.77:11000/oozie -config examples/apps/map-reduce/job.properties -run