安装Hive,有条件的同学可考虑用mysql作为元数据库安装(有一定难度,可以获得老师极度赞赏),安装完成后做简单SQL操作测试。
安装环境:
- hadoop1.2.1集群
安装MySQL:
- 配置本地yum源,
vi /etc/yum.repos.d/dvd.repo
Redhat6.6添加如下内容:
- 运行:yum -y install mysql-server命令安装完成。
- 验证安装命令:rpm -qa mysql-server
- 设置172.16.247.140为mySQL Server的IP地址。
- 修改主机名
修改当前会话中主机名Hostname mysqlserver
修改配置文件中的主机名Vi /etc/sysconfig/network
关闭防火墙:service iptables stop
- 启动mysqld,建立相应的mySQL账号并赋予足够权限。
用hive用户登录测试并创建hive数据库:
- 下载驱动包(mysql-connector-java-5.1.35.tar.gz):http://dev.mysql.com/downloads/connector/j/
- appledeMacBook-Pro:softwares apple$ tar -xvf mysql-connector-java-5.1.35.tar
- appledeMacBook-Pro:mysql-connector-java-5.1.35 apple$ scp mysql-connector-java-5.1.35-bin.jar hadoop@hadoop0:/home/hadoop/apache-hive-1.2.0-bin/lib/
- 启动metastore:hive --service metastore &
- 启动hiveserver:hive --service hiveserver &
(错误排查命令:hive-hiveconf hive.root.logger=DEBUG,console)
二、安装Hive
- 下载hive apache-hive-1.1.0-bin.tar.gz from http://hive.apache.org
- 安装scp -r ./apache-hive-1.2.0-bin hadoop@hadoop0:/home/hadoop/
- 配置vi /etc/profile
备注:最后要使用命令:source /etc/profile 使环境变量立即生效。
2.修改$HIVE_HOME/bin的hive-config.sh,增加以下三行
- 创建hive-env.sh、hive-site.xml文件
[root@hadoop0 conf]# cp hive-default.xml.template hive-site.xml
cp hive-default.xml.templatehive-site.xml添加并修改里面的配置参数
[root@hadoop0 conf]# cp hive-default.xml.template hive-default.xml
- 修改hive-env.sh文件
- 创建文件夹:
- 先启动hadoop,mysql,再启动hive
- 错误
[root@hadoop0 bin]# hive
Exception in thread "main" java.lang.NoClassDefFoundErrorrg/apache/hadoop/hive/ql/CommandNeedRetryException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.lang.ClassNotFoundExceptionrg.apache.hadoop.hive.ql.CommandNeedRetryException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 3 more
解决办法如下:
但是有一种解决办法,我也是用了这个办法启动的。
要修改master节点的hadoop的conf文件夹里的hadoop-env.sh文件,没有修改之前的是
export HADOOP_CLASSPATH=/home/hadoop/hadoop-1.2.1/myclass
修改之后为
exportHADOOP_CLASSPATH=.:$CLASSPATH:$HADOOP_CLASSPATH:$HADOOP_HOME/bin:/home/hadoop/hadoop-1.2.1/myclass
改成如下也可以:
exportHADOOP_CLASSPATH=$CLASSPATH:/home/hadoop/hadoop-1.2.1/myclass
- 解决${system:java.io.tmpdir}/${system:user.name}引起的错误:
[root@hadoop0 bin]# hive
15/05/30 08:55:39 WARN conf.HiveConf: HiveConf of namehive.metastore.local does not exist
Logging initialized using configuration injar:file:/home/hadoop/apache-hive-1.2.0-bin/lib/hive-common-1.2.0.jar!/hive-log4j.properties
Exception in thread "main"java.lang.RuntimeException: java.lang.IllegalArgumentException:java.net.URISyntaxException: Relative path in absolute URI:${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
atorg.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:519)
atorg.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
atorg.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethod)
atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
atsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Caused by: java.lang.IllegalArgumentException:java.net.URISyntaxException: Relative path in absolute URI:${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
at org.apache.hadoop.fs.Path.initialize(Path.java:148)
at org.apache.hadoop.fs.Path.<init>(Path.java:126)
atorg.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:560)
atorg.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:505)
... 7 more
Caused by: java.net.URISyntaxException: Relative path inabsolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
at java.net.URI.checkPath(URI.java:1804)
at java.net.URI.<init>(URI.java:752)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
... 10 more
解决办法如下:
1.查看hive-site.xml配置,会看到配置值含有"system:java.io.tmpdir"的配置项,添加属性:
- 新建文件夹/home/hadoop/apache-hive-1.2.0-bin/iotmp
chmod 733 iotmp
启动hive,成功!
- 测试,运行hive
hive
hive>create table a1(a string,b int);
hive>show tables
Hive端删除表a1:
服务器端刷新查看:
问题及其解决相关帖子:
http://f.dataguru.cn/thread-501728-1-1.html
转载: http://f.dataguru.cn/thread-525071-1-2.html