准备:
hive安装包 http://mirrors.hust.edu.cn/apache/ 下载apache-hive-2.1.0-bin.tar.gz
mysql RPM压缩包 http://dev.mysql.com/downloads/mysql/5.5.html#downloads
基本步骤:
一. 卸载mysql
二. 安装mysql
三. 安装hive
一. 卸载mysql
#1.检查是否已经安装MySQL组件
[root@sst_search_a /]# rpm -qa | grep -i mysql
MySQL-devel-5.6.23-1.linux_glibc2.5
MySQL-client-5.6.23-1.linux_glibc2.5
MySQL-server-5.6.23-1.linux_glibc2.5
#2.卸载前关闭MySQL服务
[root@sst_search_a /]# service mysql status
MySQL running (25673)[ OK ]
[root@sst_search_a /]# service mysql stop
Shutting down MySQL..[ OK ]
[root@sst_search_a /]# service mysql status
MySQL is not running[FAILED]
#3.收集MySQL对应文件夹信息
[root@sst_search_a /]# find / -name mysql
/etc/rc.d/init.d/mysql
/etc/logrotate.d/mysql
/var/lock/subsys/mysql
/var/lib/mysql
/var/lib/mysql/mysql
/usr/include/mysql
/usr/include/mysql/mysql
/usr/bin/mysql
/usr/share/mysql
/usr/lib64/mysql
#4.卸载MySQL各类组件及文件夹
#删除组件
[root@sst_search_a /]# rpm -ev MySQL-devel-5.6.23-1.linux_glibc2.5
[root@sst_search_a /]# rpm -ev MySQL-server-5.6.23-1.linux_glibc2.5
[root@sst_search_a /]# rpm -ev MySQL-client-5.6.23-1.linux_glibc2.5
#删除文件夹
[root@DB-Server init.d]# find / -name mysql
/var/lib/mysql
/var/lib/mysql/mysql
/usr/lib64/mysql
[root@DB-Server init.d]# rm -rf /var/lib/mysql
[root@DB-Server init.d]# rm -rf /var/lib/mysql/mysql
[root@DB-Server init.d]# rm -rf /usr/lib64/mysql
#5.确认MySQL彻底删除
[root@sst_search_a /]# rpm -qa | grep -i mysql
二、 安装MySQL#当前为root账户 #1.安装server [root@sst_search_a hive]# rpm -ivh /opt/MySQL-server-5.5.15-1.linux2.6.x86_64.rpm Preparing... ########################################### [100%] 1:MySQL-server ########################################### [100%] ...... /usr/bin/mysqladmin -u root password 'new-password' /usr/bin/mysqladmin -u root -h test1 password 'new-password' ...... #查看mysql3306端口 [root@sst_search_a hive]# netstat -nat tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN #2.安装client [root@sst_search_a hive]# rpm -ivh /opt/ MySQL-client-5.5.15-1.linux2.6.x86_64.rpm warning: MySQL-client-5.1.7-0.i386.rpm: V3 DSA signature: NOKEY, key ID 5072e1f5 Preparing... ########################################### [100%] 1:MySQL-client ########################################### [100%] #3.修改密码 [root@sst_search_a hive]# usr/bin/mysqladmin -u root password 'new-password' #mysqladmin -u用户名 -p旧密码 password 新密码 [root@sst_search_a hive]# mysql -uroot -new-password mysql> insert into mysql.user(Host,User,Password) values("localhost","hive",password("hive")); mysql> create database hive; mysql> grant all on hive.* to hive@'%' identified by 'hive'; mysql> grant all on hive.* to hive@'localhost' identified by 'hive'; mysql> flush privileges; mysql> exit; [root@sst_search_a hive]# mysql -uhive -phive mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | hive | | test | +--------------------+ 3 rows in set (0.00 sec) mysql> exit;
三、安装Hive#切换至hadoop账户 [hadoop@sst_search_a ~]$ vi .bash_profile #HIVE_HOME export HIVE_HOME=/opt/hive export PATH=$PATH:$HIVE_HOME/bin [hadoop@sst_search_a ~]$ source .bash_profile #配置hive-env.sh [hadoop@sst_search_a conf]$ cp hive-env.sh.template hive-env.sh [hadoop@sst_search_a conf]$ vi hive-env.sh export HIVE_HOME=/opt/hive [hadoop@sst_search_a conf]$ source hive-env.sh #配置hive-site.xml [hadoop@sst_search_a conf]$ cp hive-default.xml.template hive-site.xml [hadoop@sst_search_a conf]$ vi hive-site.xml #文件比较大,使用'/javax.jdo.option.ConnectionURL'查找 <property> <name>javax.jdo.option.ConnectionURL </name> <value>jdbc:mysql://localhost:3306/hive </value> </property> <property> <name>javax.jdo.option.ConnectionDriverName </name> <value>com.mysql.jdbc.Driver </value> </property> <property> <name>javax.jdo.option.ConnectionPassword </name> <value>hive </value> </property> <property> <name>hive.hwi.listen.port </name> <value>9999 </value> <description>This is the port the Hive Web Interface will listen on </descript ion> </property> <property> <name>datanucleus.autoCreateSchema </name> <value>true</value> </property> <property> <name>datanucleus.fixedDatastore </name> <value>false</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hive</value> <description>Username to use against metastore database</description> </property> <property> <name>hive.exec.local.scratchdir</name> <value>/home/hdpsrc/hive/iotmp</value> <description>Local scratch space for Hive jobs</description> </property> <property> <name>hive.downloaded.resources.dir</name> <value>/home/hdpsrc/hive/iotmp</value> <description>Temporary local directory for added resources in the remote file system.</description> </property> <property> <name>hive.querylog.location</name> <value>/home/hdpsrc/hive/iotmp</value> <description>Location of Hive run time structured log file</description> </property> #拷贝mysql-connector-java-5.1.6-bin.jar 到hive 的lib下面 [hadoop@sst_search_a conf]$ mv /opt/Desktop/mysql-connector-java-5.1.6-bin.jar /opt/hive/lib/ #把jline-2.12.jar拷贝到hadoop相应的目录下,替代jline-0.9.94.jar,否则启动会报错 [hadoop@sst_search_a conf]$ cp /opt/hive/lib/jline-2.12.jar /opt/hadoop-2.6.0/share/hadoop/yarn/lib/ [hadoop@sst_search_a conf]$ mv /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar.bak / #创建hive临时文件夹 [hadoop@sst_search_a conf]$ mkdir /opt/hive/iotmp #启动hadoop后测试hive [hadoop@sst_search_a conf]$ hive hive> show databases; OK default Time taken: 0.907 seconds, Fetched: 1 row(s)
四、遇到的问题问题1: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/........../lib/slf4j-log4j12-1.5.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/........../slf4j-log4j12-1.5.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 解决办法: jar包重复,删掉了hive/lib文件夹下的对应jar包 问题2: Logging initialized using configuration in jar:file:/opt/tuniu/hive/apache-hive-2.1.0-bin/lib/hive-common-2.1.0.jar!/hive-log4j2.properties Async: true Exception in thread "main" java.lang.RuntimeException: java.io.IOException: Permission denied at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:557) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.io.IOException: Permission denied at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.createTempFile(File.java:2024) at org.apache.hadoop.hive.common.FileUtils.createTempFile(FileUtils.java:785) at org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:904) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:555) ... 9 more 解决办法: 注意到permission denied,一定是权限问题。猜测可能是hadoop用户对hive文件夹权限不够,暴力赋予了足够的权限 chmod 777 -R /opt/hive (注意这里其实问题变形很多,还是得读一下自己的错误日志) 问题3: which: no hbase in (/opt/tuniu/java/jdk1.8.0_101/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/tuniu/hadoop/hadoop-2.6.4/bin:/opt/tuniu/hadoop/hadoop-2.6.4/sbin:/root/bin:/opt/tuniu/hive/apache-hive-2.1.0-bin/bin) 还没搞明白和hbase有什么关系。。。不影响使用,暂时搁置。