Hive安装 (hadoop2.6.0 hive 1.2.1)
三种模式
内嵌模式:元数据保持在内嵌的derby模式,只允许一个会话连接
本地独立模式:在本地安装Mysql,吧元数据放到mySql内
远程模式:元数据放置在远程的Mysql数据库
1、内嵌模式:
(1)、修改/home/lin/hadoop/apache-hive-1.2.1-bin/bin/hive-config.sh
添加java环境变量:
export JAVA_HOME=/usr/soft/jdk1.7.0_79
export HADOOP_HOME=/home/lin/hadoop/hadoop-2.6.1export HIVE_HOME=/home/lin/hadoop/apache-hive-1.2.1-bin
(2)、编辑hive-site.xml (/home/lin/hadoop/apache-hive-1.2.1-bin/conf/hive-default.xml.template复制为hive-site.xml)
修改以下参数:
hive.metastore.warehouse.dir hdfs 上的指定目录
hive.exec.scratchdir hdfs上的临时文件目录
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
</property>
<property>
<name>hive.exec.scratchdir</name>
<value>/tmp/hive</value></property>
<property>
<name>hive.server2.logging.operation.log.location</name>
<value>/tmp/hive/operation_logs</value><description>Top level directory where operation logs are stored if logging functionality is enabled</description>
</property>
<property>
<name>hive.exec.local.scratchdir</name>
<value>/tmp/hive</value><description>Local scratch space for Hive jobs</description>
</property>
<property>
<name>hive.downloaded.resources.dir</name>
<value>/tmp/hive/resources</value><description>Temporary local directory for added resources in the remote file system.</description>
</property>
(3)、配置hive的log4j: conf下 cp hive-log4j.properties.template hive-log4j.properties
#log4j.appender.EventCounter=org.apache.hadoop.hive.shims.HiveEventCounter
log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
否则会有警告:
WARN conf.HiveConf: HiveConf of name hive.metastore.local does not exist
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
(4)、启动hive bin/hive
启动时候遇到错误:
[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
at jline.TerminalFactory.create(TerminalFactory.java:101)
at jline.TerminalFactory.get(TerminalFactory.java:158)
at jline.console.ConsoleReader.<init>(ConsoleReader.java:229)
at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
发现是
jline
有问题,最后找到,原来hadoop里面的是jline0.0.94 hive里面的是jline2.12
把/home/lin/hadoop/apache-hive-1.2.1-bin/lib 里面的jline2.12替换了hadoop 中
/home/lin/hadoop/hadoop-2.6.1/share/hadoop/yarn/lib的jline0.0.94 就可以成功了!
2、独立模式
在mysql中为hive建立账号,并授予足够的权限,例如hive账号,授予all privileges
用上述账号登陆mysql,创建数据库,比如名字叫hive,用于存放hive的元数据
在本地安装MySQL客户端
配置hive-site.xml,指出使用本地mysql数据库,以及连接协议,账号,口令等
把mysql-connector-java.jar 复制到 hive的lib目录下
启动hive能进入shell表示成功
(1)、安装mysql并启动服务
联网安装:
sudo apt-get install cmake
sudo apt-get install libncurses5-dev
1. sudo apt-get install mysql-server2. sudo apt-get isntall mysql-client3. sudo apt-get install libmysqlclient-dev查看是否安装成功:
sudo netstat -tap | grep mysql
通过上述命令检查之后,如果看到有mysql 的socket处于 listen 状态则表示安装成功。
登陆mysql数据库
mysql -u root -p
创建数据库:
1) 建库:create database hivedb;
2) 建用户,赋权:grant all privileges on *.* to hivedb@"%" identified by "hivedb" with grant option;3) 删除数据库:drop database hivedb;
查看mysql版本
mysqladmin -u root -p version
查看数据库:
show databases;
查看当前字符集:
show variables like 'character%';
停止mysql:
sudo service mysql stop
启动mysql:
sudo service mysql start
**************************************************************启动mysql: 使用命令试试是否能连上mysql
telnet 192.168.0.166 3306
若拒绝连接 编辑
sudo vim /etc/mysql/my.cnf
注释掉
#bind-address = 127.0.0.1即可**************************************************************离线安装:
下载地址:http://dev.mysql.com/downloads/mysql/ 选择相关版本。
离线安装自行百度!
(2)、在内嵌模式下继续配置hive
加上驱动和数据库地址
<property>
<name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value><description>Driver class name for a JDBC metastore</description></property><property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://192.168.0.166:3306/hive?createDatabaseIfNotExist=true</value><description>JDBC connect string for a JDBC metastore</description></property><property><name>javax.jdo.option.ConnectionUserName</name><value>hive</value><description>username to use against metastore database</description></property><property><name>javax.jdo.option.ConnectionPassword</name><value>hive</value><description>password to use against metastore database</description></property>
(2)、拷贝数据驱动jar
拷贝mysql-connector-java-5.1.24-bin 到 /home/lin/hadoop/apache-hive-1.2.1-bin/lib
(4)、进入hive
若是报错:
NestedThrowables:
java.sql.SQLException: Unable to open a test connection to the given database. J DBC url = jdbc:mysql://192.168.0.166:3306/hivedb?createDatabaseIfNotExist=true, username = hivedb. Terminating connection pool (set lazyInit to true if you expe ct to start your database after your app). Original Exception: ------
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link fai lure
The last packet sent successfully to the server was 0 milliseconds ago. The driv er has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct orAccessorImpl.java:57)
解决方法:
先使用命令试试是否能连上mysql
telnet 192.168.0.166 3306
若拒绝连接 编辑
sudo vim /etc/mysql/my.cnf
注释掉
#bind-address = 127.0.0.1重启mysql即可即可
错误:
ERROR 1045 (28000): Access denied for user 'hive'@'localhost' (using password: YES)
解决办法:
删除数据库中的空白账户(匿名账户),依次执行以下两条指命,这一步是解决这个问题的核心步骤
DELETE FROM `mysql`.`user` WHERE `user`='';
FLUSH PRIVILEGES;
3、远程模式
在独立模式的基础上
添加:
<property>
<name>hive.metastore.local</name>
<value>true</value>
<description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
</property>