Impala 安装笔记2一hive和mysql安装

l   安装hive，hive-metastore hive-server

$ sudo yum install hive hive-metastore hive-server

l   安装mysql-server

$ sudo yum install mysql-server

l   启动mysql守护进程

$ sudo service mysqld start

l   下载mysql java驱动并复制到hive的lib中

$ cd ~
$ wget 'http://cdn.mysql.com/Downloads/Connector-J/mysql-connector-java-5.1.25.tar.gz'
$ tar xzf mysql-connector-java-5.1.25.tar.gz
$ sudo cp mysql-connector-java-5.1.25/mysql-connector-java-5.1.25-bin.jar /usr/lib/hive/lib/

l   Mysql安全配置启动

$ sudo /usr/bin/mysql_secure_installation

[...]
Enter current password for root (enter for none):
OK, successfully used password, moving on...
[...]
Set root password? [Y/n] y
New password:hadoophive
Re-enter new password:hadoophive
Remove anonymous users? [Y/n] Y
[...]
Disallow root login remotely? [Y/n] N
[...]
Remove test database and access to it [Y/n] Y
[...]
Reload privilege tables now? [Y/n] Y
All done!

 mysql作为hive的元数据存储数据库

在mysql中建立相应的元数据数据库，执行相应的sql脚本，并且建立供hive连接mysql的用户，赋予其权限。

$ mysql -u root –p chengyeliang
mysql> CREATE DATABASE metastore;
mysql> USE metastore;
mysql> SOURCE /usr/lib/hive/scripts/metastore/upgrade/mysql/hive-schema-0.10.0.mysql.sql;

mysql> CREATE USER 'hive'@'%' IDENTIFIED BY 'chengyeliang';
mysql> CREATE USER 'hive'@'localhost' IDENTIFIED BY 'chengyeliang';
mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'%';
mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'localhost';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON metastore.* TO 'hive'@'%';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON metastore.* TO 'hive'@'localhost';
mysql> FLUSH PRIVILEGES;
mysql> quit;

修改hive的配置文件

$ sudo vim /etc/hive/conf/hive-site.xml

 1 <?xml version="1.0"?>
 2 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
 3 <configuration>
 4   <property>
 5     <name>javax.jdo.option.ConnectionURL</name>
 6     <value>jdbc:mysql://10.127.130.199:3306/metastore?useUnicode=true&amp;characterEn
 7 coding=UTF-8</value>
 8     <description>JDBC connect string for a JDBC metastore</description>
 9   </property>
10 
11   <property>
12     <name>javax.jdo.option.ConnectionDriverName</name>
13     <value>com.mysql.jdbc.Driver</value>
14     <description>Driver class name for a JDBC metastore</description>
15   </property>
16 
17   <property>
18     <name>javax.jdo.option.ConnectionUserName</name>
19     <value>hive</value>
20     <description>username to use against metastore database</description>
21   </property>
22 <property>
23     <name>javax.jdo.option.ConnectionPassword</name>
24     <value>chengyeliang</value>
25     <description>password to use against metastore database</description>
26   </property>
27 
28   <property>
29    <name>localhost.job.tracker</name>
30    <value>localhost:8031</value>
31   </property>
32 
33   <property>
34    <name>mapreduce.framework.name</name>
35    <value>yarn</value>
36   </property>
37   <property>
38           <name>hive.metastore.uris</name>
39           <value>thrift://localhost:9083</value>
40           </property>
41           <property>
42           <name>hive.metastore.client.socket.timeout</name>
43           <value>3600</value>
44           <description>MetaStore Client socket timeout in seconds</description>
45           </property>
46 </configuration>

启动hive-metastore进程

$ sudo service hive-metastore start

或者后台启动服务：nohup hive --service metastore&

创建Hive所需的HDFS目录

$ sudo -u hdfs hadoop fs -mkdir /user/hive
$ sudo -u hdfs hadoop fs -chown hive /user/hive
$ sudo -u hdfs hadoop fs -mkdir /tmp
$ sudo -u hdfs hadoop fs -chmod 777 /tmp
$ sudo -u hdfs hadoop fs -chmod o+t /tmp
$ sudo -u hdfs hadoop fs -mkdir /data
$ sudo -u hdfs hadoop fs -chown hdfs /data
$ sudo -u hdfs hadoop fs -chmod 777 /data
$ sudo -u hdfs hadoop fs -chmod o+t /data

$ sudo chown -R hive:hive /var/lib/hive

准备10万行记录的txt文件作为待查询的数据。

sudo vim /tmp/kv1.txt

建立一张新表

hive> CREATE TABLE IF NOT EXISTS chengyeliang ( foo INT,bar STRING ) ROW FORMAT DELIMITED FIELDS TERMINATED BY "\t" LINES TERMINATED BY "\n";

Time taken: 0.415 seconds

导入数据到table—chengyeliang

hive> LOAD DATA LOCAL INPATH '/tmp/kv1.txt' OVERWRITE INTO TABLE chengyeliang;

Time taken: 1.681 seconds

上面的路径要换成自己文本文件在hdfs上的绝对路径

如果上述操作ok，则安装成功。

转载于:https://www.cnblogs.com/chengyeliang/p/3548531.html