● 文件准备
- hive-0.12.0.tar.gz https://pan.baidu.com/s/1nxnAd0P
- MySQL-client-5.1.73-1.glibc23.x86_64.rpm https://pan.baidu.com/s/1eUaXhC2
- MySQL-server-5.1.73-1.glibc23.x86_64.rpm https://pan.baidu.com/s/1ghcWWIB
● 部署MySQL
# 查找并删除本地MySQL
rpm -qa | grep mysql
rpm -e mysql-libs-5.1.66-2.el6_3.i686 --nodeps
# 安装指定版本MySQL
rpm -ivh MySQL-server-5.1.73-1.glibc23.i386.rpm
rpm -ivh MySQL-client-5.1.73-1.glibc23.i386.rpm
# 修改mysql的密码(直接输入以下命令执行)
/usr/bin/mysql_secure_installation
(注意:设置root密码 并选择删除匿名用户,允许用户远程连接)
# 登陆mysql
mysql -u root -p
完成基本的MySQL安装。
接着给Spark SQL添加账号并开通账号权限。默认使用的DB名是hiveMetadata,假设账号和密码都是spark,授权SQL可以这样写:
mysql> grant all on hiveMetastore.* to spark@'localhost' identified by 'spark';
mysql> flush privileges;
● 准备配置文件conf/hive-site.xml
接下来,我们准备启动JDBC/ODBC Server,在启动之前,需要准备以下配置文件。
如果是与现有的Hive一起工作,直接实用Hive的配置文件conf/hive-site.xml即可,或者新建一个,在conf目录下准备一个名为hive-site.xml的配置文件,其内容如下:
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hiveMetastore?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>spark</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>spark</value>
<description>password to use against metastore database</description>
</property>
<property>
<name>hive.hwi.war.file</name>
<value>lib/hive-hwi-0.12.0.war</value>
<description>This sets the path to the HWI war file, relative to ${HIVE_HOME}. </description>
</property>
<property>
<name>hive.hwi.listen.host</name>
<value>0.0.0.0</value>
<description>This is the host address the Hive Web Interface will listen on</description>
</property>
<property>
<name>hive.hwi.listen.port</name>
<value>9999</value>
<description>This is the port the Hive Web Interface will listen on</description>
</property>
</configuration>
● 启动JDBC/ODBC Server
现在可以启动JDBC/ODBC Server了,其命令是:
./sbin/start-thriftserver.sh
● 使用beeline交互式工具
JDBC/ODBC Server启动之后,我们可以用beeline来测试启动是否正常:
./bin/beeline
下面的命令可以连接到JDBC/ODBC Server:
beeline> !connect jdbc:hive2://localhost:10000
也可以在启动beeline时,直接指定JDBC Server:
./bin/beeline -u 'jdbc:hive2://localhost:10000'
可以修改系统编码,防止SQL查询结果中文显示乱码。
LANG=zh_CN.UTF-8; ./bin/beeline -u 'jdbc:hive2://localhost:10000'
● 运行Spark SQL命令行界面
./bin/spark-sql