向导
安装
1. 上传tar包,解压
tar -zxvf apache-hive-2.3.6-bin.tar.gz -C /opt/module/
2. 安装hadoop
3. 安装mysql
MySQL的安装(YUM安装)
MySQL的安装(tar.gz文件安装)
MySQL的安装(RPM文件安装)
4. 配置hive-site.xml
mv apache-hive-2.3.6-bin hive
cd /opt/module/hive/conf
vi hive-site.xml
加入以下配置,修改mysql链接
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://hadoop102:3306/metastore?createDatabaseIfNotExist=true&useSSL=false</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>000000</value>
<description>password to use against metastore database</description>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
<property>
<name>hive.cli.print.header</name>
<value>true</value>
</property>
<property>
<name>hive.cli.print.current.db</name>
<value>true</value>
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
<property>
<name>datanucleus.schema.autoCreateAll</name>
<value>true</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://hadoop102:9083</value>
</property>
</configuration>
5. 拷贝mysql驱动
cp mysql-connector-java-5.1.27-bin.jar /opt/module/hive/lib/
7. 启动hdfs,yarn
start-dfs.sh
start-yarn.sh
8. 修改hdfs目录权限
hdfs dfs -mkdir /tmp
hdfs dfs -mkdir -p /user/hive/warehouse
hdfs dfs -chmod g+w /tmp
hdfs dfs -chmod g+w /user/hive/warehouse
9. 初始化DB
bin/schematool -initSchema -dbType mysql
9. 启动hive metastore,hiveserver2,hive
#注意:hive2.x版本需要启动两个服务metastore和hiveserver2,否则会报错
#Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
nohup bin/hive --service metastore > metastore.log 2>&1 &
nohup bin/hive --service hiveserver2 > hiveserver2.log 2>&1 &
#启动客户端
bin/hive
##或者使用beeline连接,-u 指定jdbc连接, -n指定用户名,防止权限相关
bin/beeline -u 'jdbc:hive2://localhost:10000' -n bigdata
问题
1. User: hadoop is not allowed to impersonate hadoop
使用beeline登录时,报如上错误,是hadoop报错的,主要原因是hadoop引入了一个安全伪装机制。解决是需要在hadoop的etc/hadoop/目录下,修改core-site.xml,加入如下配置,重启即可。
其中的xxx是你的用户名,即你的hive、hadoop是用哪个用户安装、启动的,就写谁就行。
<property>
<name>hadoop.proxyuser.xxx.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.xxx.groups</name>
<value>*</value>
</property>
2. Column length too big for column ‘PARAM_VALUE’ (max = 21845); use BLOB or TEXT instead
原因是编码问题,重新把hive与mysql的关联的数据库编码改成latin1就行了
这是因为mysql是使用utfmb4编码的,导致该字段在编码的时候内容过长(gbk使用双字节,utf使用三字节,)
show variables like '%char%'
//修改
alter database hive_meta character set latin1;