01_Hive配置和安装

写在前面的话：

要配置hive,则需要提前配置好虚拟机和hadoop的相关组件，例如linux,hdfs,java,yarn。因此之前我们先检查这些组件的安装情况，再配置hive。而配置hive主要以下几个步骤：

• 解压安装
• 配置文件修改
• Mysql安装（可以使用其他数据库）
• log4j的修改

Linux配置（只给出要检查的步骤）

• 检查Linux配置
  1、防火墙关闭
  2、/etc/hosts的IP映射
  3、/etc/hostname 主机名
  4、ntp时间服务器（我没有检查，因为时间都是一致的）
  5、网卡信息配置

组件检查

• 检查HDFS\检查YARN\检查MR（）

	hadoop.env.sh
	yarn-env.sh
	mapred-env.sh
	core-site.xml

core-site.xml：

<configuration>
		<property>
			<name>fs.defaultFS</name>
			<value>hdfs://hadoop-senior01.itguigu.com:8020</value>
		</property>

		<property>
			<name>hadoop.tmp.dir</name>
			<value>/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/data</value>
		</property>
	</configuration>

hdfs-site.xml：

<configuration>
		<!-- 指定数据冗余份数 -->
		<property>
			<name>dfs.replication</name>
			<value>3</value>
		</property>

		<!-- 关闭权限检查-->
		<property>
			<name>dfs.permissions.enable</name>
			<value>false</value>
		</property>

		<property>
			<name>dfs.namenode.secondary.http-address</name>
			<value>hadoop-senior03.itguigu.com:50090</value>
		</property>

		<property>
			<name>dfs.namenode.http-address</name>
			<value>hadoop-senior01.itguigu.com:50070</value>
		</property>

		<property>
			<name>dfs.webhdfs.enabled</name>
			<value>true</value>
		</property>
	</configuration>

yarn-site.xml

<configuration>
		<!-- Site specific YARN configuration properties -->
		<property>
			<name>yarn.nodemanager.aux-services</name>
			<value>mapreduce_shuffle</value>
		</property>

		<property>
			<name>yarn.resourcemanager.hostname</name>
			<value>hadoop-senior02.itguigu.com</value>
		</property>

		<property>
			<name>yarn.log-aggregation-enable</name>
			<value>true</value>
		</property>

		<property>
			<name>yarn.log-aggregation.retain-seconds</name>
			<value>86400</value>
		</property>

		<!-- 任务历史服务 -->
	<property> 
		<name>yarn.log.server.url</name> 
		<value>http://hadoop-senior01.itguigu.com:19888/jobhistory/logs/</value> 
	</property> 
	</configuration>

mapred-site.xml

<configuration>
		<property> 
		<name>mapreduce.framework.name</name> 
		<value>yarn</value> 
	</property>

	<property> 
		<name>mapreduce.jobhistory.adress</name> 
		<value>hadoop-senior01.itguigu.com:10020</value> 
	</property>

	<property> 
		<name>mapreduce.jobhistory.webapp.adress</name> 
		<value>hadoop-senior01.itguigu.com:19888</value> 
	</property>

	</configuration>

创建Maven项目

具体步骤可以百度window上项目，有几个核心点就是：
1.配置pom.xml

	<dependency>
	<groupId>org.apache.hadoop</groupId>
	<artifactId>hadoop-client</artifactId>
	<version>2.5.0</version>
	</dependency>

2.需要集群中Hive的配置hive-site.xml和hdfs-site.xml放到工程的resource目录下

接下来才是真正Hive的部署与安装

1、解压Hive到安装目录

$ tar -zxf /opt/softwares/hive-0.13.1-cdh5.3.6.tar.gz -C ./

2、重命名配置文件

	$ mv hive-default.xml.template hive-site.xml
	$ mv hive-env.sh.template hive-env.sh

3、hive-env.sh

	JAVA_HOME=/opt/modules/jdk1.8.0_121
	HADOOP_HOME=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/
	export HIVE_CONF_DIR=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/conf

4、安装Mysql

	$ su - root
	# yum -y install mysql mysql-server mysql-devel
	# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
	# rpm -ivh mysql-community-release-el7-5.noarch.rpm
	# yum -y install mysql-community-server
	尖叫提示：如果使用离线绿色版本（免安装版本）需要手动初始化Mysql数据库

5、配置Mysql

	** 开启Mysql服务
		# systemctl start mysqld.service
		/etc/init.d/mysqld start
	** 设置root用户密码
		# mysqladmin -uroot password '123456'
		或者mysqladmin -u dd-p password '123456' " 回车 
	** 登录root
		# mysql -u root -p 
		然后输入密码

	** 为用户以及其他机器节点授权
		mysql> grant all on *.* to root@'hadoop-senior01.itguigu.com' identified by '123456';
		grant all on *.* to root@'hadoop104' identified by '000000';
		grant all on *.* to root@'hadoop104' identified by '000000';

		grant：授权
		all：所有权限
		*.*：数据库名称.表名称
		root：操作mysql的用户
		@''：主机名
		密码：123456
	** 刷新配置
		flush privileges;

** hive-site.xml 修改属性

//不存在，就创建
<property>
	<name>javax.jdo.option.ConnectionURL</name>
	<value>jdbc:mysql://hadoop-senior01.itguigu.com:3306/metastore?createDatabaseIfNotExist=true</value>
	<description>JDBC connect string for a JDBC metastore</description>
</property>

<property>
	<name>javax.jdo.option.ConnectionDriverName</name>
	<value>com.mysql.jdbc.Driver</value>
	<description>Driver class name for a JDBC metastore</description>
</property>
<property>
   	  <name>hive.metastore.uris</name>
  	 <value>thrift://master:9083</value>
  	 <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
           </property>

<property>
	<name>javax.jdo.option.ConnectionUserName</name>
	<value>root</value>
	<description>username to use against metastore database</description>
</property>

<property>
	<name>javax.jdo.option.ConnectionPassword</name>
	<value>123456</value>
	<description>password to use against metastore database</description>
</property>

** hive-log4j.properties

	hive.log.dir=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/logs

** 拷贝数据库驱动包到Hive根目录下的lib文件夹

	$ cp -a mysql-connector-java-5.1.27-bin.jar /opt/modules/cdh/hive-0.13.1-cdh5.3.6/lib/

** 启动Hive

	$ bin/hive