Hive实践操作理论

前言

： 实践是检验真理的唯一需求

前提

     1.当前环境已安装Hadoop全分布系统
     2.本地安装的有MySQL数据库
     3.MySQL的JDBC安装包
     4.Hive安装包

操作

     1.解压安装包
       tar -zvxf /安装包的安装路径 -C /解压后的安装路径
     
     2.更改名字（将带版本号的hive 改成hive）
       mv /解压后hive所在路径 /更改名字后的路径
     
     3.修改归属用户
      chown -R hadoop:hadoop /hive的路径
     
     4.环境配置
       关闭防火墙
       systemctl stop firewalld
       设置防火墙服务开机不启动
       systemctl disable firewalld
      
     5.卸载Linux系统中自带的MariaDB
         查询已安装的 MariaDB
         rpm -qa | grep MariaDB
         卸载MariaDB
         rpm -e --nodeps 刚刚查询出来的软件包
     
     6.安装 MySql 数据库
        进入安装包所在路径
        解压三个安装包(依据你的安装包后缀使用该命令)
        mysql-community-common
        mysql-community-libs
        mysql-community-client
     
     7.安装mysql server安装包(依据你的安装包后缀使用该命令)
        mysql-community-server
     
     8.修改MySQL数据库配置
        vi /etc/my.cnf
        
        symbolic-links=0
        设置 innodb 为默认的存储引擎
        default-storage-engine=innodb
        设置每个表的数据单独保存，而不是统一保存在 innodb 系统表空间中，单独保存有方便管理和提升性能两方面优势。
 	    innodb_file_per_table
 	    设置支持中文编码字符集
	    collation-server=utf8_general_ci
	    设置用户登录到数据库之后，在执行第一次查询之前执行 SET NAME utf8 命令，将使用的字符编码设定为utf8
	    init-connect='SET NAMES utf8'
	    将MySQL服务器字符集设定为交8
	    character-set-server=utf8
	    
	 9.启动Mysql 数据库
	      systemctl start mysqld
	 
	 10.查询Mysql数据库状态
	      systemctl status mysqld
	      若显示的是active（running）
	      则运行的正常
          
          若mysql 的进程状态是failed，则表示 mysql 数库显示启动异常，此时需要排查 /etc/my.cnf
     
     11.查询MySql数据库默认密码
          cat /var/log/mysqld.log | grep password
     
     12.Mysql 数据库初始化
          初始化数据库
          mysql_secure_installation
          
          Enter password for user root: 
          # 输入/var/log/mysqld.log 文件中查询到的默认 root 用户登录密码
          
		  The 'validate_password' plugin is installed on the server.
		  The subsequent steps will run with the existing configuration of the plugin.
        
          Using existing password for root.
          Estimated strength of the password: 100 
          
          Change the password for root ? ((Press y|Y for Yes, any other 
      	  key for No) : y
          New password: # 输入新密码 Password123$
          Re-enter new password: # 再次输入新密码 	Password123$
          
          Estimated strength of the password: 100 
          
          Do you wish to continue with the password provided?(Press y|Y 
      	  for Yes, any other key for No) : y # 输入 y
      	  
          By default, a MySQL installation has an anonymous user,
          allowing anyone to log into MySQL without having to have
          a user account created for them. This is intended only for
          testing, and to make the installation go a bit smoother.
          You should remove them before moving into a production
          environment.
          
          Remove anonymous users? (Press y|Y for Yes, any other key for 
          No) : y # 输入 y
          Success.
          
          Normally, root should only be allowed to connect from
         'localhost'. This ensures that someone cannot guess at
          the root password from the network.
	      Disallow root login remotely? (Press y|Y for Yes, any other key 
          for No) : n # 输入 n
          ... skipping.
          
          By default, MySQL comes with a database 	named 'test' that
          anyone can access. This is also intended only for testing,
          and should be removed before moving into a production
          environment.
          
          Remove test database and access to it? (Press y|Y for Yes, any 
          other key for No) : y # 输入 y - Dropping test database...
          Success.
          
          -	 Removing privileges on test database...
          Success.
     
          Reloading the privilege tables will ensure that all changesmade 
          so far will take effect immediately.Reload privilege tables now? 
          (Press y|Y for Yes, any other key for No) : y # 输入 ySuccess.All done!
     
      13.添加root用户从本地和远程访问Mysql数据库表单的授权
          mysql -uroot -p
          exit；
    
      14.配置Hive
        设置HIve环境变量
         vi /etc/profile
          # set hive environment
          export HIVE_HOVE = /usr/local/src/hive
          export PATH=$PATH:$HIVE_HOME/bin
        使环境变量配置生效
        source /etc/profile
      
      15.修改Hive组件配置文件
           su - hadoop
           cp /hive的路径/conf/hive-default.xml.template /hive的路径/conf/hive-site.xml
      
      16. 修改 hive-site.xml 文件实现 Hive 连接 Mysql 数据库，并设定Hive临时文件存储路径
    	  vi /hive的路径/conf/hive-site.xml
   
       1）设置 MySQL 数据库连接。
			<name>javax.jdo.option.ConnectionURL</name>
			<value>jdbc:mysql://master:3306/hive?
  						createDatabaseIfNotExist=true&amp;useSSL=false</value>
		<description>JDBC connect string for a JDBC metastore</description>
	
			2）配置 MySQL 数据库 root 的密码。
			<property>
		 	 <name>javax.jdo.option.ConnectionPassword</name>
			 <value>Password123$</value>
			<description>password to use againsts database</description>
			</property>
	
			3）验证元数据存储版本一致性。若默认 false，则不用修改。
			<property>
			<name>hive.metastore.schema.verification</name>
			<value>false</value>
			<description>
		
			 Enforce metastore schema version consistency.
			True: Verify that version information stored in is compatible with one from 
			Hive jars. Also disable automatic
			False: Warn if the version information stored in metastore doesn't match 
			with one from in Hive jars.
			</description>
			</property>

	4）配置数据库驱动。
		   <property>
			<name>javax.jdo.option.ConnectionDriverName</name>
			<value>com.mysql.jdbc.Driver</value>
	    	<description>Driver class name for a JDBC metastore</description>
		   </property>

	5）配置数据库用户名 javax.jdo.option.ConnectionUserName 为 root。
	    	<property>
		     <name>javax.jdo.option.ConnectionUserName</name>
		     <value>root</value>
	    	 <description>Username to use against metastore database</description>
	       </property>

   6）将以下位置的 ${system:java.io.tmpdir}/${system:user.name} 替换为
	          “/usr/local/src/hive/tmp”目录及其子目录。

    	需要替换以下 4 处配置内容：
   		   <name>hive.querylog.location</name>
           <value>/usr/local/src/hive/tmp</value>
	       <description>Location of Hive run time structured log 
    	    file</description>
         <name>hive.exec.local.scratchdir</name>
         <value>/usr/local/src/hive/tmp</value>
     	 <name>hive.downloaded.resources.dir</name>
  	 	 <value>/usr/local/src/hive/tmp/resources</value>
  		 <name>hive.server2.logging.operation.log.location</name>
  		 <value>/usr/local/src/hive/tmp/operation_logs</value>
 
  7）在 Hive 安装目录中创建临时文件夹 tmp。
    	[hadoop@master ~]$ mkdir /usr/local/src/hive/tmp
   至此，Hive 组件安装和配置完成
     
  15.初始化 hive 元数据
     cp /mysql-connector-java路径 /hive的路径/lib/
  
  16.重新启动hadoop即可
    dxc lib
    stop-all.sh
    
    start-all.sh
    dxc
    schematool -initSchema -dbType mysql
    
  17.启动 hive
      dxc
      hive