Hive单用户模式搭建

单用户模式是通过网络连接到一个数据库中，是最经常使用到的模式。

使用hive的前提：

（1）启动hadoop集群

（2）启动mysql服务

节点规划：

 

hadoop01为mysql存放元数据

hadoop02搭建hive单用户模式

 

搭建步骤：

 

 

1.上传好tar包，后解压：

tar -zxvf apache-hive-1.2.1-bin.tar.gz

把解压好的文件移动到/opt/software目录

 mv apache-hive-1.2.1-bin hive-1.2.1

 mv hive-1.2.1/ /opt/software/

 

 2. 配置环境变量

 

记得顺便重启配置文件：

 . /etc/profile

 3. 配置文件hive-site.xml

 cp hive-default.xml.template hive-site.xml

 这个命令可以把光标所在行以下除了倒数第一行都删了。

<configuration>  

<property>  

  <name>hive.metastore.warehouse.dir</name>  

  <value>/user/hive_remote/warehouse</value>  

</property>  

 <property>  

  <name>hive.metastore.local</name>  

  <value>true</value>  

</property>  

<property>  

  <name>javax.jdo.option.ConnectionURL</name>  

  <value>jdbc:mysql://hadoop01/hive_remote?createDatabaseIfNotExist=true</value>  

</property>  

<property>  

  <name>javax.jdo.option.ConnectionDriverName</name>  

  <value>com.mysql.jdbc.Driver</value>  

</property>  

<property>  

  <name>javax.jdo.option.ConnectionUserName</name>  

  <value>root</value>  

</property>  

<property>  

  <name>javax.jdo.option.ConnectionPassword</name>  

  <value>123</value>  

</property>  

</configuration>  

 

4. 安装mysql驱动包

 

把jar包移动到hive的jar包目录

mv mysql-connector-java-5.1.32-bin.jar /opt/software/hive-1.2.1/lib/

 5. 把hive中的jline的jar包拷贝到hadoop的

(/opt/software/hadoop-2.6.5/share/hadoop/yarn/lib/)目录的下，并把其中低版本的删掉，不然会报如下错：

 Logging initialized using configuration in

jar:file:/opt/software/hive-1.2.1/lib/hive-common-1.2.1.jar!/hive-log4j.properties

[ERROR] Terminal initialization failed; falling back to unsupported

java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

at jline.TerminalFactory.create(TerminalFactory.java:101)

at jline.TerminalFactory.get(TerminalFactory.java:158)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:229)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)

at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)

at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

 

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

at jline.console.ConsoleReader.<init>(ConsoleReader.java:230)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)

at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)

at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)

at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

 

[root@hadoop02 lib]# cp /opt/software/hive-1.2.1/lib/jline-2.12.jar ./

注意：Hive2.x 如果你在使用mysql之前启动过Hive，注意使用下面的命令将元数据同步到mysql，不然会报错：

schematool -dbType mysql -initSchema

 6. 验证是否配置成功：

hive> create table test01(id int,age int);

OK

Time taken: 2.148 seconds

hive> desc test01;

OK

id                   int                                      

age                  int                                      

Time taken: 0.486 seconds, Fetched: 2 row(s)

hive> insert into test01 values(1,23);

Query ID = root_20180429203346_b7a310a3-8ca0-4e2b-af9d-f55486352a98

Total jobs = 3

Launching Job 1 out of 3

Number of reduce tasks is set to 0 since there's no reduce operator

Starting Job = job_1525003393708_0001, Tracking URL = http://hadoop03:8088/proxy/application_1525003393708_0001/

Kill Command = /opt/software/hadoop-2.6.5/bin/hadoop job  -kill job_1525003393708_0001

Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0

2018-04-29 20:34:19,531 Stage-1 map = 0%,  reduce = 0%

2018-04-29 20:35:01,432 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.9 sec

MapReduce Total cumulative CPU time: 1 seconds 900 msec

Ended Job = job_1525003393708_0001

Stage-4 is selected by condition resolver.

Stage-3 is filtered out by condition resolver.

Stage-5 is filtered out by condition resolver.

Moving data to: hdfs://mycluster/user/hive_remote/warehouse/test01/.hive-staging_hive_2018-04-29_20-33-46_610_4511768652560755389-1/-ext-10000

Loading data to table default.test01

Table default.test01 stats: [numFiles=1, numRows=1, totalSize=5, rawDataSize=4]

MapReduce Jobs Launched:

Stage-Stage-1: Map: 1   Cumulative CPU: 1.9 sec   HDFS Read: 3548 HDFS Write: 75 SUCCESS

Total MapReduce CPU Time Spent: 1 seconds 900 msec

OK

Time taken: 78.059 seconds

hive>

 

 到hdfs表的存储路径查看：

 

使用命令查看这个文件

 

 当然也可以登录mysql查看元数据信息

 

表名：

 

字段：