hive问题汇总

1.Hive配置文件官网

https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin#AdminManualMetastoreAdmin-BasicConfigurationParameters

2. hive 不读hive-site.xml配置文件

当你修改完配置，但是hive启动时却不加载自己修改的hive-site.xml等配置文件的问题。它总是加载默认的配置文件。

这时候有人问，通过什么方式可以看出来，有什么区别？

解释：如果你是用默认的配置文件，hive会连接自带的derby数据库，会在hive的根目录下生成derby.log这个日志文件，并且当启动两个bin/hive的时候，就会报错（如果是mysql就不会）

原因：是我的hive-env.sh中

export HIVE_CONF_DIR=/opt/apache-hive-1.2.2-bin/conf

配的是1.2.1了

3. 建表报错：

 create table test2018042101(id int,name string);

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:For direct MetaStore DB connections, we don't support retries at the client level.)

解决：第一步：查看配置文件 确保没有出错

      第二步：运行 hive -hiveconf hive.root.logger=DEBUG,console打印hive启动过程的log。主要报错信息如下：

ERROR DataNucleus.Datastore: Error thrown executing CREATE TABLE `PARTITION_PARAMS`

(

    `PART_ID` BIGINT NOT NULL,

    `PARAM_KEY` VARCHAR(256) BINARY NOT NULL,

    `PARAM_VALUE` VARCHAR(4000) BINARY NULL,

    CONSTRAINT `PARTITION_PARAMS_PK` PRIMARY KEY (`PART_ID`,`PARAM_KEY`)

) ENGINE=INNODB : Specified key was too long; max key length is 767 bytes

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes

第三步：继续搜索解决方法，然后找到了可能是字符集的原因，可以通过在mysql中将数据库的字符集改为latin1，在mysql客户端执行以下命令：

mysql>  alter database hive character set latin1;

第四步：重启hive，查看log，问题解决了

 

4. 插入报错

insert into test201804210001 values (1,"lmx");

报错：Error during job, obtaining debugging information...

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

MapReduce Jobs Launched:

Stage-Stage-1:  HDFS Read: 0 HDFS Write: 0 FAIL

Total MapReduce CPU Time Spent: 0 msec

解决：在8088端口下查看运行日志http://192.168.1.31:8088/cluster/app/application_1524298962488_0001

 

点进去发现错误日志如下

Application application_1524298962488_0001 failed 2 times due to Error launching appattempt_1524298962488_0001_000002. Got exception: java.net.NoRouteToHostException: No Route to Host from vm1/192.168.1.31 to vm3:48923 failed on socket timeout exception: java.net.NoRouteToHostException: No route to host; For more details see: http://wiki.apache.org/hadoop/NoRouteToHost

at sun.reflect.GeneratedConstructorAccessor40.newInstance(Unknown Source)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance

(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)

at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:758)

at org.apache.hadoop.ipc.Client.call(Client.java:1479)

at org.apache.hadoop.ipc.Client.call(Client.java:1412)

at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)

at com.sun.proxy.$Proxy82.startContainers(Unknown Source)

at org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.

startContainers(ContainerManagementProtocolPBClientImpl.java:96)

at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.

java:191)

at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)

at com.sun.proxy.$Proxy83.startContainers(Unknown Source)

at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(

AMLauncher.java:118)

at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(

AMLauncher.java:250)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:748)

Caused by: java.net.NoRouteToHostException: No route to host

at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)

at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)

at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)

at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)

at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:614)

at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:712)

at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)

at org.apache.hadoop.ipc.Client.getConnection(Client.java:1528)

at org.apache.hadoop.ipc.Client.call(Client.java:1451)

... 15 more

. Failing the application.

原来是有一台datanode网络连接有问题，所以去查看这个节点防火墙是否关闭。

果然防火墙没有关闭，关闭防火墙之后插入正常：