1.启动hive时报错
java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.init(JDBCStatsPublisher.java:265)
at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:412)
Caused by: java.lang.SecurityException: sealing violation: package org.apache.derby.impl.jdbc.authentication is sealed
at java.net.URLClassLoader.getAndVerifyPackage(URLClassLoader.java:388)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:417)
复制代码
解决方案:
将mysql-connector-java-5.1.6-bin.jar包导入到$HIVE_HOME/lib目录下
复制代码
2.启动hive时报错:
[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
at jline.TerminalFactory.create(TerminalFactory.java:101)
at jline.TerminalFactory.get(TerminalFactory.java:158)
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
at jline.console.ConsoleReader.<init>(ConsoleReader.java:230)
at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
复制代码
解决方案:
将当前hive版本的$HIVE_HOME/lib目录下的jline-2.12.jar包拷贝到$HADOOP_HOME/share/hadoop/yarn/lib目录下, 并将旧版本的Hive的Jline包从$HADOOP_HOME/etc/hadoop/yarn/lib目录下删除
复制代码
3.启动hive时报错
Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
at org.apache.hadoop.fs.Path.initialize(Path.java:206)
at org.apache.hadoop.fs.Path.<init>(Path.java:172)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
at java.net.URI.checkPath(URI.java:1804)
at java.net.URI.<init>(URI.java:752)
at org.apache.hadoop.fs.Path.initialize(Path.java:203)
... 11 more
复制代码
解决方案:
1.查看hive-site.xml配置,会看到配置值含有"system:java.io.tmpdir"的配置项
2.新建文件夹${HIVE_HOME}/hive/logs
3.将含有"system:java.io.tmpdir"的配置项的值修改为${HIVE_HOME}/hive/logs
即: 新添属性为
<property>
<name>hive.exec.local.scratchdir</name>
<value>${HIVE_HOME}/logs/HiveJobsLog</value>
<description>Local scratch space for Hive jobs</description>
</property>
<property>
<name>hive.downloaded.resources.dir</name>
<value>${HIVE_HOME}/logs/ResourcesLog</value>
<description>Temporary local directory for added resources in the remote file system.</description>
</property>
<property>
<name>hive.querylog.location</name>
<value>${HIVE_HOME}/logs/HiveRunLog</value>
<description>Location of Hive run time structured log file</description>
</property>
<property>
<name>hive.server2.logging.operation.log.location</name>
<value>${HIVE_HOME}/logs/OpertitionLog</value>
<description>Top level directory where operation logs are stored if logging functionality is enabled</description>
</property>
复制代码
4.启动hive时报错
Caused by: java.sql.SQLException: Access denied for user 'root'@'master' (using password: YES)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:946)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2870)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:812)
at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:3269)
复制代码
解决方案:
mysql密码不正确, 查看hive-site.xml配置与mysql的密码是否一致
复制代码
5.操作表数据时(向表中导入数据), 报错:
FAILED: RuntimeException org.apache.hadoop.security.AccessControlException: Permission denied: user=services02, access=EXECUTE, inode="/tmp":services01:supergroup:drwx------
复制代码
解决方案:
user=services02与inode="/tmp":services01:supergroup不同时,说明hive登录的主机与HDFS的active状态的主机不一样
应把user=services02的主机变为HDFS的active状态的主机.
复制代码
二.扩展Parquet功能:
- 1.创建存储格式为parquet时,建表语句为: (Hive版本为0.12)
create table parquet_test(x int, y string)
row format serde 'parquet.hive.serde.ParquetHiveSerDe'
stored as inputformat 'parquet.hive.DeprecatedParquetInputFormat'
outputformat 'parquet.hive.DeprecatedParquetOutputFormat';
复制代码
报错:
FAILED: SemanticException [Error 10055]: Output Format must implement
HiveOutputFormat, otherwise it should be either IgnoreKeyTextOutputFormat or
SequenceFileOutputFormat
复制代码
*解决方案
因为parquet.hive.DeprecatedParquetOutputFormat类并没有在Hive的CLASSPATH中配置
单独下载parquet-hive-1.2.5.jar包(此类属于$IMPALA_HOME/lib目录下), 在$HIVE_HOME/lib目录下建立个软链就可以了
复制代码
cd $HIVE_HOME/lib
ln -s $/home/hadoop/soft/gz.zip/parquet-hive-1.2.5.jar
复制代码
2.继续提交创表语句: (Hive版本为0.12)
create table parquet_test(x int, y string)
row format serde 'parquet.hive.serde.ParquetHiveSerDe'
stored as inputformat 'parquet.hive.DeprecatedParquetInputFormat'
outputformat 'parquet.hive.DeprecatedParquetOutputFormat';
复制代码
报错:
Exception in thread "main" java.lang.NoClassDefFoundError: parquet/hadoop/api/WriteSupport
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
复制代码
解决方案:
通过yum下载Parquet
sodu yum -y install parquet
下载parquet的jar包在/usr/lib/parquet目录下, 将/usr/lib/parquet目录下的所有jar(除javadoc.jar和sources.jar外)拷贝到$HIVE_HOME/lib目录下.
若yum无法下载parquet资源包, 则这是需要配置yum源, 请自行百度寻找资料
复制代码