Caused by: java.lang.IllegalArgumentException: Pathname:scratch_dir is not a valid DFS filename

最新推荐文章于 2024-06-21 17:07:04 发布

躺平的赶海人

最新推荐文章于 2024-06-21 17:07:04 发布

阅读量669

点赞数

分类专栏： Hive

本文链接：https://blog.csdn.net/anzhenxi3529/article/details/104723249

版权

Hive 专栏收录该内容

8 篇文章 0 订阅

订阅专栏

E:\hive\apache-hive-0.14.0-bin\bin>hive

Logging initialized using configuration in file:/E:/hive/apache-hive-0.14.0-bin/conf/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/E:/hadoop/hadoop-2.6.5/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/imp
l/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/E:/hive/apache-hive-0.14.0-bin/lib/hive-jdbc-0.14.0-standalone.jar!/org/slf4j/impl/St
aticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/E:/hbase/hbase-1.3.5/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.c
lass]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: Pathname /E:/hive/apache-hive
-0.14.0-bin/my_hive/scratch_dir from E:/hive/apache-hive-0.14.0-bin/my_hive/scratch_dir is not a valid DFS filename.
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.IllegalArgumentException: Pathname /E:/hive/apache-hive-0.14.0-bin/my_hive/scratch_dir from E:/hive
/apache-hive-0.14.0-bin/my_hive/scratch_dir is not a valid DFS filename.
        at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:196)
        at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105)
        at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
        at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
        at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
        at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:520)
        at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:478)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:430)
        ... 8 more

在hive-site.xml文件中修改如下：

<property>
    <name>hive.exec.scratchdir</name>
    <value>/tmp</value>
    <description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/&lt;username&gt; is created, with ${hive.scratch.dir.permission}.</description>
  </property>

E:\hive\apache-hive-0.14.0-bin\bin>hadoop fs -chmod -R 0777 /tmp

E:\hive\apache-hive-0.14.0-bin\bin>hive

Logging initialized using configuration in file:/E:/hive/apache-hive-0.14.0-bin/conf/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/E:/hadoop/hadoop-2.6.5/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/imp
l/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/E:/hive/apache-hive-0.14.0-bin/lib/hive-jdbc-0.14.0-standalone.jar!/org/slf4j/impl/St
aticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/E:/hbase/hbase-1.3.5/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.c
lass]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
hive>