win10启动hadoop时遇到的坑

目录

背景

1、?Exception in thread “main” java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): file:/// has no authority.

2、?Exiting with status 1: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory D: mphadoop-123dfs ame is in an inconsistent state: storage directory does not exist or is not accessible.

3、?Exiting with status 1: java.io.IOException: All specified directories are not accessible or do not exist.

4、NameNode is not formatted.

1、URI has an authority component

2、?util.ExitUtil: Exiting with status 1: java.lang.NullPointerException

5、Error occurred during initialization of VM

启动Hadoop

---

背景

昨日编译好了hadoop3.0.2源码，今日启动之，孰料想，又出现一堆错误

1、Exception in thread “main” java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): file:/// has no authority.

启动不了nameNode，解决方法：

在hadoop文件夹hadoop-3.0.2etchadoopcore-site.xml的标签下，添加如下内容

fs.default.name

hdfs://localhost:9000

2、Exiting with status 1: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory D: mphadoop-123dfs ame is in an inconsistent state: storage directory does not exist or is not accessible.

dfs ame文件夹没有创建。解决方法：

在某处创建dfs文件夹，然后在dfs文件夹里创建name和data文件夹，并且在hadoop文件夹etchadoopcore-site.xml的标签下，添加如下内容：

hadoop.tmp.dir

D:/ProgramCoding/hadoop-3.0.2/workspace

dfs.data.dir

D:/ProgramCoding/hadoop-3.0.2/workspace/dfs/data

dfs.name.dir

D:/ProgramCoding/hadoop-3.0.2/workspace/dfs/name

3、Exiting with status 1: java.io.IOException: All specified directories are not accessible or do not exist.

解决方法：

检查core_site.xml文件里所有文件分隔符，必须是/而不是windows的

4、NameNode is not formatted.

解决方法：

在cmd里运行命令hadoop namenode -format格式化namenode

格式化namenode时，又遇到一些问题

1、URI has an authority component

打开hadoop文件夹etchadoophdfs-site.xml，在标签里，输入如下内容

dfs.namenode.name.dir

file:/D:/ProgramCoding/hadoop-3.0.2/workspace/dfs/namenode/name

dfs.datanode.data.dir

file:/D:/ProgramCoding/hadoop-3.0.2/workspace/dfs/namenode/data

前提是保证两个标签下的文件夹都存在，如不存在，就创建

2、util.ExitUtil: Exiting with status 1: java.lang.NullPointerException

java.lang.NullPointerException

at java.io.File.(File.java:277)

at org.apache.hadoop.hdfs.server.namenode.NNStorage.setStorageDirectories(NNStorage.java:300)

at org.apache.hadoop.hdfs.server.namenode.NNStorage.(NNStorage.java:161)

at org.apache.hadoop.hdfs.server.namenode.FSImage.(FSImage.java:127)

at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:829)

at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1213)

at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1320)

解决方法：检查hadoop文件夹hadoop-3.0.2etchadoophdfs-site.xml文件里的值，绝对路径前面必须加上file:/

解决完这两个问题后，就可以正常格式化namenode了

5、Error occurred during initialization of VM

Could not reserve enough space for 1024000KB object heap

内存预留过大问题，解决方法：

修改文件hadoop文件夹etchadoophadoop-env.cmd，添加或修改HADOOP_HEAPSIZE的值为200（默认1000）

启动Hadoop

解决完上面问题后，就可以start-all了

出现四个窗口，命令运行不结束就行