当我给自己的程序打包,然后放入Hadoop中运行时出现异常
- 17/03/29 22:38:37 INFO client.RMProxy: Connecting to ResourceManager at master/172.18.63.28:8032
- 17/03/29 22:38:37 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
- 17/03/29 22:38:37 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/hadoop/.staging/job_1490689337938_0001
- Exception in thread "main" org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot delete /tmp/hadoop-yarn/staging/hadoop/.staging/job_1490689337938_0001. Name node is in safe mode.
- The reported blocks 48 needs additional 5 blocks to reach the threshold 0.9990 of total blocks 53.
- The number of live datanodes 2 has reached the minimum number 0. Safe mode will be turned off automatically once the thresholds have been reached.
- at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1327)
- at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3713)
- at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:953)
- at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:611)
- at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
- at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
- at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
- at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
- at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
- at java.security.AccessController.doPrivileged(Native Method)
- at javax.security.auth.Subject.doAs(Subject.java:422)
- at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
- at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045)
- at org.apache.hadoop.ipc.Client.call(Client.java:1475)
- at org.apache.hadoop.ipc.Client.call(Client.java:1412)
- at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
- at com.sun.proxy.$Proxy10.delete(Unknown Source)
- at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:540)
- at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
- at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
- at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
- at java.lang.reflect.Method.invoke(Method.java:498)
- at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
- at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
- at com.sun.proxy.$Proxy11.delete(Unknown Source)
- at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:2044)
- at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:707)
- at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:703)
- at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
- at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:714)
- at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:251)
- at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
- at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
- at java.security.AccessController.doPrivileged(Native Method)
- at javax.security.auth.Subject.doAs(Subject.java:422)
- at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
- at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
- at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
- at hadooptext.wordCountDriver.main(wordCountDriver.java:63)
- at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
- at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
- at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
- at java.lang.reflect.Method.invoke(Method.java:498)
- at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
- at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Exception in thread "main" org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot delete /tmp/hadoop-yarn/staging/hadoop/.staging/job_1490689337938_0001. Name node is in safe mode.
这是说明NameNode处于安全模式
那么为什么NameNode会处于安全模式呢
1、NameNode发现集群中DataNode丢失达到一定比例(0.01%)时会进入安全模式,此时只允许查看数据不允许对数据进行任何操作。
2、HDFS集群即使启动正常,启动只会依旧会进入安全模式一段时间,这时你不需要理会他,稍等片刻即可。
3、集群升级维护时手动进入安全模式吗,命令如下
hadoop dfsadmin -safemode enter
那么如何退出安全模式呢?
使用命令
hadoop dfsadmin -safemode leave
然后再运行你打包的jar程序即可