cdh运维

1在调用hbase的api，进行一条一条的put数据上去，发现当put到几千条条数据就挂掉

slave02:50010:DataXceiver error processing WRITE_BLOCK operation  src: /10.10.109.199:56026 dst: /10.10.109.200:50010
java.io.IOException: Premature EOF from inputStream
	at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:203)
	at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)
	at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)
	at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)
	at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:501)
	at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:901)
	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:808)
	at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:169)
	at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:106)
	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:246)
	at java.lang.Thread.run(Thread.java:748)

2datanode启动报java.io.FileNotFoundException: /dfs/dn/current/VERSION (Permission denied)

Failed to add storage directory [DISK]file:/dfs/dn/
java.io.FileNotFoundException: /dfs/dn/current/VERSION (Permission denied)
	at java.io.RandomAccessFile.open0(Native Method)
	at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
	at org.apache.hadoop.hdfs.server.common.StorageInfo.readPropertiesFile(StorageInfo.java:245)
	at org.apache.hadoop.hdfs.server.common.StorageInfo.readProperties(StorageInfo.java:231)
	at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:759)
	at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadStorageDirectory(DataStorage.java:302)
	at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:418)
	at org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:397)
	at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:575)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1561)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1521)
	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:341)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:219)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:673)
	at java.lang.Thread.run(Thread.java:748)

由错误可以知道是由于启动时候读取 /dfs/dn/current/VERSION没有权限
查看master的文件权限：

查看slave的文件

查看 /dfs/dn/current/VERSION的用户组跟用户，为root权限，正常启动需要的是hdfs权限

进行赋权操作

[root@slave01 current]# chown -R hdfs /dfs/dn/current/VERSION 
[root@slave01 current]# chgrp -R hdfs /dfs/dn/current/VERSION 

[root@slave02 current]# ll
total 8
drwxr-xr-x 4 hdfs hdfs 4096 Jun  1 11:25 BP-293734685-10.10.109.198-1557142304160
-rw-r--r-- 1 hdfs hdfs  198 Jun  3 18:12 VERSION
[root@slave02 current]# 

到此集群成功启动

3将本地文件拷贝到hdfs上去，结果上错误：Name node is in safe mode

查看日志文件

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot create directory /tmp/hive/hdfs/dbaa2301-9f86-42f6-b9d8-44aaca6e6479. Name node is in safe mode.
The reported blocks 0 needs additional 2701 blocks to reach the threshold 0.9990 of total blocks 2703.
The number of live datanodes 0 needs an additional 1 live datanodes to reach the minimum number 1.
Safe mode will be turned off automatically once the thresholds have been reached.

将本地文件拷贝到hdfs上去，结果上错误：Name node is in safe mode

这是因为在分布式文件系统启动的时候，开始的时候会有安全模式，当分布式文件系统处于安全模式的情况下，文件系统中的内容不允许修改也不允许删除，直到安全模式结束。安全模式主要是为了系统启动的时候检查各个DataNode上数据块的有效性，同时根据策略必要的复制或者删除部分数据块。运行期通过命令也可以进入安全模式。在实践过程中，系统启动的时候去修改和删除文件也会有安全模式不允许修改的出错提示，只需要等待一会儿即可。

可以通过以下命令来手动离开安全模式：

bin/hadoop dfsadmin -safemode leave

用户可以通过dfsadmin -safemode value 来操作安全模式，参数value的说明如下：

enter - 进入安全模式

leave - 强制NameNode离开安全模式

get - 返回安全模式是否开启的信息wait - 等待，一直到安全模式结束
4find命令查找文件名

在这里插入代码片