haadmin介绍
hdfs haadmin命令可以查看、修改集群某一台服务器的状态
1[hadoop@hadoop001 ~]$ hdfs haadmin
2Usage: DFSHAAdmin [-ns <nameserviceId>]
3 [-transitionToActive <serviceId> [--forceactive]]
4 [-transitionToStandby <serviceId>]
5 [-failover [--forcefence] [--forceactive] <serviceId> <serviceId>]
6 [-getServiceState <serviceId>]
7 [-checkHealth <serviceId>]
8 [-help <command>]
9
10Generic options supported are
11-conf <configuration file> specify an application configuration file
12-D <property=value> use value for given property
13-fs <local|namenode:port> specify a namenode
14-jt <local|resourcemanager:port> specify a ResourceManager
15-files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster
16-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
17-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
18
19The general command line syntax is
20bin/hadoop command [genericOptions] [commandOptions]
-getServiceState 查看namenode处于什么状态
1[hadoop@ruozedata001 ~]$ hdfs haadmin -getServiceState nn1
2standby
3[hadoop@ruozedata001 ~]$ hdfs haadmin -getServiceState nn2
4active
-failover 可以修改namenode状态
1[hadoop@ruozedata001 ~]$ hdfs haadmin -failover nn1 nn2
2将nn1的active状态转换到nn2
fsck介绍
hdfs fsck可以检查你的文件是否有损坏情况
1[hadoop@hadoop001 ~]$ hdfs fsck
2Usage: DFSck <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]] [-maintenance]
3 <path> start checking from this path
4 -move move corrupted files to /lost+found
5 -delete delete corrupted files
6 -files print out files being checked
7 -openforwrite print out files opened for write
8 -includeSnapshots include snapshot data if the given path indicates a snapshottable directory or there are snapshottable directories under it
9 -list-corruptfileblocks print out list of missing blocks and files they belong to
10 -blocks print out block report
11 -locations print out locations for every block
12 -racks print out network topology for data-node locations
13
14 -maintenance print out maintenance state node details
15 -blockId print out which file this blockId belongs to, locations (nodes, racks) of this block, and other diagnostics info (under replicated, corrupted or not, etc)
16
17Please Note:
18 1. By default fsck ignores files opened for write, use -openforwrite to report such files. They are usually tagged CORRUPT or HEALTHY depending on their block allocation status
19 2. Option -includeSnapshots should not be used for comparing stats, should be used only for HEALTH check, as this may contain duplicates if the same file present in both original fs tree and inside snapshots.
20
21Generic options supported are
22-conf <configuration file> specify an application configuration file
23-D <property=value> use value for given property
24-fs <local|namenode:port> specify a namenode
25-jt <local|resourcemanager:port> specify a ResourceManager
26-files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster
27-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
28-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
29
30The general command line syntax is
31bin/hadoop command [genericOptions] [commandOptions]
32
33Generic options supported are
34-conf <configuration file> specify an application configuration file
35-D <property=value> use value for given property
36-fs <local|namenode:port> specify a namenode
37-jt <local|resourcemanager:port> specify a ResourceManager
38-files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster
39-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
40-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
41
42The general command line syntax is
43bin/hadoop command [genericOptions] [commandOptions]
hdfs fsck / 从/目录检查全部的文件
1[hadoop@hadoop001 ~]$ hdfs fcsk /
2Error: Could not find or load main class fcsk
3[hadoop@hadoop001 ~]$ hdfs fsck /
4Connecting to namenode via http://hadoop001:50070/fsck?ugi=hadoop&path=%2F
5FSCK started by hadoop (auth:SIMPLE) from /10.9.0.62 for path / at Tue Aug 20 15:50:12 CST 2019
6....................................................................................................
7....................................................................................................
8.............................................Status: HEALTHY
9 Total size: 140053299 B
10 Total dirs: 81
11 Total files: 245
12 Total symlinks: 0
13 Total blocks (validated): 239 (avg. block size 585997 B)
14 Minimally replicated blocks: 239 (100.0 %)
15 Over-replicated blocks: 0 (0.0 %)
16 Under-replicated blocks: 0 (0.0 %)
17 Mis-replicated blocks: 0 (0.0 %)
18 Default replication factor: 1
19 Average block replication: 1.0
20 Corrupt blocks: 0
21 Missing replicas: 0 (0.0 %)
22 Number of data-nodes: 1
23 Number of racks: 1
24FSCK ended at Tue Aug 20 15:50:12 CST 2019 in 49 milliseconds
25
26
27The filesystem under path '/' is HEALTHY
[hadoop@hadoop001 ~]$ hdfs fsck / -delete
可以删除损坏的文件
注意:使用删除命令要注意,确定文件是否重要以及是否可以恢复。