查看CPU:cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c
查看内存:free –m
查看磁盘:df -m
处理器信息:dmidecode | grep -A48 'Processor Information$'
查看目录的大小
[hadoop@masternode1 ~]$ hdfs dfs -ls /
Found 8 items
drwxr-xr-x - hadoop supergroup 0 2016-11-09 15:38 /datatmp
drwxr-xr-x - hadoop supergroup 0 2016-10-10 11:18 /flumeTest
drwxr-xr-x - hauser supergroup 0 2016-11-04 16:58 /hauser_home
drwxr-xr-x - hadoop supergroup 0 2016-11-09 17:10 /hbase
drwx-wx-wx - hadoop supergroup 0 2016-10-08 16:31 /hive
drwxr-xr-x - hadoop supergroup 0 2016-10-09 12:57 /spark
drwx-wx-wx - hadoop supergroup 0 2016-10-26 14:35 /tmp
drwx------ - hadoop supergroup 0 2016-10-26 14:35 /user
[hadoop@masternode1 ~]$ hadoop fsck /datatmp
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
Connecting to namenode via http://masternode2:50070/fsck?ugi=hadoop&path=%2Fdatatmp
FSCK started by hadoop (auth:SIMPLE) from /192.168.237.230 for path /datatmp at Thu Nov 10 11:38:17 CST 2016
....................................................................................................
...Status: HEALTHY
Total size: 10800425600 B
Total dirs: 1
Total files: 103
Total symlinks: 0
Total blocks (validated): 103 (avg. block size 104858500 B)
Minimally replicated blocks: 103 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 3.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 7
Number of racks: 1
FSCK ended at Thu Nov 10 11:38:17 CST 2016 in 27 milliseconds
Did you mean -du? This command begins with a dash.
[hadoop@masternode1 ~]$ hadoop fs -du -s /datatmp
10800425600 /datatmp
[hadoop@masternode1 ~]$ hadoop fs -count -q /datatmp
none inf none inf 1 103 10800425600 /datatmp
[hadoop@masternode1 ~]$
创建目录
hadoop dfs -mkdir /home
上传文件或目录到hdfs
hadoop dfs -put hello /
hadoop dfs -put hellodir/ /
查看目录
hadoop dfs -ls /
创建一个空文件
hadoop dfs -touchz /wahaha
删除一个文件
hadoop dfs -rm /wahaha
删除一个目录
hadoop dfs -rmr /home
重命名
hadoop dfs -mv /hello1 /hello2
查看文件
hadoop dfs -cat /hello
将制定目录下的所有内容merge成一个文件,下载到本地
hadoop dfs -getmerge /hellodir wa
使用du文件和目录大小
hadoop dfs -du /
将目录拷贝到本地
hadoop dfs -copyToLocal /home localdir
查看dfs的情况
hadoop dfsadmin -report
查看正在跑的Java程序
jps
确定要转为active的namenode的id,这里将namenode1设为active,使用命令行工具进行状态切换:
$HADOOP_HOME/bin/hdfs haadmin -failover --forcefence --forceactive nn2 nn1
注意:此处“nn2 nn1”的顺序表示active状态由nn2转换到nn1上(虽然nn2在转化前也是standby状态)。
获取状态
hdfs haadmin -getServiceState mnn
核查心跳
hdfs haadmin -checkHealth snn
查看mr 状态的命令
yarn rmadmin –getServiceState rm1
状态切换的命令
yarn rmadmin –transitionToStandby rm1
- hadoop fs -appendToFile localfile /user/hadoop/hadoopfile
- hadoop fs -appendToFile localfile1 localfile2 /user/hadoop/hadoopfile
- hadoop fs -appendToFile localfile hdfs://nn.example.com/hadoop/hadoopfile
- hadoop fs -appendToFile - hdfs://nn.example.com/hadoop/hadoopfile Reads the input from stdin.
- hadoop fs -cat hdfs://nn1.example.com/file1 hdfs://nn2.example.com/file2
- hadoop fs -cat file:///file3 /user/hadoop/file4
- hadoop fs -checksum hdfs://nn1.example.com/file1
- hadoop fs -checksum file:///etc/hosts
- hadoop fs -chgrp [-R] GROUP URI [URI ...]
- hadoop fs -chmod [-R] <MODE[,MODE]... | OCTALMODE> URI [URI ...]
- adoop fs -count hdfs://nn1.example.com/file1 hdfs://nn2.example.com/file2
- hadoop fs -count -q hdfs://nn1.example.com/file1
- hadoop fs -count -q -h hdfs://nn1.example.com/file1
- hadoop fs -count -q -h -v hdfs://nn1.example.com/file1
- hadoop fs -count -u hdfs://nn1.example.com/file1
- hadoop fs -count -u -h hdfs://nn1.example.com/file1
- hadoop fs -count -u -h -v hdfs://nn1.example.com/file1
- hadoop fs -cp /user/hadoop/file1 /user/hadoop/file2
- hadoop fs -cp /user/hadoop/file1 /user/hadoop/file2 /user/hadoop/dir
- hadoop fs -get /user/hadoop/file localfile
- hadoop fs -get hdfs://nn.example.com/user/hadoop/file localfile
- hadoop dfs -df /user/hadoop/dir1
- hadoop fs -find / -name test -print
- hadoop fs -getfacl /file
- hadoop fs -getfacl -R /dir
- hadoop fs -getfattr -d /file
- hadoop fs -getfattr -R -n user.myAttr /dir
- hadoop fs -getmerge -nl /src /opt/output.txt
- hadoop fs -getmerge -nl /src/file1.txt /src/file2.txt /output.txt
- hadoop fs -mkdir /user/hadoop/dir1 /user/hadoop/dir2
- hadoop fs -mkdir hdfs://nn1.example.com/user/hadoop/dir hdfs://nn2.example.com/user/hadoop/dir
- hadoop fs -mv /user/hadoop/file1 /user/hadoop/file2
- hadoop fs -mv hdfs://nn.example.com/file1 hdfs://nn.example.com/file2 hdfs://nn.example.com/file3 hdfs://nn.example.com/dir1
- hadoop fs -put localfile /user/hadoop/hadoopfile
- hadoop fs -put localfile1 localfile2 /user/hadoop/hadoopdir
- hadoop fs -put localfile hdfs://nn.example.com/hadoop/hadoopfile
- hadoop fs -put - hdfs://nn.example.com/hadoop/hadoopfile Reads the input from stdin.
- hadoop fs -rmdir /user/hadoop/emptydir
- hadoop fs -setfacl -m user:hadoop:rw- /file
- hadoop fs -setfacl -x user:hadoop /file
- hadoop fs -setfacl -b /file
- hadoop fs -setfacl -k /dir
- hadoop fs -setfacl --set user::rw-,user:hadoop:rw-,group::r--,other::r-- /file
- hadoop fs -setfacl -R -m user:hadoop:r-x /dir
- hadoop fs -setfacl -m default:user:hadoop:r-x /dir
- hadoop fs -setfattr -n user.myAttr -v myValue /file
- hadoop fs -setfattr -n user.noValue /file
- hadoop fs -setfattr -x user.myAttr /file
- hadoop fs -setrep -w 3 /user/hadoop/dir1
- hadoop fs -stat "%F %u:%g %b %y %n" /file
- hadoop fs -tail pathname
- hadoop fs -test -[defsz] URI
- hadoop fs -touchz pathname
- hadoop fs -truncate 55 /user/hadoop/file1 /user/hadoop/file2
- hadoop fs -truncate -w 127 hdfs://nn1.example.com/user/hadoop/file1