启动后可通过命令行使用hadoop。
(1)所有命令 (先将$HADOOP_HOME/bin加入到.bashrc的$PATH变量中)
[hadoop@node14 hadoop-0.21.0]$ ll $HADOOP_HOME/bin
total 88
-rwxr-xr-x 1 hadoop hadoop 4131 Aug 17 2010 hadoop
-rwxr-xr-x 1 hadoop hadoop 8658 Aug 17 2010 hadoop-config.sh
-rwxr-xr-x 1 hadoop hadoop 3841 Aug 17 2010 hadoop-daemon.sh
-rwxr-xr-x 1 hadoop hadoop 1242 Aug 17 2010 hadoop-daemons.sh
-rwxr-xr-x 1 hadoop hadoop 4130 Aug 17 2010 hdfs
-rwxr-xr-x 1 hadoop hadoop 1201 Aug 17 2010 hdfs-config.sh
-rwxr-xr-x 1 hadoop hadoop 3387 Aug 17 2010 mapred
-rwxr-xr-x 1 hadoop hadoop 1207 Aug 17 2010 mapred-config.sh
-rwxr-xr-x 1 hadoop hadoop 2720 Aug 17 2010 rcc
-rwxr-xr-x 1 hadoop hadoop 2058 Aug 17 2010 slaves.sh
-rwxr-xr-x 1 hadoop hadoop 1367 Aug 17 2010 start-all.sh
-rwxr-xr-x 1 hadoop hadoop 1018 Aug 17 2010 start-balancer.sh
-rwxr-xr-x 1 hadoop hadoop 1778 Aug 17 2010 start-dfs.sh
-rwxr-xr-x 1 hadoop hadoop 1255 Aug 17 2010 start-mapred.sh
-rwxr-xr-x 1 hadoop hadoop 1359 Aug 17 2010 stop-all.sh
-rwxr-xr-x 1 hadoop hadoop 1069 Aug 17 2010 stop-balancer.sh
-rwxr-xr-x 1 hadoop hadoop 1277 Aug 17 2010 stop-dfs.sh
-rwxr-xr-x 1 hadoop hadoop 1163 Aug 17 2010 stop-mapred.sh
(2) hadoop命令
[hadoop@node14 hadoop-0.21.0]$ hadoop
Usage: hadoop [--config confdir] COMMAND
where COMMAND is one of:
fs run a generic filesystem user client
version print the version
jar <jar> run a jar file
distcp <srcurl> <desturl> copy file or directories recursively
archive -archiveName NAME -p <parent path> <src>* <dest> create a hadoop archive
classpath prints the class path needed to get the
Hadoop jar and the required libraries
daemonlog get/set the log level for each daemon
or
CLASSNAME run the class named CLASSNAME
Most commands print help when invoked w/o parameters.
(3) hadoop fs
[hadoop@node14 hadoop-0.21.0]$ hadoop fs
Usage: java FsShell
[-ls <path>]
[-lsr <path>]
[-df [<path>]]
[-du [-s] [-h] <path>]
[-dus <path>]
[-count[-q] <path>]
[-mv <src> <dst>]
[-cp <src> <dst>]
[-rm [-skipTrash] <path>]
[-rmr [-skipTrash] <path>]
[-expunge]
[-put <localsrc> ... <dst>]
[-copyFromLocal <localsrc> ... <dst>]
[-moveFromLocal <localsrc> ... <dst>]
[-get [-ignoreCrc] [-crc] <src> <localdst>]
[-getmerge <src> <localdst> [addnl]]
[-cat <src>]
[-text <src>]
[-copyToLocal [-ignoreCrc] [-crc] <src> <localdst>]
[-moveToLocal [-crc] <src> <localdst>]
[-mkdir <path>]
[-setrep [-R] [-w] <rep> <path/file>]
[-touchz <path>]
[-test -[ezd] <path>]
[-stat [format] <path>]
[-tail [-f] <file>]
[-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
[-chown [-R] [OWNER][:[GROUP]] PATH...]
[-chgrp [-R] GROUP PATH...]
[-help [cmd]]
Generic options supported are
-conf <configuration file> specify an application configuration file
-D <property=value> use value for given property
-fs <local|namenode:port> specify a namenode
-jt <local|jobtracker:port> specify a job tracker
-files <comma separated list of files> specify comma separated files to be co pied to the map reduce cluster
-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]
(4)HDFS操作
hadoop fs
hadoop fs -ls
hadoop fs -mkdir firstdir //HDFS创建目录
hadoop fs -rmr firstdir //HDFS删除目录
hadoop fs -put test.txt first.txt //从本地目录中将文件放入HDFS
hadoop fs -cat first.txt
hadoop fs -df
hadoop fs -get first.txt FirstTXTfromHDFS.txt //从HDFS取文件到本地
若文件写入遇到异常
(0)检查和机器名是否正确
node14配置了外部IP和内部IP,在/etc/hosts中加入两条IP与机器名的对应表,如果外部IP放在内部IP的前面,
则通过netstat -npl查看时,发现9000和9001是外部IP占据,故应在/etc/hosts中内部IP放在外部的IP的前面。
或者在conf中配置文件中,全部用IP,而不要用机器名。
(1)关闭防火墙
sudo /etc/init.d/iptables stop
(2)查看磁盘空间是否正常
df -hl
(3)检查目录是否正常
hadoop.tmp.dir默认:/tmp/hadoop-${user.name}
删除/tmp下面的文件,重新hadoop namenode -format,重启所有进程。
(4)单独启动各个进程
在namenode和datanode上分别启动节点
$hadoop-daemon.sh start namenode
$hadoop-daemon.sh start datanode