1.ssh command not found
解决 yum remove openssh
yum -y install openssh*
2. centos 网管设置
nameserver 8.8.8.8
nameserver 8.8.4.4
3. 默认网关设置
设置文件 /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=localhost.localdomain
GATEWAY=192.168.1.1
4. IP设置
设置文件 /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=00:0C:29:EF:89:9A
TYPE=Ethernet
UUID=765d7345-0ea4-4a0b-8aa7-fd744b769ab6
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.1.224
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
5.环境变量设置
设置文件 /etc/profile
export HADOOP_HOME_WARN_SUPPRESS=1
export HADOOP_HOME=/usr/local/hadoop
export JAVA_HOME=/usr/local/jdk
export PATH=.:$HADOOP_HOME:$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools/jar
6.设置无密码登录
cd ~ 可以进入你的home目录
1.在A机中执行 cd ~/.ssh
2.ssh-keygen -t rsa --------- 然后一路回车 就会按照默认的选项将生成的密钥保存在.ssh/id_rsa文件中
3. cp id_rsa.pub authorized_keys 这步完成后,正常情况下就可以无密码登陆本机了 即ssh localhost 无需输入密码
4. scp authorized_keys root@192.168.1.224:/root/.ssh
5. 进入B机 chomd 600 authorized_keys
正常情况下上面几步执行完成后,从主机A所在机器向主机A、主机B所在机器发起ssh连接,只有在第一次登录时需要输入密码,以后则不需要。
hadoop中运行bin/hadoop fs -mkdir /test-hadoop报错
mkdir: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /test-hadoop. Name node is in safe mode.
解决方法: bin/hadoop dfsadmin -safemode leave #关闭safe mode
hadoop 的文件命令格式如下:
hadoop fs -cmd <args>
//建立目录
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -mkdir /test-hadoop
//査看目录
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -ls /
Found 2 items
drwxr-xr-x - hadoop supergroup 0 2012-02-02 13:32 /test-hadoop
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp
//査看目录包括子目录
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -lsr /
drwxr-xr-x - hadoop supergroup 0 2012-02-02 13:32 /test-hadoop
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred
drwx------ - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred/system
-rw------- 2 hadoop supergroup 4 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred/system/jobtracker.info
//添加文件
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -put /home/hadoop/hadoop-0.20.203.0rc1.tar.gz /test-hadoop
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -lsr /
drwxr-xr-x - hadoop supergroup 0 2012-02-02 13:34 /test-hadoop
-rw-r--r-- 2 hadoop supergroup 60569605 2012-02-02 13:34 /test-hadoop/hadoop-0.20.203.0rc1.tar.gz
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred
drwx------ - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred/system
-rw------- 2 hadoop supergroup 4 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred/system/jobtracker.info
//获取文件
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -get /test-hadoop/hadoop-0.20.203.0rc1.tar.gz /tmp/
[hadoop@hadoop1 hadoop]$ ls /tmp/*.tar.gz
/tmp/1.tar.gz /tmp/hadoop-0.20.203.0rc1.tar.gz
//删除文件
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -rm /test-hadoop/hadoop-0.20.203.0rc1.tar.gz
Deleted hdfs://namenode:9000/test-hadoop/hadoop-0.20.203.0rc1.tar.gz
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -lsr /
drwxr-xr-x - hadoop supergroup 0 2012-02-02 13:57 /test-hadoop
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred
drwx------ - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred/system
-rw------- 2 hadoop supergroup 4 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred/system/jobtracker.info
drwxr-xr-x - hadoop supergroup 0 2012-02-02 13:36 /user
-rw-r--r-- 2 hadoop supergroup 321 2012-02-02 13:36 /user/hadoop
//删除目录
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -rmr /test-hadoop
Deleted hdfs://namenode:9000/test-hadoop
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -lsr /
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop
drwxr-xr-x - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred
drwx------ - hadoop supergroup 0 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred/system
-rw------- 2 hadoop supergroup 4 2012-02-02 11:32 /tmp/hadoop-hadoop/mapred/system/jobtracker.info
drwxr-xr-x - hadoop supergroup 0 2012-02-02 13:36 /user
-rw-r--r-- 2 hadoop supergroup 321 2012-02-02 13:36 /user/hadoop
//hadoop fs 帮助(部分)
[hadoop@hadoop1 hadoop]$ bin/hadoop fs -help
hadoop fs is the command to execute fs commands. The full syntax is:
hadoop fs [-fs <local | file system URI>] [-conf <configuration file>]
[-D <propertyproperty=value>] [-ls <path>] [-lsr <path>] [-du <path>]
[-dus <path>] [-mv <src> <dst>] [-cp <src> <dst>] [-rm [-skipTrash] <src>]
[-rmr [-skipTrash] <src>] [-put <localsrc> ... <dst>] [-copyFromLocal <localsrc> ... <dst>]
[-moveFromLocal <localsrc> ... <dst>] [-get [-ignoreCrc] [-crc] <src> <localdst>
[-getmerge <src> <localdst> [addnl]] [-cat <src>]
[-copyToLocal [-ignoreCrc] [-crc] <src> <localdst>] [-moveToLocal <src> <localdst>]
[-mkdir <path>] [-report] [-setrep [-R] [-w] <rep> <path/file>]
[-touchz <path>] [-test -[ezd] <path>] [-stat [format] <path>]
[-tail [-f] <path>] [-text <path>]
[-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
[-chown [-R] [OWNER][:[GROUP]] PATH...]
[-chgrp [-R] GROUP PATH...]
[-count[-q] <path>]
[-help [cmd]]
在Hadoop添加文件中可能出现
WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /test-hadoop/hadoop-1.1.2.tar.gz could only be replicated to 0 nodes, instead of 1
解决方法:
先执行 sh stop-all.sh
重新启动namenode
# hadoop-daemon.sh start namenode
starting namenode, logging to /usr/hadoop-0.21.0/bin/../logs/hadoop-root-namenode-www.keli.com.out
重新启动datanode
# hadoop-daemon.sh start datanode