hadoop环境搭建

http://wgkgood.blog.51cto.com/1192594/1332340

仿照如上文章进行了环境搭建

内容大致总结为

1. 配置集群内ssh免密码互相登录。请参照http://www.cnblogs.com/vigarbuaa/p/3579291.html

2. 安装hadoop. 配置如下文件

core-site.xml/mapred-site.xml/hdfs-site.xml/hadoop-env.sh/masters/slaves

3. 格式化namenode

 

root@kali:/data/hadoop/bin# hadoop namenode -format
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

14/03/03 21:56:52 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************

 

4.  启动服务

root@kali:/data/hadoop/sbin# ./start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [192.168.0.100]
192.168.0.100: ssh: connect to host 192.168.0.100 port 22: Connection refused
The authenticity of host '192.168.0.101 (192.168.0.101)' can't be established.
ECDSA key fingerprint is 64:9a:7d:2c:61:40:dc:66:8b:00:12:0e:98:67:2d:22.
Are you sure you want to continue connecting (yes/no)? yes
192.168.0.101: Warning: Permanently added '192.168.0.101' (ECDSA) to the list of known hosts.
192.168.0.101: starting datanode, logging to /data/hadoop/logs/hadoop-root-datanode-kali.out
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is 64:9a:7d:2c:61:40:dc:66:8b:00:12:0e:98:67:2d:22.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
0.0.0.0: starting secondarynamenode, logging to /data/hadoop/logs/hadoop-root-secondarynamenode-kali.out
starting yarn daemons
resourcemanager running as process 5507. Stop it first.
192.168.0.101: starting nodemanager, logging to /data/hadoop/logs/yarn-root-nodemanager-kali.out

5. 验证服务是否启动

root@kali:/data/hadoop/sbin# jps -l
5507 org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
5874 org.apache.hadoop.hdfs.server.datanode.DataNode
6196 org.apache.hadoop.yarn.server.nodemanager.NodeManager
6291 sun.tools.jps.Jps
6005 org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode