hadoop_cluser

修改主机名：vi /etc/sysconfig/network

修改主机名和IP的映射关系：vim /etc/hosts

java安装

卸载之前版本的java

1.卸载java：rpm -qa | grep java

  卸载： rpm -e --nodeps  查询出来的包名

1.cd /opt

tar -zxvf jdk-7u75-linux-x64.tar.gz 

2.vi /etc/profile

        export JAVA_HOME=/opt/jdk1.7.0_75

        export PATH=$PATH:$JAVA_HOME/bin

防火墙

        #查看防火墙状态

        service iptables status

        #关闭防火墙

        service iptables stop

        #查看防火墙开机启动状态

        chkconfig iptables --list

        #关闭防火墙开机启动

        chkconfig iptables off

安装zookeeper

1.安装配置zooekeeper集群

1.1解压

tar -zxvf zookeeper-3.4.3.tar.gz 

1.2修改配置

cd /opt/zookeeper-3.4.3/conf/

cp zoo_sample.cfg zoo.cfg

vim  /opt/zookeeper-3.4.3/conf/zoo.cfg

修改：

dataDir=/opt/zookeeper-3.4.3/data

dataLogDir=/opt/zookeeper-3.4.3/logs

server.1=s1:2888:3888

server.2=s2:2888:3888

server.3=s3:2888:3888

保存退出

然后创建一个tmp文件夹

mkdir  /opt/zookeeper-3.4.3/data

mkdir /opt/zookeeper-3.4.3/logs

再创建一个空文件

touch  /opt/zookeeper-3.4.3/data/myid

最后向该文件写入IDaa

echo 1 >  /opt/zookeeper-3.4.3/data /myid

1.3将配置好的zookeeper拷贝到其他节点(首先分别在 hadoop2、 hadoop3

scp -r  /opt/zookeeper-3.4.3/  hadoop2:/opt/

scp -r  /opt/zookeeper-3.4.3/  hadoop3:/opt/

注意：修改hadoop1、hadoop2对应 /opt/zookeeper-3.4.3/tmp /myid内

hadoop2：

echo 2 >  /opt/zookeeper-3.4.3/data /myid

hadoop3：

echo 3 >  /opt/zookeeper-3.4.3/data /myid

    1.4启动zookeeper

  /opt/zookeeper-3.4.3/bin/zkServer.sh start

  /opt/zookeeper-3.4.3/ bin/zkServer.sh status

 报错

[ root@s1 bin]# /opt/zookeeper-3.4.3/bin/zkServer.sh status 
JMX enabled by default 
Using config: /opt/zookeeper-3.4.3/bin/../conf/zoo.cfg 
Error contacting service. It is probably not running.

多查看zookeeper.out得知

org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing /opt/zookeeper-3.4.3/bin/../conf/zoo.cfg 

        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:121) 

        at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101) 

        at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78) 

Caused by: java.lang.IllegalArgumentException: serverid null is not a number 

        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:358) 

        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:117) 

        ... 2 more

多一个节点：QuorumPeerMain

解决方法：

1.核对了myid之后，发现确实有问题，修改之，问题 依旧

2.文件夹都在的

3.修改zkServer.sh文件，问题依旧： http://blog.sina.com.cn/s/blog_72827fb101018yn9.html

4.现在查询2181端口：netstat -an |grep 2181已经启动，而且 zookeeper.out里面确实没有报错信息了，问题仍没有解决，

tail -f zookeeper.out报错    

2015-02-01 19:22:52,136 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181: FastLeaderElection@764] - Notification time out: 60000 
2015-02-01 19:23:52,139 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181: QuorumCnxManager@368] - Cannot open channel to 2 at election address s2/10.6.0.211:3888 
java.net.ConnectException: 拒绝连接 
at java.net.PlainSocketImpl.socketConnect(Native Method) 
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) 
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) 
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) 
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) 
at java.net.Socket.connect(Socket.java:579) 
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:354) 
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:388) 
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:755) 
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:716) 
2015-02-01 19:23:52,144 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181: QuorumCnxManager@368] - Cannot open channel to 3 at election address s3/10.6.0.86:3888 

[root@hadoopname snapshot]# echo ruok | nc hadoopName 2181

imok

每一个客户端都要启动一次zookeeper

安装hadoop

1.

 hadoop-env.sh

export JAVA_HOME=/opt/jdk1.7.0_75

export HADOOP_COMMON_LIB_NATIVE_DIR=/opt/hadoop-2.4.1/lib/native

export HADOOP_OPTS="-Djava.library.path=/opt/hadoop-2.4.1/lib"

2.core-site.xml，做如下配置：

<!-- 指定hdfs的nameservice为ns1 --> 

<property><name>fs.defaultFS</name><value>hdfs://ns1</value></property> 

<!-- 指定zookeeper地址 -->