hadoop配置总结

install  (hadoop-2.2.0.tar.gz + hbase-0.96.0-hadoop2-bin.tar.gz)
------------------------------------------------------------------------------

A:  ssh (root or other same user)

master:

1 ssh-keygen -t rsa
2 cp id_rsa.pub authorized_keys  

slave1 2 3: (same user as master_using )

1 ssh-keygen -t rsa
2 cp id_rsa.pub datanode1.pub          (datanode2 3 4...)
3 scp datanode1.pub master:/root/.ssh

master:

1 cat datanode1.pub >> authorized_keys
2 (cat datanode2.pub datanode3 4 5..... >> )
3 scp authorized_keys slave1:/root/.ssh/ (slave2 3 4....)

gedit /etc/ssh/sshd_config:

1 RSAAuthentication yes
          PubkeyAuthentication yes
          AuthorizedKeysFile .ssh/authorized_keys
2 PasswordAuthentication yes  (two ways of ssh_login)
3 GSSAPIAuthentication no     

 GSSAPIStrictAcceptorCheck no

gedit /etc/hosts

1 127.0.0.1 localhost 
2 192.168.0.110 slave1 
3 192.168.0.111 master
    ssh master; ssh slave1 

------------------------------------------------------------------------------

B hadoop2.2

 master:

1 hadoop-env.sh
2 core-site.xml
3 hdfs-site.xml 
4 yarn-site.xml
5 yarn-env.sh  
6 slaves
7 scp -r hadoop slave1:/usr/local/   (slave2 3 ....)

C hbase0.96(hbase-0.96.0-hadoop2-bin.tar.gz)

master:

1 hbase-env.sh
2 hbase-site.xml
3 regionservers
4 replace  /hbase/lib/hadoop-common-2.1.0-beta.jar
                   /hbase/lib/hadoop-hdfs-2.1.0-beta.jar

  /hbase/lib/hadoop-auth-2.1.0-beta.jar

 with         hadoop-common-2.2.0.jar

    hadoop-hdfs-2.2.0.jar

                   hadoop-auth-2.2.0.jar 
5 scp -r hbase slave1:/usr/local (slave2 3 ....)

D start hadoop 

1 bin/hdfs namenode -format
2 sbin/start-all.sh
3 jps (master)

4705 SecondaryNameNode

6722 Jps

4446 NameNode

4562 DataNode

4837 ResourceManager

4 master:50070 

E start hbase 

1 ./start-hbase
2 jps 

5537 Main

4705 SecondaryNameNode

5364 HMaster

4446 NameNode

5305 HQuorumPeer

6408 Jps

4562 DataNode

4837 ResourceManager

3 master:60010
4 ./hbase shell

list create put get 

----------------------------------------------------------------------------- 

reference :  

1 http://blog.csdn.net/java_kfc/article/details/17258657
2 http://blog.sina.com.cn/s/blog_72827fb10101a3dx.html 

Q:   Can't get connection to ZooKeeper: KeeperErrorCode = ConnectionLoss for /hba
A:  /etc/init.d/iptables stop 

3 setting error

-----------------------------------------------------------------------------

Question:

1 find the fit version -> hadoop 2.2 and hbase-0.96_hadoop2:  you will fail
2 /etc/init.d/iptables stop -> zoopkeeper stop connection failed...
3 del tmp and dfs/name ...  init 6  try again
4 start: ./hdfs namenode -format -> ./start-all.sh -> jps -> ./start-hbase.sh -> ./hbase shell
5 read log to find errors and ways 
6 different version different setting;  the best way to solve the problem is to read the document...