由于项目需要现在我的分布式环境已换位hadoop-0.20.2,并搭建了hbase-0.90.5,几经坎坷终于成功了,其中也遇到很多问题这里记录下来
分布式hadoop-0.20.2再上一篇文章里写过了,差别不大,其中主节点是node31,从节点是node29,node30。这里只写hbase的搭建。
1 首先下载hbase到主节点node31上,解压
2 配置环境变量
export HBASE_HOME=/home/ruc/hbase-0.90.5
export PATH=$PATH:$HBASE_HOME/bin
export CLASSPATH=$CLASSPATH:$HBASE_HOME/*.jar:$HBASE_HOME/lib/*.jar
3 修改hbase-env.sh
export JAVA_HOME=/usr/lib/jvm/jdk/jdk1.6.0_33
export HBASE_MANAGES_ZK=true
去掉这两行的注释
4 修改hbase-site.xml
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://node31:9000/hbase</value>这个要和hadoop的core-site.xml配置相同
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>由于是分布式的这个属性设为true
</property>
<property>
<name>zookeeper.session.timeout</name>
<value>300000</value>这个规定了访问超时的限制时长,默认180000,但是后来报错zookeeper连接有问题所以调大了
<description>ZooKeeper session timeout.
HBase passes this to the zk quorum as suggested maximum time for a
session. See http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions
"The client sends a requested timeout, the server responds with the
timeout that it can give the client. "
In milliseconds.
</description>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>node29,node30</value>规定了运行ZooKeeper的服务器,这里设置为两个从节点的名称
<description>Comma separated list of servers in the ZooKeeper Quorum.
For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
By default this is set to localhost for local and pseudo-distributed modes
of operation. For a fully-distributed setup, this should be set to a full
list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh
this is the list of servers which we will start/stop ZooKeeper on.
</description>
</property>
</configuration>
5 修改regionservers
添加从节点名称
node29
node30
6 用hadoop的hadoop-0.20.2-core.jar包替换hbase的lib下的hadoop包,不然会报版本错误
7 配置好后将hbase文件发送到其他从节点上
8 在主节点命令行输入start-hbase.sh
启动hbase
输入hbase shell
进入hbase shell界面
输入hbase命令查看是否命令可正常执行。如create ‘test',’cf‘
打开打开浏览器查看node31:60010页是否正常
1、hbase建表报错
12/08/18 15:50:02 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
12/08/18 15:50:02 WARN zookeeper.ZKUtil: hconnection-0x139386cb37b0003 Unable to set watcher on znode (/hbase)
查看日志有
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy interface org.apache.hadoop.hbase.ipc.HRegionInterface to localhost/127.0.0.1:52955 after attempts=1
修改/etc/hosts
127.0.1.1这行注释掉
2、其实hbase经常报错
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy interface org.apache.hadoop.hbase.ipc.HRegionInterface to localhost/127.0.0.1:52955 after attempts=1
总之都是/etc/hosts的问题
我最终只好在/etc/hosts中只保留集群用到的三个ip和对应名称
其余像127.0.0.1等等都删掉了
3、浏览器node31:60010如下显示
You are currently running the HMaster without HDFS append support enabled. This may result in data loss. Please see theHBase wiki for details.
需要添加hadoop-append包可参看http://www.cnblogs.com/ylqmf/archive/2012/02/28/2371669.html