开源大数据集群部署（十九）Hbase部署

作者：櫰木

1 HBASE 安装部署

hbase组件	部署主机
HMaster	hd1.dtstack.com,hd2.dtstack.com
HRegionServer	hd3.dtstack.com,hd2.dtstack.com,hd1.dtstack.com

2 创建hbase Kerberos主体

在每台机器上进行生成

bash /data/kerberos/getkeytabs.sh /etc/security/keytab/hbase.keytab hbase
bash /data/kerberos/getkeytabs.sh /etc/security/keytab/hbase.keytab HTTP

3 安装

在hd1.dtstack.com主机root权限下执行

• 解压安装包

[root@hd1.dtstack.com software]# pwd
[root@hd1.dtstack.com software]#tar -zvxf hbase-2.1.0-bin.tar.gz

• 设置环境变量

[root@hd3.dtstack.com software]# cat  >>/etc/profile<<EOF
export HBASE_HOME=/opt/hbase
export HBASE_CONF_DIR=/opt/hbase/conf
EOF
[root@hd3.dtstack.com software]# source /etc/profile

• 修改配置文件

注意，在hadoop开启kerberos情况下，需要在core-site.xml添加以下参数

cat /opt/hadoop/etc/hadoop/core-site.xml
    <property>
      <name>hadoop.rpc.protection</name>
      <value>authentication,privacy</value>
</property>
在hdfs-site.xml中修改和添加以下参数
cat /opt/hadoop/etc/hadoop/core-site.xml
   <property>
        <name>dfs.data.transfer.protection</name>
        <value>authentication,privacy</value>
    </property>
    <property>
      <name>dfs.encrypt.data.transfer.cipher.suites</name>
      <value>AES/CTR/NoPadding</value>
    </property>

如果你的Hadoop已经有HA，或者你已经有一个独立的ZooKeeper集群，那么你就需要在hbase-env.sh中把HBase自带的ZooKeeper关掉以防止端口冲突

[root@hd1.dtstack.com conf]# cat >hbase-env.sh<<EOF
HBASE_ROOT_LOGGER=INFO,DRFA
export JAVA_HOME=/opt/java
export HBASE_MANAGES_ZK=false
#export HBASE_OPTS="-XX:+UseConcMarkSweepGC"
export HBASE_OPTS="$HBASE_OPTS -Djava.security.auth.login.config=/opt/hbase/conf/zk-jaas-client.conf -Dzookeeper.sasl.client=true -Dzookeeper.sasl.client.username=zookeeper -Dzookeeper.sasl.clientconfig=Client"
export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -Djava.security.auth.login.config=/opt/hbase/conf/zk-jaas.conf  -Djavax.security.auth.useSubjectCredsOnly=false -javaagent:/opt/prometheus/jmx_prometheus_javaagent-0.3.1.jar=9530:/opt/prometheus/hbase_master.yml"
export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Djava.security.auth.login.config=/opt/hbase/conf/zk-jaas.conf  -Djavax.security.auth.useSubjectCredsOnly=false -javaagent:/opt/prometheus/jmx_prometheus_javaagent-0.12.0.jar=9531:/opt/prometheus/hbase_regionserver.yml"
EOF
cat >zk-jaas-client.conf<<EOF
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=false
useTicketCache=true;
};
EOF
cat >zk-jaas.conf<<EOF
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
useTicketCache=false
keyTab="/etc/security/keytab/hbase.keytab"
principal="hbase/hd1.dtstack.com@DTSTACK.COM";
};
com.sun.security.jgss.krb5.initiate {
com.sun.security.auth.module.Krb5LoginModule required
renewTGT=false
doNotPrompt=true
useKeyTab=true
storeKey=true
useTicketCache=false
keyTab="/etc/security/keytab/hbase.keytab"
principal ="hbase/hd1.dtstack.com@DTSTACK.COM";
};
EOF

说明：

• 启动参数值可根据具体情况调整

修改hbase-site.xml,内容如下：

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
-->
<configuration>
  <!--
    The following properties are set