HugeGraph 添加kerberos认证Hbase backend

背景

最近流行图计算，公司也正好推进使用，正好趁着这个机会学习一波。不过，第一步搭建就遇到一个坑，官网不支持kerberos认证的hbase，捣鼓了一天，终于把hugeGraph服务给搭建完毕，正好分享一下过程。

线上环境

集群hbase版本：2.1.0
HugeGraph-Server 版本： 0.10.4

更改步骤

1、下载官网代码：
git 地址为：https://github.com/hugegraph/hugegraph/releases
2、在com.baidu.hugegraph.backend.store.hbase.HbaseOptions 加入以下方法

 public static final ConfigOption<Boolean> HBASE_HAVEKERBEROS =
        new ConfigOption<>(
            "hbase.kerberos",
            "The hbase have kerberos.",
            disallowEmpty(),
            false
        );
    public static final ConfigOption<String> HBASE_MASTER_PRINCIPAL =
        new ConfigOption<>(
            "hbase.master.kerberos.principal",
            "The hbase master kerberos principal.",
            null,
            ""
        );
    public static final ConfigOption<String> HBASE_REGION_PRINCIPAL =
        new ConfigOption<>(
            "hbase.regionserver.kerberos.principal",
            "The hbase regionserver kerberos principal.",
            null,
            ""
        );
    public static final ConfigOption<String> HBASE_PRINCIPAL =
        new ConfigOption<>(
            "hbase.principal",
            "The hbase principal .",
            null,
            ""
        );
    public static final ConfigOption<String> HBASE_PRINCIPAL_KEYTAB_PATH =
        new ConfigOption<>(
            "hbase.principal.keytab.path",
            "The hbase principal keytab path.",
            null,
            ""
        );

3、在com.baidu.hugegraph.backend.store.hbase.HbaseSessions中该下这个方法

public synchronized void open() throws IOException {
        LOG.info("HbaseSessions open");
        HugeConfig config = this.config();
        String hosts = config.get(HbaseOptions.HBASE_HOSTS);
        int port = config.get(HbaseOptions.HBASE_PORT);
        String znodeParent = config.get(HbaseOptions.HBASE_ZNODE_PARENT);

        Configuration hConfig = HBaseConfiguration.create();
        Boolean havekerberos = config.get(HbaseOptions.HBASE_HAVEKERBEROS);
        if(havekerberos){
            String master_principal = config.get(HbaseOptions.HBASE_MASTER_PRINCIPAL);
            String region_principal = config.get(HbaseOptions.HBASE_REGION_PRINCIPAL);
            String user = config.get(HbaseOptions.HBASE_PRINCIPAL);
            String keytab = config.get(HbaseOptions.HBASE_PRINCIPAL_KEYTAB_PATH);
            hConfig.set("hadoop.security.authentication", "kerberos"); // 启用kerberos
            hConfig.set("hbase.master.kerberos.principal",master_principal);
            hConfig.set("hbase.regionserver.kerberos.principal",region_principal);
            hConfig.set("hbase.security.authentication","kerberos");
            hConfig.set("hbase.zookeeper.property.clientPort", String.valueOf(port));
            hConfig.set("hbase.zookeeper.quorum", hosts);
            hConfig.set("zookeeper.znode.parent",znodeParent);

            LOG.info("master_principal:"+master_principal);
            UserGroupInformation.setConfiguration(hConfig);
            UserGroupInformation.loginUserFromKeytab(user, keytab);
        }else {

            hConfig.set(HConstants.ZOOKEEPER_QUORUM, hosts);
            hConfig.set(HConstants.ZOOKEEPER_CLIENT_PORT, String.valueOf(port));
            hConfig.set(HConstants.ZOOKEEPER_ZNODE_PARENT, znodeParent);
        }
        hConfig.setInt("zookeeper.recovery.retry",
                       config.get(HbaseOptions.HBASE_ZK_RETRY));

        // Set hbase.hconnection.threads.max 64 to avoid OOM(default value: 256)
        hConfig.setInt("hbase.hconnection.threads.max",
                       config.get(HbaseOptions.HBASE_THREADS_MAX));

        this.hbase = ConnectionFactory.createConnection(hConfig);
    }

4、重新编译打包放入服务器集群…/hugegraph-0.10.4/lib内，并删掉原jar包

 mvn clean install -Dmaven.test.skip=true

5、更改集群上hugegraph.properties文件

hbase.hosts=xxx
hbase.port=2181
hbase.znode_parent=/hbase
hbase.threads_max=64
hbase.kerberos=true
hbase.master.kerberos.principal=hbase/_HOST@xxx
hbase.regionserver.kerberos.principal=hbase/_HOST@xxx
hbase.principal=xxx
hbase.principal.keytab.path=xxx

6、修改初始化脚本 bin/init-store.sh
最后加上$JAVA_HOME/jre/lib/ext

exec $JAVA -cp $LIB/hugegraph-dist*.jar -Djava.ext.dirs=$JAVA_HOME/jre/lib/ext:$LIB:$PLUGINS \
com.baidu.hugegraph.cmd.InitStore $CONF/gremlin-server.yaml

7、初始化数据库、启动服务即可

后记

在部署完毕HugeGraph-Server之后，部署HugeGraph-Studio意外遇到Not allowed to access thread group via Gremlin这个坑，我注掉了相关core下的一些代码也没解决，后来通过测试HugeGraph-Client时，意外就不报错了，也是有点神奇的。

2020-05-29 17:52:27 790553 [gremlin-server-exec-8] [WARN ] org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler [] - Invalid request - responding with 500 Internal Server Error and Not allowed to access thread group via Gremlin
java.lang.SecurityException: Not allowed to access thread group via Gremlin