一、安装包下载
下载地址:https://github.com/JanusGraph/janusgraph/releases/
选择对应Hbase版本下载(公司Hbase版本1.4.11,我选择了0.3.3版本)
注:高版本客户端连接低版本Hbase会报错,报错会详细再说
二、上传至服务器
上传到要部署服务的机器
三、文件解压
mkdir /opt/janusgraph/
mv ~/wangtianyu03/janusgraph-0.3.3.tar.gz ../opt/janusgraph/
cd /opt/janusgraph/
tar -zxvf janusgraph-0.3.3.tar.gz
四、修改配置文件
cd janusgraph-0.3.3/conf
cp janusgraph-hbase-es.properties http-janusgraph-hbase-es.properties
cp gremlin-server/gremlin-server.yaml gremlin-server/http-gremlin-server.yaml
1.修改 http-janusgraph-hbase-es.properties
确认这些属性是否存在,不在则需要添加
storage.batch-loading=true
storage.backend=hbase #使用HBase作为存储后端
storage.hostname= #HBase的Zookeeper地址
storage.hbase.table=tablename #HBase存储JanusGraph元数据的表名
storage.hbase.ext.hbase.zookeeper.property.clientPort=2181
index.search.backend=elasticsearch #ES作为索引
index.search.hostname=test01.es.58dns.org,test02.es.58dns.org,test03.es.58dns.org #es服务ip。
index.search.port=9299 #es服务端口号
index.search.elasticsearch.http.auth.basic.username=test_user #es用户名
index.search.elasticsearch.http.auth.basic.password=test_pwd
index.search.elasticsearch.http.auth.type=basic
2.修改http-gremlin-server.yaml
(1)设置为http连接: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer
(2)更改配置文件路径为自己复制编辑的配置文件
3.启动server
需指定配置文件启动
./bin/gremlin-server.sh conf/gremlin-server/http-gremlin-server.yaml
4.验证是否启动成功
curl -XPOST -Hcontent-type:application/json -d '{"gremlin":"g.V().count()"}' http://localhost:8182
五、一些报错的坑
1.版本问题
(1)报错信息
高版本客户端查询低版本Hbase导致初始化失败
2288 [main] WARN org.janusgraph.diskstorage.hbase.HBaseStoreManager - Unexpected exception during getDeployment()
java.lang.RuntimeException: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend
at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getDeployment(HBaseStoreManager.java:379)
at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getFeatures(HBaseStoreManager.java:418)
at org.janusgraph.graphdb.configuration.builder.GraphDatabaseConfigurationBuilder.build(GraphDatabaseConfigurationBuilder.java:51)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:161)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:132)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:78)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:70)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:104)
at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.lambda$new$0(DefaultGraphManager.java:57)
at java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:671)
at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.(DefaultGraphManager.java:55)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.(ServerGremlinExecutor.java:80)
......
Caused by: org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column family table does not exist in region hbase:meta,,1.1588230740 in table 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}, {NAME => 'info', BLOOMFILTER => 'NONE', VERSIONS => '3', IN_MEMORY => 'true', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', CACHE_DATA_IN_L1 => 'true', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}
at org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:8371)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:7363)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2259)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36609)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2380)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:99)
at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:89)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:282)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:269)
at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:129)
at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:386)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:360)
at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1078)
at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:403)
at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:450)
at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:447)
at org.apache.hadoop.hbase.client.RpcRetryingCallable.call(RpcRetryingCallable.java:58)
at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3089)
at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3081)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:447)
at org.janusgraph.diskstorage.hbase.HBaseAdmin1_0.tableExists(HBaseAdmin1_0.java:111)
at org.janusgraph.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:725)
(2)解决办法
只需要找到对应的版本重覆上面步骤,done。。
2. Hbase zookeeper.znode.parent 属性
(1)报错信息
Hbase的配置文件zookeeper.znode.parent
与 janusgraph 的 properties文件配置不一致 ,不能读取Hbase信息(Hbase默认为/hbase)
1577 [main] WARN org.apache.hadoop.hbase.client.ConnectionImplementation - Retrieve cluster id failed
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.shaded.org.apache.zookeeper.KeeperException$NoNodeException
: KeeperErrorCode = NoNode for /hbase-unsecure/hbaseid
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:549)
at org.apache.hadoop.hbase.client.ConnectionImplementation.<init>(ConnectionImplementation.java:287)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:220)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:115)
at org.janusgraph.diskstorage.hbase.HBaseCompat1_0.createConnection(HBaseCompat1_0.java:43)
at org.janusgraph.diskstorage.hbase.HBaseStoreManager.<init>(HBaseStoreManager.java:336)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:440)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:411)
at org.janusgraph.graphdb.configuration.builder.GraphDatabaseConfigurationBuilder.build(GraphDatabaseConfigurationBu
ilder.java:50)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:161)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:132)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:77)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:69)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:103)
at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.lambda$new$0(DefaultGraphManager.java:57)
at java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:671)
at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.<init>(DefaultGraphManager.java:55)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.<init>(DefaultGraphManager.java:55)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:80)
at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:122)
at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:86)
at org.apache.tinkerpop.gremlin.server.GremlinServer.main(GremlinServer.java:345)
Caused by: org.apache.hadoop.hbase.shaded.org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /hba
se-unsecure/hbaseid
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:177)
at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:342)
at java.lang.Thread.run(Thread.java:748)
5899 [main] INFO org.apache.hadoop.hbase.client.RpcRetryingCallerImpl - Call exception, tries=6, retries=16, started=4151 ms ag
o, cancelled=false, msg=org.apache.hadoop.hbase.shaded.org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = No
Node for /hbase-unsecure, details=row 'hdp_lbg_ectech:lm_app_luna_intelligentpath' on table 'hbase:meta' at null, see https://s.a
pache.org/timeout
9931 [main] INFO org.apache.hadoop.hbase.client.RpcRetryingCallerImpl - Call exception, tries=7, retries=16, started=8183 ms ag
o, cancelled=false, msg=org.apache.hadoop.hbase.shaded.org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = No
Node for /hbase-unsecure, details=row 'hdp_lbg_ectech:lm_app_luna_intelligentpath' on table 'hbase:meta' at null, see https://s.a
pache.org/timeout
19949 [main] INFO org.apache.hadoop.hbase.client.RpcRetryingCallerImpl - Call exception, tries=8, retries=16, started=18201 ms
ago, cancelled=false, msg=org.apache.hadoop.hbase.shaded.org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
NoNode for /hbase-unsecure, details=row 'hdp_lbg_ectech:lm_app_luna_intelligentpath' on table 'hbase:meta' at null, see https://s
.apache.org/timeout
(2)解决方案:
http-janusgraph-hbase-es.properties 文件增加参数
storage.hbase.ext.zookeeper.znode.parent=/hbase
六、API调试
public static void main(String[] args) {
JanusGraphFactory.Builder builder = JanusGraphFactory.build()
.set("storage.hostname", "tjtx162-0-201.58os.org,tjtx162-1-111.58os.org,tjtx162-1-213.58os.org,tjtx162-1-42.58os.org,tjtx162-1-77.58os.org")
.set("storage.hbase.ext.zookeeper.znode.parent","/hbase")
.set("storage.backend", "hbase")
.set("storage.hbase.table", "tablename");
JanusGraph graph = null;
try{
graph = builder.open();
boolean open = graph.isOpen();
System.out.println("is open:" + open);
} catch (Exception e){
System.out.println(e);
} finally {
if(null != graph){
graph.close();
}
}
}
结果:
连接获取成功
七、附赠一份server监控重启的shell脚本
#! /bin/bash
monitor=`ps -ef | grep GremlinServer | grep -v grep | wc -l `
if [ $monitor -eq 0 ]
then
echo "Manipulator program is not running, restart Manipulator"
sh /opt/janusgraph/janusgraph-0.3.3/bin/gremlin-server.sh conf/gremlin-server/http-gremlin-server.yaml > /opt/janusgraph/janusgraph-
0.3.3/log/janus_server.log 2>&1 &
else
echo "Manipulator program is running"
fi
#crontab -e 编辑定时任务,每五分钟执行一次
*/5 * * * * nohup sh /home/work/wangtianyu03/restart.sh > /home/work/wangtianyu03/restart.log 2>&1