TigerGraph 基准测试指导文档
本文详细说明了如何在 tiger 3.1.0
上完成图数据库基准测试项。
1. 安装Tiger
PS:以下安装步骤只需在m1节点
上进行操作
安装详情查阅tiger官方文档
-
下载安装包
wget https://dl.tigergraph.com/enterprise-edition/tigergraph-3.1.0-offline.tar.gz
-
解压安装包,解压完成之后会生成一个
tigergraph-3.1.0-offline
目录,进入该目录tar -zxvf tigergraph-3.1.0-offline.tar.gz cd tigergraph-3.1.0-offline
-
修改集群配置文件
install_conf.json
# 将注释的配置按照注释进行修改,其他配置不变 { "BasicConfig": { "TigerGraph": { "Username": "tigergraph", # Tiger管理员账号 "[comment]":"Provide password for tigergraph user, if the user already exists, we won't change the password. If the password is empty, we will set it to default value 'tigergraph'.", "Password": "tigergraph", # Tiger管理员密码 "SSHPort": 22, "[comment]":"(Optional)Provide valid private key file below to replace tigergraph.rsa and tigergraph.pub, which will be generated by default.", "PrivateKeyFile": "", "PublicKeyFile": "" }, "RootDir": { # Tiger存储信息的几个目录,最好在同一个目录下 "AppRoot": "/home/tigergraph/tigergraph/app", "DataRoot": "/home/tigergraph/tigergraph/data", "LogRoot": "/home/tigergraph/tigergraph/log", "TempRoot": "/home/tigergraph/tigergraph/tmp" }, "License": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJJc3N1ZXIiOiJUaWdlckdyYXBoIEluYy4iLCJBdWRpZW5jZSI6IlRpZ2VyR3JhcGggRnJlZSIsIlN0YXJ0VGltZSI6MTYwNzA0MDUzMiwiRW5kVGltZSI6MTY0MTI1ODUzMiwiSXNzdWVUaW1lIjoxNjA3MDQ0MTMyLCJFZGl0aW9uIjoiRW50ZXJwcmlzZSIsIlZlcnNpb24iOiJBbGwiLCJIb3N0Ijp7Ik1heENQVUNvcmUiOjEwMDAwMDAwMDAwMDAwMDAsIk1heFBoeXNpY2FsTWVtb3J5Qnl0ZXMiOjEwMDAwMDAwMDAwMDAwMDAsIk1heENsdXN0ZXJOb2RlTnVtYmVyIjoxMDI0fSwiVG9wb2xvZ3kiOnsiTWF4VmVydGV4TnVtYmVyIjoxMDAwMDAwMDAwMDAwMDAwLCJNYXhFZGdlTnVtYmVyIjoxMDAwMDAwMDAwMDAwMDAwLCJNYXhHcmFwaE51bWJlciI6MTAyNCwiTWF4VG9wb2xvZ3lCeXRlcyI6NTM2ODcwNjM3MTJ9LCJHU1QiOnsiRW5hYmxlIjp0cnVlLCJab29tQ2hhcnRzTGljZW5zZSI6IntcbiAgXCJsaWNlbnNlXCI6IFwiWkNCLTRtZmk3NDRsdjogWm9vbUNoYXJ0cyBFbnRlcnByaXNlIGxpY2VuY2UgZm9yIFRpZ2VyR3JhcGggZm9yIG9mZmxpbmUgdXNlOyB1cGdyYWRlcyB1bnRpbDogMjAyMi0xMi0zMVwiLFxuICBcImxpY2Vuc2VLZXlcIjogXCI4Zjg2ZWQzM2Y0ZWJmYmE4YTI4NjlkZGUzMmYzYTMwZGI4NGEzOTgxYmEzNmZhZWZlYTMxNDhhYzY4MzkxZTlhYzUzZDU3YmI5MjdmNjY5YWI1ZWJhYzJhYmQ3YTFkNDBiM2UxNDRmZjIwMDYyZWNiZmIwZjJiN2I0ZWFmZWIwYzU2NTc2NjBhZGExZDc1MWNhODU3NWZhYTE1ZWQwODI0NzkwZWQxMjJkY2Q4NjcyZTJiN2QzN2MwNmE3MzFhYTc2MDIwM2FhYmMwYjYzOWEzMjBhOGQxNmI0YmFiNGY5NTJiNTMwOTUzMWI4MDkxNjYwZDVjOGMzNGY0NmMyNjZiOGZiNzc2YzFmN2Y0MTdlMGQ5Y2JkZGFlOTExNTFlY2Y3YmMzZDlkNDgyNWE2MjAwYzk0MWMyMDE4ZDY4YjkyOWE5ZWY2MzQ2MDE5NjFhYmU1MGI0ZTk0ZmY5Y2VjMjA1ODEwYmVlZmRkM2NlZDU2YjM5NTVjYmE0YWIyMGNiNzc5MWE0NmQxNzIwMzNiZmI0ZDIyMDM4ODZhZTllZDFkOWMzOWIyOTM2ODc3Yjc2NzY4ZjQwNWQ5Y2MwY2JlODVjOTE2NDllMDI5YTA0NDFlOGFmYjA2MzY4MTMxZGM1YTc1NDEyOTc1NjFlMDRlMGM1MzE1ZmFjMDdhYzViOWViM1wiXG59In0sIlJ1bnRpbWVNZW1vcnkiOnsiTWF4VXNlclJlc2lkZW50U2V0Qnl0ZXMiOjEwMDAwMDAwMDAwMDAwMDB9fQ.qiaDEdFtJ8Mpzy_QklRJjA8eMV5TrKfHBQiOewdV-A3j8NiI8cuZqcgTkVzz-3Y6OVVWJgd4Fx1b7M5OgRQrQpCWbCt3-xVMuOZ8BqUWWsRa-nM-ZqJq5Nbxtfkde-xGw683ScsrTLEiUkUukD0082AziYfbsP52Us6vKb8mXuCTioHfnMQP3gduHXhY2eWRrKzRPQC5kxqFZUIY3ZzbuITbYcpK9MkioTliDFtXSMVSFn5KBjGGwN7OTKhgAZ6dmzW-6oPaw-IA8cCGJNhJQXADZf7ueHC_pVRVYTsrJSgFVhGBcwwspFbOJAwPIjUBCXGAsqt5PXC-zI_Kd5Q2yw", "[comment]":"You can add more nodes by string 'node_id: IP', appending to the following json array. Otherwise, it installs single node locally by default.", "NodeList": [ # Tiger集群节点的IP "m1: 192.168.40.104", "m2: 192.168.40.115", "m3: 192.168.40.116" ] }, "AdvancedConfig": { "[comment]": "Keep the default ClusterConfig if installing locally", "ClusterConfig": { "[comment]": "All nodes must have the same login configurations", "LoginConfig": { "SudoUser": "flf", # Tiger集群节点的账号,要求所有节点的账号相同并且有sudo权限 "[comment]": "choose login method: 'P' for SSH using password or 'K' for SSH using key file (e.g. ec2_key.pem)", "Method": "P", # 改为P表示用密码方式进行SSH连接 "P": "123456", # Tiger集群节点的密码 "K": "/path/to/my_key.pem_rsa" }, "[comment]": "To install a high-availability cluster, please specify the ReplicationFactor greater than 1", "ReplicationFactor": 1 # 复制因子,为1时数据分片存储,为节点数量时表示高可用,测试值改为1 } } }
-
执行安装脚本安装tiger
# 安装Tiger之前先在所有节点上安装sshpass sudo apt-get install sshpass # 安装Tiger sudo ./install.sh -n
-
切换到
tigergraph
用户修改请求响应超时时间# 切换到 tigergraph 用户 su tigergraph # 将超时全部修改为 3600(1小时) gadmin config set RESTPP.Factory.DefaultQueryTimeoutSec 7200 # 设置配置生效 gadmin config apply # 重启tiger gadmin restart all
2. 导入数据
2.1 Graph500
-
下载数据集
wget https://www.galaxybase.com/public/download/graph500.zip
-
解压数据集
unzip graph500.zip
-
移入指定目录
# 将数据移动到指定目录 mv graph500 /home/tigergraph/data/
-
将
Graph500/gsql
目录下的graph500.gsql
文件上传到服务器中的/home/tigergraph/data/graph500/
目录下 -
切换到
tigergraph
用户,运行gsql文件
导入数据和安装查询gsql graph500.gsql
2.2 Twitter-2010
-
下载数据集
wget https://www.galaxybase.com/public/download/twitter2010.zip
-
解压数据集
unzip twitter2010.zip
-
移入指定目录
# 将数据移动到指定目录 mv twitter2010 /home/tigergraph/data/
-
将
Twitter-2010/gsql
目录下的twitter2010.gsql
文件上传到服务器中的/home/tigergraph/data/twitter2010/
目录下 -
运行
gsql文件
导入数据和安装查询gsql twitter2010.gsql
3. 运行测试脚本
3.1 打包测试项目
-
为保证能够整体打包完成,切换到
galaxybase-release
目录,运行以下语句,将lib
下的galaxybase-bolt-driver-3.0.1.jar
添加到本地仓库中mvn install:install-file -DgroupId=com.galaxybase -DartifactId=bolt-driver -Dversion=3.0.1 -Dpackaging=jar -Dfile=lib\galaxybase-bolt-driver-3.0.1.jar
-
cd
至项目根目录,运行mvn package
,获得 jar 包:benchmark-tiger-1.0.0-SNAPSHOT-jar-with-dependencies.jar
mvn package
3.2 执行测试
3.2.1 Graph500
-
将
tiger
模块下的测试jar包benchmark-tiger-1.0.0-SNAPSHOT-jar-with-dependencies.jar
移动到Graph500
目录mv benchmark-tiger-1.0.0-SNAPSHOT-jar-with-dependencies.jar Graph500/
-
将
application.yaml
中的 IP 改为部署Tiger集群的m1节点的IP -
使用
java -jar benchmark-tiger-1.0.0-SNAPSHOT-jar-with-dependencies.jar
运行测试程序,运行结束后当前目录下会生成4个日志文件,为本次测试的结果。
3.2.2 Twitter-2010
-
将测试jar包
benchmark-tiger-1.0.0-SNAPSHOT-jar-with-dependencies.jar
移动到Twitter-2010
目录mv benchmark-tiger-1.0.0-SNAPSHOT-jar-with-dependencies.jar Twitter-2010/
-
将
application.yaml
中的 IP 改为部署Tiger集群的m1节点的IP -
使用
java -jar benchmark-tiger-1.0.0-SNAPSHOT-jar-with-dependencies.jar
运行测试程序,运行结束后当前目录下会生成4个日志文件,为本次测试的结果。
application.yaml文件是本项目的配置文件,可以指定图名称、样本集、测试项以及测试次数等,其配置文档详见根项目下的README.md