按照官网的步骤来,一步一个坑,浪费了我一天的时间!多数是因为国内的网络,下载和拉取速度太慢,导致安装失败。所以写了这个文章,来记录我成功安装的案例,希望可以帮到想用esrally做基准测试的人!
网上也有文章不过都过时了!官网呢,又不适合中国网络情况的使用。我甚至严重怀疑,官网的步骤,他们有没有执行一遍试试。因为用哪些命令肯定会遇到问题。比如:
sudo yum install git我不相信这条命令,能帮我们下载新版本的git!
注意目前esrally最新版本是 2.5.0!
官网地址(英文的最新版):Quickstart — Rally 2.5.0 documentation
中文文档(0.9版本的):安装 — Rally 0.9.0 文档
准备环境-需要的环境介绍
我在看文档的时候,其实蛮想知道我为什么要安装这些东西,以及不安装行不行,我需要安装什么版本的。我在看着官网折腾这个的时候浪费了挺多时间,所以特地写一篇博客来带大家避坑!
操作系统 centos7,我是在centos7下安装的,其它版本,操作系统不适用,其它版本我没有验证能否成功。esrally只能运行在linux下!
安装过程需要网络环境。即使有网络,在装一些东西的时候依旧有很多问题,所以我在安装一些东西的时候,也会用安装包安装,来减少问题的出现。
另外说一句,esrally文档是写给外国人的,因为一些网络环境我们不具备,所以根据文档操作,一大堆问题。
想要运行esrally必备的工具
- python3.8以上版本,我看官网的例子是用python3.8.10的版本。为了减少问题,我也用python3.8.10。说一下为什么要用它,因为esrally是用python写的,安装也用到了pip3。这些都属于必备的!
- git,版本1.9及以上版本。用git是因为es用到的数据都放在了网上,用到的测试用例也放在git上,所以需要我们在用的时候拉取。这个也是必须的,没有不行!
- JDK。用这个是因为,esrally支持自动构建es集群,就是帮你搭建集群。es低版本(7版本之前)并没有内嵌JDK,所以这些版本需要JDK,并配置好环境。如果你是自建集群,对已有的集群进行测试,那就不需要JDK了。如果需要它帮你搭建集群,你就需要将JDK环境配置好。
- esrally,es基准测试的主角。
如何在linux centos下安装python环境
这篇文章是我在用esrally的时候写的,我跟着官网做的时候遇到很多坑,主要是浪费了时间,实在没必要。我的这个方案更简单!请看我这篇文章:
centos下编译安装各个版本的python_水的精神的博客-CSDN博客
如何升级git,或者说安装git
centos7默认安装了git,遗憾的是它的版本是1.8的。也就是低于官网要求的1.9版本。所以需要升级。在网上找文档折腾的半天,都不靠谱。所以建议看我这篇文章(特简单,三条命令):
Linux centos下三条命令安装升级git_水的精神的博客-CSDN博客
如何安装JDK
下载安装jdk13
https://www.oracle.com/java/technologies/javase-jdk13-downloads.html
复制
下载,上传到服务器上。
然后rpm安装
rpm -ivh jdk-13.0.2_linux-x64_bin.rpm
配置环境变量,在 /etc/profile文件的末尾加入以下内容
export JAVA_HOME=/usr/java/jdk-13.0.2
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
并执行下列命令使环境变量生效
source /etc/profile
安装esrally
官网给的操作是使用一条命令安装,开起来很简单:
pip3 install esrally
但是我执行不成功。可能我对python不是很熟悉的原因把。我用的其它方式来安装的(离线安装 esrally 是最靠谱的,解决拉取问题!我在尝试上边的命令的时候,遇到了一堆问题。不想解决了!所以用离线的方式来安装它):
- 安装上面文档提到的所有依赖包
- 下载 最新发布的离线包 ,并将它拷贝到目标机器。
- 解压安装包
tar -xzf esrally-dist-*.tar.gz
。 - 执行安装脚本
sudo ./esrally-dist-*/install.sh
。
在执行第四条的时候,还是遇到了问题(我把遇到几个问题贴出来,希望可以帮到你们):
执行 ./install.sh 第一个问题:
ERROR: Could not find a version that satisfies the requirement py-cpuinfo==7.0.0 (from esrally) (from versions: 7.0.0)
ERROR: No matching distribution found for py-cpuinfo==7.0.0
解决命令:
pip3 install py-cpuinfo -i https://pypi.tuna.tsinghua.edu.cn/simple/ --trusted-host pypi.tuna.tsinghua.edu.cn
执行 ./install.sh 第二个问题(python pip相关的问题):
问题
WARNING: You are using pip version 21.1.1; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/local/python38/bin/python3.8 -m pip install --upgrade pip' command.
解决命令:
python3 -m pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple/ --trusted-host pypi.tuna.tsinghua.edu.cn
第三个问题:
解决问题:
pip3 install --upgrade setuptools -i https://pypi.tuna.tsinghua.edu.cn/simple/ --trusted-host pypi.tuna.tsinghua.edu.cn
以上三个问题解决完以后,esrally已经可以成功安装了!
尝试运行一下,因为我已经安装过了es集群。所以已经可以使用了!
命令如下:
esrally race --track=percolator --target-hosts=10.10.10.10:9200,10.10.10.10:9201,10.10.10.10:9202 --pipeline=benchmark-only
如果你没有建好的集群,尝试使用这个命令来运行(这个会帮你创建一个es集群,耐心等待,时间比较长):
esrally --distribution-version=6.0.0
我的案例运行结果如下 :
[root@estest esrally-track]# esrally race --track=percolator --target-hosts=10.10.10.10:9200,10.10.10.10:9201,10.10.10.10:9202 --pipeline=benchmark-only
____ ____
/ __ \____ _/ / /_ __
/ /_/ / __ `/ / / / / /
/ _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
/____/
[INFO] Race id is [7f280f60-6b48-41db-9ab3-f6977170c417]
[WARNING] Could not update tracks. Continuing with your locally available state.
[WARNING] Could not update tracks. Continuing with your locally available state.
[INFO] Downloading track data (121.1 kB total size) [100.0%]
[INFO] Decompressing track data from [/root/.rally/benchmarks/data/percolator/queries-2.json.bz2] to [/root/.rally/benchmarks/data/percolator/queries-2.json] (resulting size: [0.1... [OK]
[INFO] Preparing file offset table for [/root/.rally/benchmarks/data/percolator/queries-2.json] ... [OK]
[INFO] Racing on track [percolator], challenge [append-no-conflicts] and car ['external'] with version [6.8.0].
[WARNING] merges_total_time is 8107257 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] merges_total_throttled_time is 1274256 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] indexing_total_time is 79966971 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] refresh_total_time is 3806308 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] flush_total_time is 522750 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
Running delete-index [100% done]
Running create-index [100% done]
Running check-cluster-health [100% done]
Running index [100% done]
Running refresh-after-index [100% done]
Running force-merge [100% done]
Running refresh-after-force-merge [100% done]
Running wait-until-merges-finish [100% done]
Running percolator_with_content_president_bush [100% done]
Running percolator_with_content_saddam_hussein [100% done]
Running percolator_with_content_hurricane_katrina [100% done]
Running percolator_with_content_google [100% done]
Running percolator_no_score_with_content_google [100% done]
Running percolator_with_highlighting [100% done]
Running percolator_with_content_ignore_me [100% done]
Running percolator_no_score_with_content_ignore_me [100% done]
------------------------------------------------------
_______ __ _____
/ ____(_)___ ____ _/ / / ___/_________ ________
/ /_ / / __ \/ __ `/ / \__ \/ ___/ __ \/ ___/ _ \
/ __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/
/_/ /_/_/ /_/\__,_/_/ /____/\___/\____/_/ \___/
------------------------------------------------------
| Metric | Task | Value | Unit |
|---------------------------------------------------------------:|-------------------------------------------:|--------------:|-------:|
| Cumulative indexing time of primary shards | | 1334.87 | min |
| Min cumulative indexing time across primary shards | | 0.39835 | min |
| Median cumulative indexing time across primary shards | | 15.4366 | min |
| Max cumulative indexing time across primary shards | | 26.0545 | min |
| Cumulative indexing throttle time of primary shards | | 0 | min |
| Min cumulative indexing throttle time across primary shards | | 0 | min |
| Median cumulative indexing throttle time across primary shards | | 0 | min |
| Max cumulative indexing throttle time across primary shards | | 0 | min |
| Cumulative merge time of primary shards | | 135.302 | min |
| Cumulative merge count of primary shards | | 805 | |
| Min cumulative merge time across primary shards | | 0.03545 | min |
| Median cumulative merge time across primary shards | | 1.25905 | min |
| Max cumulative merge time across primary shards | | 8.2117 | min |
| Cumulative merge throttle time of primary shards | | 21.2376 | min |
| Min cumulative merge throttle time across primary shards | | 0 | min |
| Median cumulative merge throttle time across primary shards | | 0 | min |
| Max cumulative merge throttle time across primary shards | | 3.46082 | min |
| Cumulative refresh time of primary shards | | 63.4783 | min |
| Cumulative refresh count of primary shards | | 4067 | |
| Min cumulative refresh time across primary shards | | 0.00725 | min |
| Median cumulative refresh time across primary shards | | 0.758333 | min |
| Max cumulative refresh time across primary shards | | 1.46298 | min |
| Cumulative flush time of primary shards | | 8.7125 | min |
| Cumulative flush count of primary shards | | 233 | |
| Min cumulative flush time across primary shards | | 0 | min |
| Median cumulative flush time across primary shards | | 0.0939167 | min |
| Max cumulative flush time across primary shards | | 0.46015 | min |
| Total Young Gen GC time | | 11.337 | s |
| Total Young Gen GC count | | 777 | |
| Total Old Gen GC time | | 0 | s |
| Total Old Gen GC count | | 0 | |
| Store size | | 84.4625 | GB |
| Translog size | | 2.95658 | GB |
| Heap used for segments | | 172.674 | MB |
| Heap used for doc values | | 0.014431 | MB |
| Heap used for terms | | 141.275 | MB |
| Heap used for norms | | 0.038147 | MB |
| Heap used for points | | 7.70514 | MB |
| Heap used for stored fields | | 23.6414 | MB |
| Segment count | | 85 | |
| Total Ingest Pipeline count | | 0 | |
| Total Ingest Pipeline time | | 0 | s |
| Total Ingest Pipeline failed | | 0 | |
| error rate | index | 0 | % |
| Min Throughput | percolator_with_content_president_bush | 47.9 | ops/s |
| Mean Throughput | percolator_with_content_president_bush | 48.16 | ops/s |
| Median Throughput | percolator_with_content_president_bush | 48.16 | ops/s |
| Max Throughput | percolator_with_content_president_bush | 48.43 | ops/s |
| 50th percentile latency | percolator_with_content_president_bush | 10.2912 | ms |
| 90th percentile latency | percolator_with_content_president_bush | 13.3935 | ms |
| 99th percentile latency | percolator_with_content_president_bush | 16.1495 | ms |
| 100th percentile latency | percolator_with_content_president_bush | 16.4619 | ms |
| 50th percentile service time | percolator_with_content_president_bush | 9.44676 | ms |
| 90th percentile service time | percolator_with_content_president_bush | 12.2333 | ms |
| 99th percentile service time | percolator_with_content_president_bush | 15.5729 | ms |
| 100th percentile service time | percolator_with_content_president_bush | 15.6444 | ms |
| error rate | percolator_with_content_president_bush | 0 | % |
| Min Throughput | percolator_with_content_saddam_hussein | 50.07 | ops/s |
| Mean Throughput | percolator_with_content_saddam_hussein | 50.07 | ops/s |
| Median Throughput | percolator_with_content_saddam_hussein | 50.07 | ops/s |
| Max Throughput | percolator_with_content_saddam_hussein | 50.08 | ops/s |
| 50th percentile latency | percolator_with_content_saddam_hussein | 3.68269 | ms |
| 90th percentile latency | percolator_with_content_saddam_hussein | 4.41383 | ms |
| 99th percentile latency | percolator_with_content_saddam_hussein | 8.45466 | ms |
| 100th percentile latency | percolator_with_content_saddam_hussein | 9.41373 | ms |
| 50th percentile service time | percolator_with_content_saddam_hussein | 2.81123 | ms |
| 90th percentile service time | percolator_with_content_saddam_hussein | 3.50136 | ms |
| 99th percentile service time | percolator_with_content_saddam_hussein | 7.73842 | ms |
| 100th percentile service time | percolator_with_content_saddam_hussein | 8.28083 | ms |
| error rate | percolator_with_content_saddam_hussein | 0 | % |
| Min Throughput | percolator_with_content_hurricane_katrina | 50.13 | ops/s |
| Mean Throughput | percolator_with_content_hurricane_katrina | 50.15 | ops/s |
| Median Throughput | percolator_with_content_hurricane_katrina | 50.15 | ops/s |
| Max Throughput | percolator_with_content_hurricane_katrina | 50.17 | ops/s |
| 50th percentile latency | percolator_with_content_hurricane_katrina | 8.31714 | ms |
| 90th percentile latency | percolator_with_content_hurricane_katrina | 9.77985 | ms |
| 99th percentile latency | percolator_with_content_hurricane_katrina | 11.9031 | ms |
| 100th percentile latency | percolator_with_content_hurricane_katrina | 16.0777 | ms |
| 50th percentile service time | percolator_with_content_hurricane_katrina | 6.9336 | ms |
| 90th percentile service time | percolator_with_content_hurricane_katrina | 8.12209 | ms |
| 99th percentile service time | percolator_with_content_hurricane_katrina | 11.4533 | ms |
| 100th percentile service time | percolator_with_content_hurricane_katrina | 14.9582 | ms |
| error rate | percolator_with_content_hurricane_katrina | 0 | % |
| Min Throughput | percolator_with_content_google | 26.57 | ops/s |
| Mean Throughput | percolator_with_content_google | 26.77 | ops/s |
| Median Throughput | percolator_with_content_google | 26.76 | ops/s |
| Max Throughput | percolator_with_content_google | 26.97 | ops/s |
| 50th percentile latency | percolator_with_content_google | 83.5241 | ms |
| 90th percentile latency | percolator_with_content_google | 104.656 | ms |
| 99th percentile latency | percolator_with_content_google | 115.239 | ms |
| 100th percentile latency | percolator_with_content_google | 119.399 | ms |
| 50th percentile service time | percolator_with_content_google | 34.601 | ms |
| 90th percentile service time | percolator_with_content_google | 44.8482 | ms |
| 99th percentile service time | percolator_with_content_google | 52.8977 | ms |
| 100th percentile service time | percolator_with_content_google | 53.925 | ms |
| error rate | percolator_with_content_google | 0 | % |
| Min Throughput | percolator_no_score_with_content_google | 99.79 | ops/s |
| Mean Throughput | percolator_no_score_with_content_google | 99.79 | ops/s |
| Median Throughput | percolator_no_score_with_content_google | 99.79 | ops/s |
| Max Throughput | percolator_no_score_with_content_google | 99.79 | ops/s |
| 50th percentile latency | percolator_no_score_with_content_google | 3.85806 | ms |
| 90th percentile latency | percolator_no_score_with_content_google | 4.32838 | ms |
| 99th percentile latency | percolator_no_score_with_content_google | 4.84981 | ms |
| 100th percentile latency | percolator_no_score_with_content_google | 5.66612 | ms |
| 50th percentile service time | percolator_no_score_with_content_google | 2.98046 | ms |
| 90th percentile service time | percolator_no_score_with_content_google | 3.37479 | ms |
| 99th percentile service time | percolator_no_score_with_content_google | 4.04749 | ms |
| 100th percentile service time | percolator_no_score_with_content_google | 5.13068 | ms |
| error rate | percolator_no_score_with_content_google | 0 | % |
| Min Throughput | percolator_with_highlighting | 48.79 | ops/s |
| Mean Throughput | percolator_with_highlighting | 48.94 | ops/s |
| Median Throughput | percolator_with_highlighting | 48.94 | ops/s |
| Max Throughput | percolator_with_highlighting | 49.09 | ops/s |
| 50th percentile latency | percolator_with_highlighting | 5.55708 | ms |
| 90th percentile latency | percolator_with_highlighting | 6.13534 | ms |
| 99th percentile latency | percolator_with_highlighting | 10.0739 | ms |
| 100th percentile latency | percolator_with_highlighting | 21.7479 | ms |
| 50th percentile service time | percolator_with_highlighting | 4.72898 | ms |
| 90th percentile service time | percolator_with_highlighting | 5.13776 | ms |
| 99th percentile service time | percolator_with_highlighting | 9.4044 | ms |
| 100th percentile service time | percolator_with_highlighting | 21.1273 | ms |
| error rate | percolator_with_highlighting | 0 | % |
| Min Throughput | percolator_with_content_ignore_me | 0.08 | ops/s |
| Mean Throughput | percolator_with_content_ignore_me | 0.08 | ops/s |
| Median Throughput | percolator_with_content_ignore_me | 0.08 | ops/s |
| Max Throughput | percolator_with_content_ignore_me | 0.08 | ops/s |
| 50th percentile latency | percolator_with_content_ignore_me | 10721.8 | ms |
| 90th percentile latency | percolator_with_content_ignore_me | 11339.3 | ms |
| 99th percentile latency | percolator_with_content_ignore_me | 11920.1 | ms |
| 100th percentile latency | percolator_with_content_ignore_me | 12114.7 | ms |
| 50th percentile service time | percolator_with_content_ignore_me | 10719.2 | ms |
| 90th percentile service time | percolator_with_content_ignore_me | 11337.3 | ms |
| 99th percentile service time | percolator_with_content_ignore_me | 11916.3 | ms |
| 100th percentile service time | percolator_with_content_ignore_me | 12112 | ms |
| error rate | percolator_with_content_ignore_me | 0 | % |
| Min Throughput | percolator_no_score_with_content_ignore_me | 15.03 | ops/s |
| Mean Throughput | percolator_no_score_with_content_ignore_me | 15.05 | ops/s |
| Median Throughput | percolator_no_score_with_content_ignore_me | 15.05 | ops/s |
| Max Throughput | percolator_no_score_with_content_ignore_me | 15.06 | ops/s |
| 50th percentile latency | percolator_no_score_with_content_ignore_me | 31.3156 | ms |
| 90th percentile latency | percolator_no_score_with_content_ignore_me | 50.9371 | ms |
| 99th percentile latency | percolator_no_score_with_content_ignore_me | 58.0967 | ms |
| 100th percentile latency | percolator_no_score_with_content_ignore_me | 58.1975 | ms |
| 50th percentile service time | percolator_no_score_with_content_ignore_me | 30.048 | ms |
| 90th percentile service time | percolator_no_score_with_content_ignore_me | 49.9852 | ms |
| 99th percentile service time | percolator_no_score_with_content_ignore_me | 57.0299 | ms |
| 100th percentile service time | percolator_no_score_with_content_ignore_me | 57.3463 | ms |
| error rate | percolator_no_score_with_content_ignore_me | 0 | % |
[WARNING] No throughput metrics available for [index]. Likely cause: The benchmark ended already during warmup.
----------------------------------
[INFO] SUCCESS (took 1691 seconds)
----------------------------------