前言
最近想将locust的测试结果用Grafana展示,网上找了很多方法,都不是很满意,于是去看了官方文档,找到一种方法实现,官方文档:https://github.com/SvenskaSpel/locust-plugins
一.安装TimescaleDB数据库
1.拉取TimescaleDB镜像
docker pull timescale/timescaledb:latest-pg12
2.运行TimescaleDB容器。POSTGRES_PASSWORD是数据库的密码,默认用户名为:postgres
docker run -d --name timescaledb -p 5432:5432 -v /opt/data:/var/lib/postgresql/data -e POSTGRES_PASSWORD=123456 timescale/timescaledb:latest-pg12
3.进入imescaleDB数据库容器
docker exec -it timescaledb /bin/bash
4.进入imescaleDB数据库
psql -U postgres -h localhost
5.创建一个数据库
CREATE database tutorial;
6.可以通过navicat连接数据
7.因为TimescaleDB数据库里的表是透明的,故无法通过工具查看到
8.可以通过查询的方式实现
select * from public.request;
select * from public.testrun;
select * from public.user_count;
select * from public.events;
二.导入数据表
将官方给的的sql文件导入tutorial数据库https://github.com/SvenskaSpel/locust-plugins/blob/master/locust_plugins/timescale_schema.sql
timescale_schema.sql
--
-- TIMESCALEDB init schema
-- tested with version 11
--
SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = off;
CREATE TABLE public.testrun (
id timestamp with time zone NOT NULL,
testplan text NOT NULL,
profile_name text,
num_clients integer NOT NULL,
rps double precision,
description text,
end_time timestamp with time zone,
env character varying(10) NOT NULL,
username character varying(64),
gitrepo character varying(120),
rps_avg numeric,
resp_time_avg numeric,
changeset_guid character varying(36),
fail_ratio double precision,
requests integer
);
ALTER TABLE public.testrun OWNER TO postgres;
--
-- Name: testrun testrun_pkey; Type: CONSTRAINT; Schema: public; Owner: postgres
--
ALTER TABLE ONLY public.testrun
ADD CONSTRAINT testrun_pkey PRIMARY KEY (id);
--
-- Name: testrun_id_idx; Type: INDEX; Schema: public; Owner: postgres
--
CREATE INDEX testrun_id_idx ON public.testrun USING btree (id DESC);
CREATE TABLE public.request (
"time" timestamp with time zone NOT NULL,
run_id timestamp with time zone NOT NULL,
exception text,
greenlet_id integer NOT NULL,
loadgen text NOT NULL,
name text NOT NULL,
request_type text NOT NULL,
response_length integer,
response_time double precision,
success smallint NOT NULL,
testplan character varying(30) NOT NULL,
pid integer,
context jsonb
);
ALTER TABLE public.request OWNER TO postgres;
--
-- Name: request_time_idx; Type: INDEX; Schema: public; Owner: postgres
--
CREATE INDEX request_time_idx ON public.request USING btree ("time" DESC);
--
-- Name: run_id_idx; Type: INDEX; Schema: public; Owner: postgres
--
CREATE INDEX run_id_idx ON public.request USING btree (run_id);
CREATE TABLE public.user_count (
testplan character varying(30) NOT NULL,
user_count integer NOT NULL,
"time" timestamp with time zone NOT NULL,
run_id timestamp with time zone
);
ALTER TABLE public.user_count OWNER TO postgres;
--
-- Name: user_count_time_idx; Type: INDEX; Schema: public; Owner: postgres
--
CREATE INDEX user_count_time_idx ON public.user_count USING btree ("time" DESC);
CREATE TABLE public.events (
time timestamp with time zone NOT NULL,
text text NOT NULL
);
SELECT create_hypertable('request', 'time');
SELECT create_hypertable('user_count', 'time');
SELECT create_hypertable('events', 'time');
三.编写locust脚本
备注:
1.这个是我封装过后的,用于记录的。可以跳过直接看后面的基础脚本。
2.此脚本只能在Linux下环境下运行,不然会报错。
3.若看不懂脚本的,可以先看官方例子https://github.com/SvenskaSpel/locust-plugins/blob/master/examples/timescale_listener_ex.py
from locust_plugins import listeners
from locust import TaskSet, constant_pacing
from locust.contrib.fasthttp import FastHttpUser
import jsonpath
from locust import events
import os
os.environ['PGHOST'] = '这里填TimescaleDB数据库IP'
os.environ['PGUSER'] = 'postgres'
os.environ['PGPASSWORD'] = '123456'
os.environ['PGDATABASE'] = 'tutorial'
token = '********************'
locust_data = {'environment': 'http://127.0.0.1:10000', 'task_name': 'locust_test', 'task': [
{'path': '/environment?name=&page=1&page_size=10', 'method': 'get', 'header': {"Authorization": token}, 'data': {},
'assert_key': 'msg', 'assert_value': 'success', 'task': 1}]}
class Example:
def __init__(self, path, method, header, parameter, assert_key, assert_value):
"""接口信息"""
self.path = path
self.method = method.upper()
self.header = header
self.parameter = parameter
self.assert_key = assert_key
self.assert_value = assert_value
def example_test(self, obj):
"""任务模板"""
if 'json' in str(self.header):
json = self.parameter
data = None
else:
data = self.parameter
json = None
with obj.client.request(method=self.method, path=self.path, headers=self.header, data=data, json=json,
catch_response=True) as response:
try:
assert_value = jsonpath.jsonpath(response.json(), '$..{}'.format(self.assert_key))
if assert_value:
if self.assert_value.isdigit():
# 判断是否断言状态码
if int(self.assert_value) in assert_value:
response.success()
else:
response.failure("断言失败!。返回值:{}".format(response.text))
else:
# 不是状态码
if self.assert_value in assert_value:
response.success()
else:
response.failure("断言失败!。返回值:{}".format(response.text))
else:
response.failure("断言失败!。返回值:{}".format(response.text))
except KeyError:
response.failure("断言失败!。返回值:{}".format(response.text))
class Task:
"""任务类"""
class UserBehavior(TaskSet):
"""设置任务"""
task_name = locust_data['task_name']
all_data = locust_data['task']
"""通过反射机制循环添加任务"""
for i in locust_data['task']:
setattr(Task, task_name + str(all_data.index(i)),
Example(path=i['path'], method=i['method'], header=i['header'], parameter=i['data'],
assert_key=i['assert_key'], assert_value=i['assert_value']).example_test)
"""循环设置task权重"""
tasks = {}
for i in all_data:
tasks[getattr(Task, task_name + str(all_data.index(i)))] = i['task']
class WebUser(FastHttpUser):
"""运行"""
host = locust_data['environment']
tasks = [UserBehavior]
wait_time = constant_pacing(1) # 计时性的,也就是每1秒钟触发执行一次任务,而不管任务有没有执行完
@events.init.add_listener
def on_locust_init(environment, **_kwargs):
listeners.Timescale(env=environment, testplan="timescale_listener_ex")
这是基础脚本
from locust_plugins import run_single_user, listeners
from locust import HttpUser, task, events
import os
os.environ['PGHOST'] = '这里填TimescaleDB数据库IP'
os.environ['PGUSER'] = 'postgres'
os.environ['PGPASSWORD'] = '123456'
os.environ['PGDATABASE'] = 'tutorial'
class MyUser(HttpUser):
@task
def index(self):
self.client.post("/login", {"username": "admin", "password": "admin"})
host = "http://127.0.0.1:10000"
@events.init.add_listener
def on_locust_init(environment, **_kwargs):
listeners.Timescale(env=environment, testplan="timescale_listener_ex")
四.环境准备,运行脚本
1.在Linux环境下安装python,具体怎么安装就不多说了
2.安装两个库,如果用的我封装的脚本需要安装一个josnpath的库
pip install locust
pip install locust-plugins
3.普通模式运行
locust -f test.py --headless -u 1 -r 1 --run-time 10s
如下图说明成功入库
4.前往数据库查询数据
五.部署Grafana
参考:https://blog.csdn.net/qq_36076898/article/details/106878817
1.设置Grafana,并导入dashboard https://grafana.com/grafana/dashboards/10878
2.Granfana连接TimescaleDB数据库
3.导入json文件
4.采用分布式运行脚本,不知道怎么用分布式的请看https://blog.csdn.net/qq_36076898/article/details/109015136
locust -f test.py --master --web-host=0.0.0.0 --headless -u 4 -r 1 --run-time 10s --expect-workers=2
locust -f test.py --worker --master-host=127.0.0.1
locust -f test.py --worker --master-host=127.0.0.1