文章目录
简介
Testground是一个用于测试(testing) 、基准测试(benchmarking)、模拟分布式和p2p系统的平台。它设计为多语言且与运行时无关，可以需要时才从2个实例优雅地扩展到10000个实例。目前主要用于测试ipfs和libp2p。
编写测试计划需要Testground相关sdk，目前官方只提供go-sdk和js-sdk（libp2p目前只有go和js版比较完整）。
架构和概念
-
测试计划(test plans)：testground的能处理和部署的基本单位
-
测试用例(test cases)：测试计划包含一个或多个测试用例
-
构建器(builder): 构建器一个组件，它需要测试计划源代码和可选的SDK源代码，将他们编译成制品（二进制程序或镜像），最后给运行器使用
-
运行器(runner)：运行器是一个组件，它接受由构建器生成的制品，并在测试计划内，在testground上，使用指定数量的实例和测试参数来调度测试用例的测试运行。
------------- ----------- ------------------ ---------- ---------------
| plan code | -> | builder | -> | build artifact | -> | runner | -> | test output |
------------- ----------- ------------------ ---------- ---------------
-
守护进程和客户端(daemon and client)：
Testground运行时使用传统的daemon/client架构，通过HTTP进行通信。参考“命令”章节 -
同步服务(Synchronization service)：同步服务提供了简单但功能强大且可组合的原语（eg： barriers, signals, pubsub, latches, semaphores, etc），以协调和编排分布式测试实例
-
网络(Networking)：所有的运行器都有两个网络：控制网络和数据网络，分开是为了避免干扰测试。数据网络用于测试计划实例之间通信，控制网络用于测试计划实例与基础服务（如同步服务、influxdb）之间通信，控制网络无需做任何使用或配置，由sidecar自动帮你完成。
-
边车(sidecar)：sidecar是一个单独的进程，负责测试计划实例的网络管理和流量整形。它在主机上以特权模式运行，并通过同步服务监听来自测试计划实例的网络配置请求。sidecar仅支持在Docker和Kubernetes环境中运行。
-
运行时环境（runenv)：与测试计划运行时环境进行交互
命令
testground help
NAME:
testground - a platform for testing, benchmarking, and simulating distributed and p2p systems at scale
USAGE:
testground [global options] command [command options] [arguments...]
DESCRIPTION:
testground is a platform for testing, benchmarking, and simulating distributed and p2p systems at scale.
COMMANDS:
build request the daemon to build a test plan
collect collect the output assets of the supplied run into a .tgz archive
daemon start a long-running testground daemon process
describe describe a test plan
healthcheck validate/fix the preconditions for the runner to be able to operate properly
logs get the current status for a certain task
plan manage the plans known to the client
run request the daemon to (build and) run a test case
sidecar run the sidecar process
status get the current status for a certain task
tasks get a list of the existing tasks
terminate terminate all jobs and supporting processes of a runner
version print version numbers
help, h Shows a list of commands or help for one command
快速开始
操作系统：ubuntu
安装依赖
安装 go
wget https://raw.githubusercontent.com/canha/golang-tools-install-script/master/goinstall.sh
chmod +x goinstall.sh
./goinstall.sh
source /root/.bashrc
安装 docker
apt-get install docker-ce docker-ce-cli containerd.io
构建testground
$ git clone https://github.com/testground/testground.git
$ cd testground
$ make install # builds testground and the Docker image, used by the local:docker runner.
启动testground daemon
$ testground daemon # will start the daemon listening on localhost:8042 by default.
Dec 16 04:29:29.722835 INFO creating home directory at /root/testground
Dec 16 04:29:29.723010 INFO no .env.toml found at /root/testground/.env.toml; running with defaults
Dec 16 04:29:29.724441 INFO listen and serve {"addr": "127.0.0.1:8042"}
Dec 16 04:29:29.724499 INFO daemon listening {"addr": "127.0.0.1:8042"}
Dec 16 04:29:29.724548 INFO supervisor worker started {"worker_id": 0}
Dec 16 04:29:29.724591 INFO supervisor worker started {"worker_id": 1}
.env.toml是执行testground daemon或testground client启动时读取的配置文件
.env.toml默认在$TESTGROUND_HOME/.env.toml，$TESTGROUND_HOME默认为 $HOME/testground 。
如修改daemon端口
[daemon]
listen = ":8080"
[client]
endpoint = "localhost:8080"
导入测试计划
打开另外一个控制台
$ testground plan import --from ./plans/network
Dec 16 04:31:15.765219 INFO using home directory: /root/testground
Dec 16 04:31:15.765256 INFO no .env.toml found at /root/testground/.env.toml; running with defaults
created symlink /root/testground/plans/network -> ./plans/network
imported plans:
network ping-pong
network traffic-allowed
network traffic-blocked
运行测试计划
$ testground run single --plan=network --testcase=ping-pong \
--builder=docker:go --runner=local:docker \
--instances=2
Dec 16 04:32:50.464581 INFO created a synthetic composition file for this job; all instances will run under singleton group "single"
Dec 16 04:32:50.464615 INFO using home directory: /root/testground
Dec 16 04:32:50.464641 INFO no .env.toml found at /root/testground/.env.toml; running with defaults
Dec 16 04:32:50.464648 INFO testground client initialized {"addr": "http://localhost:8042"}
>>> Result:
Dec 16 04:32:50.472961 INFO run is queued with ID: bvcorsmria9uuaf1otag
后台会启动相关docker实例
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dde7056c9649 iptestground/sidecar:edge "testground sidecar …" 31 seconds ago Up 30 seconds 0.0.0.0:32768->6060/tcp testground-sidecar
a6a164e5bb1f influxdb:1.8 "/entrypoint.sh infl…" 33 seconds ago Up 30 seconds 0.0.0.0:8086->8086/tcp testground-influxdb
150cb105ff72 redis "docker-entrypoint.s…" 48 seconds ago Up 47 seconds 0.0.0.0:6379->6379/tcp testground-redis
f848fd7dac7a bitnami/grafana "/run.sh" About a minute ago Up 56 seconds 0.0.0.0:3000->3000/tcp testground-grafana
b7b1f67225b0 goproxy/goproxy "/usr/bin/tini -- /g…" 4 minutes ago Up 4 minutes 8081/tcp testground-goproxy
收到run命令后 daemon后台日志更新
latest: Pulling from goproxy/goproxy
05e7bc50f07f: Pulling fs layer
77bcff5d981b: Pulling fs layer
78a343aaa7d2: Pulling fs layer
fa2128e2ad96: Pulling fs layer
b37663944960: Pulling fs layer
f03cc7845d28: Pulling fs layer
4bc3bad59fe7: Pulling fs layer
f20e9cc61ca7: Pulling fs layer
05873b763a9d: Pulling fs layer
fa2128e2ad96: Waiting
b37663944960: Waiting
f20e9cc61ca7: Waiting
05873b763a9d: Waiting
4bc3bad59fe7: Waiting
f03cc7845d28: Waiting
Dec 16 04:33:29.178804 INFO created container {"container_name": "testground-goproxy", "id": "b7b1f67225b02352f28c34f175112a7736c46d6108d2c30870164d186526f332"}
Dec 16 04:33:29.178872 INFO starting container {"container_name": "testground-goproxy", "id": "b7b1f67225b02352f28c34f175112a7736c46d6108d2c30870164d186526f332"}
Dec 16 04:33:29.463650 INFO started container {"container_name": "testground-goproxy", "id": "b7b1f67225b02352f28c34f175112a7736c46d6108d2c30870164d186526f332"}
Step 1/22 : ARG BUILD_BASE_IMAGE
Step 2/22 : ARG RUNTIME_IMAGE=busybox:1.31.1-glibc
Step 3/22 : FROM ${BUILD_BASE_IMAGE} AS builder
1.14.4-buster:
e9afc4f90ab0:
989e6b19a265:
af14b6c2f878:
5573c4b30949:
d4020e2aa747:
85383fed8a84:
0bdb3996894f:
d4020e2aa747:
85383fed8a84:
0bdb3996894f:
5573c4b30949:
Dec 16 04:36:06.156168 INFO build completed {"default_tag": "5bcbbce3d199:latest", "took": "2m36s"}
Dec 16 04:36:06.165452 INFO got docker image id {"image_id": "863b1b70a69f"}
Dec 16 04:36:06.236677 INFO tagging image {"image_id": "863b1b70a69f", "tag": "tg-plan-network:863b1b70a69f"}
Dec 16 04:36:06.240635 INFO build succeeded {"plan": "network", "groups": ["single"], "builder": "docker:go", "artifact": "863b1b70a69f"}
Dec 16 04:36:06.240732 INFO performing healthcheck on runner
Dec 16 04:36:21.287373 INFO container not found; creating {"container_name": "testground-grafana"}
latest: Pulling from bitnami/grafana
75ea27cfafcd: Pulling fs layer
393639f3d70e: Pulling fs layer
fa4e1b225c35: Pulling fs layer
e9abb92ba38b: Pulling fs layer
f0bec968111c: Pulling fs layer
245801004eb4: Pulling fs layer
be11d53bd97e: Pulling fs layer
c15225e91572: Pulling fs layer
b1a19f378875: Pulling fs layer
2eeccf76c006: Pulling fs layer
245801004eb4: Waiting
be11d53bd97e: Waiting
c15225e91572: Waiting
b1a19f378875: Waiting
2eeccf76c006: Waiting
e9abb92ba38b: Waiting
f0bec968111c: Waiting
Dec 16 04:36:42.773833 INFO created container {"container_name": "testground-grafana", "id": "f848fd7dac7a5e07348a70c9883626bedc3bdc6f1a8ff0d8a045f86eb90878af"}
Dec 16 04:36:42.773886 INFO starting container {"container_name": "testground-grafana", "id": "f848fd7dac7a5e07348a70c9883626bedc3bdc6f1a8ff0d8a045f86eb90878af"}
Dec 16 04:36:43.083155 INFO started container {"container_name": "testground-grafana", "id": "f848fd7dac7a5e07348a70c9883626bedc3bdc6f1a8ff0d8a045f86eb90878af"}
Dec 16 04:36:43.084898 INFO container not found; creating {"container_name": "testground-redis"}
latest: Pulling from library/redis
6ec7b7d162b2: Pulling fs layer
1f81a70aa4c8: Pulling fs layer
968aa38ff012: Pulling fs layer
884c313d5b0b: Pulling fs layer
6e858785fea5: Pulling fs layer
78bcc34f027b: Pulling fs layer
884c313d5b0b: Waiting
6e858785fea5: Waiting
78bcc34f027b: Waiting
Dec 16 04:36:51.602900 INFO created container {"container_name": "testground-redis", "id": "150cb105ff720026a3c0b4d18e7f0dd2b7d825034145e2e62e96031cb3106bb8"}
Dec 16 04:36:51.602959 INFO starting container {"container_name": "testground-redis", "id": "150cb105ff720026a3c0b4d18e7f0dd2b7d825034145e2e62e96031cb3106bb8"}
Dec 16 04:36:51.912297 INFO started container {"container_name": "testground-redis", "id": "150cb105ff720026a3c0b4d18e7f0dd2b7d825034145e2e62e96031cb3106bb8"}
Dec 16 04:36:51.914329 INFO container not found; creating {"container_name": "testground-influxdb"}
1.8: Pulling from library/influxdb
22f9b9782fc3: Pulling fs layer
2c75c9e56a8a: Pulling fs layer
b31d19b99daa: Pulling fs layer
d52143fc259e: Pulling fs layer
f7e0dbfc9f2c: Pulling fs layer
e5980bdfb50a: Pulling fs layer
83e480e68ce4: Pulling fs layer
a1e61efff46e: Pulling fs layer
f7e0dbfc9f2c: Waiting
e5980bdfb50a: Waiting
83e480e68ce4: Waiting
a1e61efff46e: Waiting
d52143fc259e: Waiting
Dec 16 04:37:07.949810 INFO created container {"container_name": "testground-influxdb", "id": "a6a164e5bb1f5844235131724645c88e0b5fa4adc1d89b25766e44a7203923c2"}
Dec 16 04:37:07.949866 INFO starting container {"container_name": "testground-influxdb", "id": "a6a164e5bb1f5844235131724645c88e0b5fa4adc1d89b25766e44a7203923c2"}
Dec 16 04:37:08.269577 INFO started container {"container_name": "testground-influxdb", "id": "a6a164e5bb1f5844235131724645c88e0b5fa4adc1d89b25766e44a7203923c2"}
Dec 16 04:37:08.272133 INFO container not found; creating {"container_name": "testground-sidecar"}
Dec 16 04:37:08.286054 INFO found existing image: iptestground/sidecar:edge {"container_name": "testground-sidecar"}
Dec 16 04:37:08.318706 INFO created container {"container_name": "testground-sidecar", "id": "dde7056c9649c095fad8fe2621c0a7fb98afc1f72ee15e29031b244065fd96e0"}
Dec 16 04:37:08.318756 INFO starting container {"container_name": "testground-sidecar", "id": "dde7056c9649c095fad8fe2621c0a7fb98afc1f72ee15e29031b244065fd96e0"}
Dec 16 04:37:08.616388 INFO started container {"container_name": "testground-sidecar", "id": "dde7056c9649c095fad8fe2621c0a7fb98afc1f72ee15e29031b244065fd96e0"}
Dec 16 04:37:08.617047 WARN some healthchecks failed, but continuing
Dec 16 04:37:08.617125 INFO starting run {"run_id": "bvcorsmria9uuaf1otag", "plan": "network", "case": "ping-pong", "runner": "local:docker", "instances": 2}
Dec 16 04:37:23.654664 INFO creating container {"runner": "local:docker", "run_id": "bvcorsmria9uuaf1otag", "name": "tg-network-ping-pong-bvcorsmria9uuaf1otag-single-0"}
Dec 16 04:37:23.696350 INFO creating container {"runner": "local:docker", "run_id": "bvcorsmria9uuaf1otag", "name": "tg-network-ping-pong-bvcorsmria9uuaf1otag-single-1"}
Dec 16 04:37:23.721501 INFO starting containers {"runner": "local:docker", "run_id": "bvcorsmria9uuaf1otag", "count": 2}
Dec 16 04:37:23.721586 INFO starting container {"runner": "local:docker", "run_id": "bvcorsmria9uuaf1otag", "id": "887f8e76e8fe4702e95fb8c9e3c37affc0a0819e69bf4257e14aa315316a8606", "group": "single", "group_index": 0}
Dec 16 04:37:23.721584 INFO starting container {"runner": "local:docker", "run_id": "bvcorsmria9uuaf1otag", "id": "dc612f79de359186b7f45f880544dbc1cc28f84d3b9800edb24c0b4b991272f9", "group": "single", "group_index": 1}
Dec 16 04:37:24.431125 INFO 0.7058s MESSAGE << single[001] (dc612f) >> registering default http handler at: http://[::]:6060/ (pprof: http://[::]:6060/debug/pprof/)
Dec 16 04:37:24.431236 INFO 0.7058s START << single[001] (dc612f) >> {"plan":"network","case":"ping-pong","run":"bvcorsmria9uuaf1otag","outputs_path":"/outputs","instances":2,"group":"single","group_instances":2,"network":"16.0.0.0/16","start_time":"0001-01-01T00:00:00Z"}
Dec 16 04:37:24.431297 INFO 0.7072s MESSAGE << single[001] (dc612f) >> waiting for network to initialize
Dec 16 04:37:24.433791 INFO 0.7122s OTHER << sidecar >> Dec 16 04:37:24.433494 INFO resolved route to host {"host": "testground-redis", "ip": "192.18.0.3"}
Dec 16 04:37:24.434095 INFO 0.7125s OTHER << sidecar >> Dec 16 04:37:24.433835 INFO resolved route to host {"host": "testground-influxdb", "ip": "192.18.0.4"}
Dec 16 04:37:24.434852 INFO 0.7124s MESSAGE << sidecar >> InfluxDB unavailable; no metrics will be dispatched: no InfluxDB URL in $INFLUXDB_URL env var
Dec 16 04:37:24.437338 INFO started containers {"runner": "local:docker", "run_id": "bvcorsmria9uuaf1otag", "count": 2}
Dec 16 04:37:24.438027 INFO 0.7155s MESSAGE << single[000] (887f8e) >> registering default http handler at: http://[::]:6060/ (pprof: http://[::]:6060/debug/pprof/)
Dec 16 04:37:24.441391 INFO 0.7160s START << single[000] (887f8e) >> {"plan":"network","case":"ping-pong","run":"bvcorsmria9uuaf1otag","outputs_path":"/outputs","instances":2,"group":"single","group_instances":2,"network":"16.0.0.0/16","start_time":"0001-01-01T00:00:00Z"}
Dec 16 04:37:24.441461 INFO 0.7199s OTHER << sidecar >> Dec 16 04:37:24.437914 INFO successfully resolved route to host {"container_id": "dc612f79de359186b7f45f880544dbc1cc28f84d3b9800edb24c0b4b991272f9"}
Dec 16 04:37:24.441523 INFO 0.7179s MESSAGE << sidecar >> InfluxDB unavailable; no metrics will be dispatched: no InfluxDB URL in $INFLUXDB_URL env var
Dec 16 04:37:24.442145 INFO 0.7200s MESSAGE << single[000] (887f8e) >> waiting for network to initialize
Dec 16 04:37:24.442219 INFO 0.7206s OTHER << sidecar >> Dec 16 04:37:24.441887 INFO successfully resolved route to host {"container_id": "887f8e76e8fe4702e95fb8c9e3c37affc0a0819e69bf4257e14aa315316a8606"}
Dec 16 04:37:24.447949 INFO 0.7264s OTHER << sidecar >> Dec 16 04:37:24.447316 INFO external routing disabled
Dec 16 04:37:24.447998 INFO 0.7264s OTHER << sidecar >> Dec 16 04:37:24.447376 INFO waiting for all networks to be ready {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag"}
Dec 16 04:37:24.448022 INFO 0.7265s OTHER << sidecar >> Dec 16 04:37:24.447526 INFO external routing disabled
Dec 16 04:37:24.448045 INFO 0.7265s OTHER << sidecar >> Dec 16 04:37:24.447584 INFO waiting for all networks to be ready {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag"}
Dec 16 04:37:24.448385 INFO 0.7268s OTHER << sidecar >> Dec 16 04:37:24.447989 INFO all networks ready {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag"}
Dec 16 04:37:24.448421 INFO 0.7269s OTHER << sidecar >> Dec 16 04:37:24.448051 INFO all networks ready {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag"}
Dec 16 04:37:25.430196 INFO 1.7083s MESSAGE << single[001] (dc612f) >> network initialisation successful
Dec 16 04:37:25.430546 INFO 1.7087s MESSAGE << single[001] (dc612f) >> network initialization complete
Dec 16 04:37:25.430776 INFO 1.7089s MESSAGE << single[001] (dc612f) >> claimed sequence numbers; global=1, group(single)=1
Dec 16 04:37:25.431296 INFO 1.7089s MESSAGE << single[001] (dc612f) >> before sync.MustBoundClient
Dec 16 04:37:25.431637 INFO 1.7090s MESSAGE << single[001] (dc612f) >> before netclient.MustConfigureNetwork
Dec 16 04:37:25.431695 INFO 1.7101s OTHER << sidecar >> Dec 16 04:37:25.431223 INFO applying network change {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag", "network": {"network":"default","IPv4":null,"IPv6":null,"enable":true,"default":{"latency":100000000,"jitter":0,"bandwidth":1048576,"filter":0,"loss":0,"corrupt":0,"corrupt_corr":0,"reorder":0,"reorder_corr":0,"duplicate":0,"duplicate_corr":0},"rules":null,"callback_state":"network-configured","routing_policy":"deny_all"}}
Dec 16 04:37:25.431725 INFO 1.7102s OTHER << sidecar >> Dec 16 04:37:25.431289 INFO external routing already disabled
Dec 16 04:37:25.442301 INFO 1.7205s MESSAGE << single[000] (887f8e) >> network initialisation successful
Dec 16 04:37:25.442485 INFO 1.7206s MESSAGE << single[000] (887f8e) >> network initialization complete
Dec 16 04:37:25.442629 INFO 1.7208s MESSAGE << single[000] (887f8e) >> claimed sequence numbers; global=2, group(single)=2
Dec 16 04:37:25.442679 INFO 1.7208s MESSAGE << single[000] (887f8e) >> before sync.MustBoundClient
Dec 16 04:37:25.442886 INFO 1.7210s MESSAGE << single[000] (887f8e) >> before netclient.MustConfigureNetwork
Dec 16 04:37:25.443379 INFO 1.7218s OTHER << sidecar >> Dec 16 04:37:25.443065 INFO applying network change {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag", "network": {"network":"default","IPv4":null,"IPv6":null,"enable":true,"default":{"latency":100000000,"jitter":0,"bandwidth":1048576,"filter":0,"loss":0,"corrupt":0,"corrupt_corr":0,"reorder":0,"reorder_corr":0,"duplicate":0,"duplicate_corr":0},"rules":null,"callback_state":"network-configured","routing_policy":"deny_all"}}
Dec 16 04:37:25.443412 INFO 1.7218s OTHER << sidecar >> Dec 16 04:37:25.443115 INFO external routing already disabled
Dec 16 04:37:26.443858 INFO 2.7218s MESSAGE << single[000] (887f8e) >> I am 2
Dec 16 04:37:26.443957 INFO 2.7219s MESSAGE << single[000] (887f8e) >> before reconfiguring network
Dec 16 04:37:26.444252 INFO 2.7227s OTHER << sidecar >> Dec 16 04:37:26.443772 INFO applying network change {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag", "network": {"network":"default","IPv4":"16.0.1.2/24","IPv6":null,"enable":true,"default":{"latency":100000000,"jitter":0,"bandwidth":1048576,"filter":0,"loss":0,"corrupt":0,"corrupt_corr":0,"reorder":0,"reorder_corr":0,"duplicate":0,"duplicate_corr":0},"rules":null,"callback_state":"ip-changed","routing_policy":"deny_all"}}
Dec 16 04:37:26.444320 INFO 2.7227s OTHER << sidecar >> Dec 16 04:37:26.443845 INFO external routing already disabled
Dec 16 04:37:27.432034 INFO 3.7101s MESSAGE << single[001] (dc612f) >> I am 1
Dec 16 04:37:27.432123 INFO 3.7102s MESSAGE << single[001] (dc612f) >> before reconfiguring network
Dec 16 04:37:27.432496 INFO 3.7109s OTHER << sidecar >> Dec 16 04:37:27.432089 INFO applying network change {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag", "network": {"network":"default","IPv4":"16.0.1.1/24","IPv6":null,"enable":true,"default":{"latency":100000000,"jitter":0,"bandwidth":1048576,"filter":0,"loss":0,"corrupt":0,"corrupt_corr":0,"reorder":0,"reorder_corr":0,"duplicate":0,"duplicate_corr":0},"rules":null,"callback_state":"ip-changed","routing_policy":"deny_all"}}
Dec 16 04:37:27.432570 INFO 3.7110s OTHER << sidecar >> Dec 16 04:37:27.432148 INFO external routing already disabled
Dec 16 04:37:28.844504 INFO 5.1226s MESSAGE << single[000] (887f8e) >> waiting until ready
Dec 16 04:37:28.944588 INFO 5.2227s MESSAGE << single[001] (dc612f) >> waiting until ready
Dec 16 04:37:28.944659 INFO 5.2227s MESSAGE << single[001] (dc612f) >> writing my id
Dec 16 04:37:28.944701 INFO 5.2227s MESSAGE << single[001] (dc612f) >> reading their id
Dec 16 04:37:29.044740 INFO 5.3228s MESSAGE << single[000] (887f8e) >> writing my id
Dec 16 04:37:29.044806 INFO 5.3228s MESSAGE << single[000] (887f8e) >> reading their id
Dec 16 04:37:29.044836 INFO 5.3229s MESSAGE << single[000] (887f8e) >> returning their id
Dec 16 04:37:29.044868 INFO 5.3229s MESSAGE << single[000] (887f8e) >> reading my id
Dec 16 04:37:29.144940 INFO 5.4229s MESSAGE << single[001] (dc612f) >> returning their id
Dec 16 04:37:29.145003 INFO 5.4229s MESSAGE << single[001] (dc612f) >> reading my id
Dec 16 04:37:29.145051 INFO 5.4229s MESSAGE << single[001] (dc612f) >> done
Dec 16 04:37:29.145092 INFO 5.4230s MESSAGE << single[001] (dc612f) >> ping RTT was 200.218681ms [200ms, 215ms]
Dec 16 04:37:29.244921 INFO 5.5230s MESSAGE << single[000] (887f8e) >> done
Dec 16 04:37:29.244988 INFO 5.5230s MESSAGE << single[000] (887f8e) >> ping RTT was 200.237375ms [200ms, 215ms]
Dec 16 04:37:29.245577 INFO 5.5240s OTHER << sidecar >> Dec 16 04:37:29.245161 INFO applying network change {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag", "network": {"network":"default","IPv4":"16.0.1.2/24","IPv6":null,"enable":true,"default":{"latency":10000000,"jitter":0,"bandwidth":1048576,"filter":0,"loss":0,"corrupt":0,"corrupt_corr":0,"reorder":0,"reorder_corr":0,"duplicate":0,"duplicate_corr":0},"rules":null,"callback_state":"latency-reduced","routing_policy":"deny_all"}}
Dec 16 04:37:29.245618 INFO 5.5241s OTHER << sidecar >> Dec 16 04:37:29.245230 INFO external routing already disabled
Dec 16 04:37:30.145665 INFO 6.4241s OTHER << sidecar >> Dec 16 04:37:30.145286 INFO applying network change {"sidecar": true, "run_id": "bvcorsmria9uuaf1otag", "network": {"network":"default","IPv4":"16.0.1.1/24","IPv6":null,"enable":true,"default":{"latency":10000000,"jitter":0,"bandwidth":1048576,"filter":0,"loss":0,"corrupt":0,"corrupt_corr":0,"reorder":0,"reorder_corr":0,"duplicate":0,"duplicate_corr":0},"rules":null,"callback_state":"latency-reduced","routing_policy":"deny_all"}}
Dec 16 04:37:30.145743 INFO 6.4242s OTHER << sidecar >> Dec 16 04:37:30.145346 INFO external routing already disabled
Dec 16 04:37:30.245715 INFO 6.5238s MESSAGE << single[000] (887f8e) >> ping pong
Dec 16 04:37:30.245791 INFO 6.5238s MESSAGE << single[000] (887f8e) >> waiting until ready
Dec 16 04:37:31.145850 INFO 7.4239s MESSAGE << single[001] (dc612f) >> ping pong
Dec 16 04:37:31.145931 INFO 7.4239s MESSAGE << single[001] (dc612f) >> waiting until ready
Dec 16 04:37:31.145974 INFO 7.4239s MESSAGE << single[001] (dc612f) >> writing my id
Dec 16 04:37:31.146012 INFO 7.4239s MESSAGE << single[001] (dc612f) >> reading their id
Dec 16 04:37:31.155794 INFO 7.4340s MESSAGE << single[000] (887f8e) >> writing my id
Dec 16 04:37:31.155849 INFO 7.4340s MESSAGE << single[000] (887f8e) >> reading their id
Dec 16 04:37:31.155884 INFO 7.4340s MESSAGE << single[000] (887f8e) >> returning their id
Dec 16 04:37:31.155956 INFO 7.4340s MESSAGE << single[000] (887f8e) >> reading my id
Dec 16 04:37:31.165858 INFO 7.4440s MESSAGE << single[001] (dc612f) >> returning their id
Dec 16 04:37:31.165934 INFO 7.4441s MESSAGE << single[001] (dc612f) >> reading my id
Dec 16 04:37:31.165979 INFO 7.4441s MESSAGE << single[001] (dc612f) >> done
Dec 16 04:37:31.166008 INFO 7.4441s MESSAGE << single[001] (dc612f) >> ping RTT was 20.155907ms [20ms, 35ms]
Dec 16 04:37:31.175929 INFO 7.4541s MESSAGE << single[000] (887f8e) >> done
Dec 16 04:37:31.175993 INFO 7.4541s MESSAGE << single[000] (887f8e) >> ping RTT was 20.151251ms [20ms, 35ms]
Dec 16 04:37:31.176510 INFO 7.4544s OK << single[000] (887f8e) >>
Dec 16 04:37:32.166867 INFO 8.4446s OK << single[001] (dc612f) >>
Dec 16 04:37:33.292353 INFO deleting containers {"runner": "local:docker", "run_id": "bvcorsmria9uuaf1otag", "ids": ["887f8e76e8fe4702e95fb8c9e3c37affc0a0819e69bf4257e14aa315316a8606", "dc612f79de359186b7f45f880544dbc1cc28f84d3b9800edb24c0b4b991272f9"]}
Dec 16 04:37:33.292454 INFO deleting container {"runner": "local:docker", "run_id": "bvcorsmria9uuaf1otag", "id": "dc612f79de359186b7f45f880544dbc1cc28f84d3b9800edb24c0b4b991272f9"}
Dec 16 04:37:33.292486 INFO deleting container {"runner": "local:docker", "run_id": "bvcorsmria9uuaf1otag", "id": "887f8e76e8fe4702e95fb8c9e3c37affc0a0819e69bf4257e14aa315316a8606"}
Dec 16 04:37:33.367388 INFO run finished successfully {"run_id": "bvcorsmria9uuaf1otag", "plan": "network", "case": "ping-pong", "runner": "local:docker", "instances": 2}
Dec 16 04:37:33.371499 INFO worker completed task {"worker_id": 1, "task_id": "bvcorsmria9uuaf1otag"}
查看测试结果
等待测试完成 最后的参数 是上一步生成的ID
testground collect --runner=local:docker bvcorsmria9uuaf1otag
Dec 16 04:39:49.893320 INFO using home directory: /root/testground
Dec 16 04:39:49.893376 INFO no .env.toml found at /root/testground/.env.toml; running with defaults
Dec 16 04:39:49.893384 INFO testground client initialized {"addr": "http://localhost:8042"}
>>> Result:
Dec 16 04:39:49.895647 INFO created file: bvcorsmria9uuaf1otag.tgz
也可以使用浏览器打开http://localhost:8042查看task状态
bvcorsmria9uuaf1otag.tgz在当前命令运行目录,展开看一下
tar zxvf bvcorsmria9uuaf1otag.tgz
bvcorsmria9uuaf1otag/.
bvcorsmria9uuaf1otag/single
bvcorsmria9uuaf1otag/single/0
bvcorsmria9uuaf1otag/single/0/diagnostics.out
bvcorsmria9uuaf1otag/single/0/results.out
bvcorsmria9uuaf1otag/single/0/run.err
bvcorsmria9uuaf1otag/single/0/run.out
bvcorsmria9uuaf1otag/single/1
bvcorsmria9uuaf1otag/single/1/diagnostics.out
bvcorsmria9uuaf1otag/single/1/results.out
bvcorsmria9uuaf1otag/single/1/run.err
bvcorsmria9uuaf1otag/single/1/run.out
目前还没有用于输出后处理的工具, 下面是查看test plan中发送消息的方法
cd bvcorsmria9uuaf1otag/
find . | grep run.out | xargs awk '{print FILENAME, " >>> ", $0 }'
./single/1/run.out >>> {"ts":1608093444427361773,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"registering default http handler at: http://[::]:6060/ (pprof: http://[::]:6060/debug/pprof/)"}}}
./single/1/run.out >>> {"ts":1608093444427401493,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"start_event":{"runenv":{"plan":"network","case":"ping-pong","run":"bvcorsmria9uuaf1otag","params":{},"instances":2,"outputs_path":"/outputs","network":"16.0.0.0/16","group":"single","group_instances":2}}}}
./single/1/run.out >>> {"ts":1608093444428796602,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"waiting for network to initialize"}}}
./single/1/run.out >>> {"ts":1608093445429821498,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"network initialisation successful"}}}
./single/1/run.out >>> {"ts":1608093445430298189,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"network initialization complete"}}}
./single/1/run.out >>> {"ts":1608093445430442397,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"claimed sequence numbers; global=1, group(single)=1"}}}
./single/1/run.out >>> {"ts":1608093445430456376,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"before sync.MustBoundClient"}}}
./single/1/run.out >>> {"ts":1608093445430606804,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"before netclient.MustConfigureNetwork"}}}
./single/1/run.out >>> {"ts":1608093447431679260,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"I am 1"}}}
./single/1/run.out >>> {"ts":1608093447431744844,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"before reconfiguring network"}}}
./single/1/run.out >>> {"ts":1608093448944236931,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"waiting until ready"}}}
./single/1/run.out >>> {"ts":1608093448944299384,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"writing my id"}}}
./single/1/run.out >>> {"ts":1608093448944309506,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"reading their id"}}}
./single/1/run.out >>> {"ts":1608093449144454226,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"returning their id"}}}
./single/1/run.out >>> {"ts":1608093449144505708,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"reading my id"}}}
./single/1/run.out >>> {"ts":1608093449144513021,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"done"}}}
./single/1/run.out >>> {"ts":1608093449144530942,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"ping RTT was 200.218681ms [200ms, 215ms]"}}}
./single/1/run.out >>> {"ts":1608093451145431608,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"ping pong"}}}
./single/1/run.out >>> {"ts":1608093451145461808,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"waiting until ready"}}}
./single/1/run.out >>> {"ts":1608093451145485363,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"writing my id"}}}
./single/1/run.out >>> {"ts":1608093451145498039,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"reading their id"}}}
./single/1/run.out >>> {"ts":1608093451165601143,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"returning their id"}}}
./single/1/run.out >>> {"ts":1608093451165631349,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"reading my id"}}}
./single/1/run.out >>> {"ts":1608093451165637387,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"done"}}}
./single/1/run.out >>> {"ts":1608093451165648728,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"ping RTT was 20.155907ms [20ms, 35ms]"}}}
./single/1/run.out >>> {"ts":1608093452166135516,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"success_event":{"group":"single"}}}
./single/0/run.out >>> {"ts":1608093444437018218,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"registering default http handler at: http://[::]:6060/ (pprof: http://[::]:6060/debug/pprof/)"}}}
./single/0/run.out >>> {"ts":1608093444437523109,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"start_event":{"runenv":{"plan":"network","case":"ping-pong","run":"bvcorsmria9uuaf1otag","params":{},"instances":2,"outputs_path":"/outputs","network":"16.0.0.0/16","group":"single","group_instances":2}}}}
./single/0/run.out >>> {"ts":1608093444441555387,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"waiting for network to initialize"}}}
./single/0/run.out >>> {"ts":1608093445442067519,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"network initialisation successful"}}}
./single/0/run.out >>> {"ts":1608093445442201374,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"network initialization complete"}}}
./single/0/run.out >>> {"ts":1608093445442370951,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"claimed sequence numbers; global=2, group(single)=2"}}}
./single/0/run.out >>> {"ts":1608093445442389712,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"before sync.MustBoundClient"}}}
./single/0/run.out >>> {"ts":1608093445442570525,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"before netclient.MustConfigureNetwork"}}}
./single/0/run.out >>> {"ts":1608093446443389508,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"I am 2"}}}
./single/0/run.out >>> {"ts":1608093446443420568,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"before reconfiguring network"}}}
./single/0/run.out >>> {"ts":1608093448844148740,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"waiting until ready"}}}
./single/0/run.out >>> {"ts":1608093449044346615,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"writing my id"}}}
./single/0/run.out >>> {"ts":1608093449044408126,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"reading their id"}}}
./single/0/run.out >>> {"ts":1608093449044455012,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"returning their id"}}}
./single/0/run.out >>> {"ts":1608093449044475854,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"reading my id"}}}
./single/0/run.out >>> {"ts":1608093449244549492,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"done"}}}
./single/0/run.out >>> {"ts":1608093449244590074,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"ping RTT was 200.237375ms [200ms, 215ms]"}}}
./single/0/run.out >>> {"ts":1608093450245355750,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"ping pong"}}}
./single/0/run.out >>> {"ts":1608093450245389999,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"waiting until ready"}}}
./single/0/run.out >>> {"ts":1608093451155532642,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"writing my id"}}}
./single/0/run.out >>> {"ts":1608093451155568983,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"reading their id"}}}
./single/0/run.out >>> {"ts":1608093451155576253,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"returning their id"}}}
./single/0/run.out >>> {"ts":1608093451155584572,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"reading my id"}}}
./single/0/run.out >>> {"ts":1608093451175660560,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"done"}}}
./single/0/run.out >>> {"ts":1608093451175696272,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"message_event":{"message":"ping RTT was 20.151251ms [20ms, 35ms]"}}}
./single/0/run.out >>> {"ts":1608093451175962656,"msg":"","group_id":"single","run_id":"bvcorsmria9uuaf1otag","event":{"success_event":{"group":"single"}}}
附：上面运行的测试用例
manifest.toml
name = "network"
[defaults]
builder = "exec:go"
runner = "local:exec"
[builders."docker:go"]
enabled = true
[builders."exec:go"]
enabled = true
[runners."local:docker"]
enabled = true
[runners."local:exec"]
enabled = true
[runners."cluster:swarm"]
enabled = true
[runners."cluster:k8s"]
enabled = true
# seq 0
[[testcases]]
name = "ping-pong"
instances = { min = 2, max = 2, default = 2 }
main.go
package main
import (
"github.com/testground/sdk-go/network"
"github.com/testground/sdk-go/run"
)
var testcases = map[string]interface{}{
"ping-pong": run.InitializedTestCaseFn(pingpong),
"traffic-allowed": routingPolicyTest(network.AllowAll),
"traffic-blocked": routingPolicyTest(network.DenyAll),
}
func main() {
run.InvokeMap(testcases)
}
pingpong.go
package main
import (
"context"
"fmt"
"net"
"time"
"github.com/testground/sdk-go/network"
"github.com/testground/sdk-go/run"
"github.com/testground/sdk-go/runtime"
"github.com/testground/sdk-go/sync"
)
func pingpong(runenv *runtime.RunEnv, initCtx *run.InitContext) error {
ctx, cancel := context.WithTimeout(context.Background(), 300*time.Second)
defer cancel()
runenv.RecordMessage("before sync.MustBoundClient")
client := initCtx.SyncClient
netclient := initCtx.NetClient
oldAddrs, err := net.InterfaceAddrs()
if err != nil {
return err
}
config := &network.Config{
// Control the "default" network. At the moment, this is the only network.
Network: "default",
// Enable this network. Setting this to false will disconnect this test
// instance from this network. You probably don't want to do that.
Enable: true,
Default: network.LinkShape{
Latency: 100 * time.Millisecond,
Bandwidth: 1 << 20, // 1Mib
},
CallbackState: "network-configured",
RoutingPolicy: network.DenyAll,
}
runenv.RecordMessage("before netclient.MustConfigureNetwork")
netclient.MustConfigureNetwork(ctx, config)
seq := client.MustSignalAndWait(ctx, "ip-allocation", runenv.TestInstanceCount)
// Make sure that the IP addresses don't change unless we request it.
if newAddrs, err := net.InterfaceAddrs(); err != nil {
return err
} else if !sameAddrs(oldAddrs, newAddrs) {
return fmt.Errorf("interfaces changed")
}
runenv.RecordMessage("I am %d", seq)
ipC := byte((seq >> 8) + 1)
ipD := byte(seq)
config.IPv4 = runenv.TestSubnet
config.IPv4.IP = append(config.IPv4.IP[0:2:2], ipC, ipD)
config.IPv4.Mask = []byte{255, 255, 255, 0}
config.CallbackState = "ip-changed"
var (
listener *net.TCPListener
conn *net.TCPConn
)
if seq == 1 {
listener, err = net.ListenTCP("tcp4", &net.TCPAddr{Port: 1234})
if err != nil {
return err
}
defer listener.Close()
}
runenv.RecordMessage("before reconfiguring network")
netclient.MustConfigureNetwork(ctx, config)
switch seq {
case 1:
conn, err = listener.AcceptTCP()
case 2:
conn, err = net.DialTCP("tcp4", nil, &net.TCPAddr{
IP: append(config.IPv4.IP[:3:3], 1),
Port: 1234,
})
default:
return fmt.Errorf("expected at most two test instances")
}
if err != nil {
return err
}
defer conn.Close()
// trying to measure latency here.
err = conn.SetNoDelay(true)
if err != nil {
return err
}
pingPong := func(test string, rttMin, rttMax time.Duration) error {
buf := make([]byte, 1)
runenv.RecordMessage("waiting until ready")
// wait till both sides are ready
_, err = conn.Write([]byte{0})
if err != nil {
return err
}
_, err = conn.Read(buf)
if err != nil {
return err
}
start := time.Now()
// write sequence number.
runenv.RecordMessage("writing my id")
_, err = conn.Write([]byte{byte(seq)})
if err != nil {
return err
}
// pong other sequence number
runenv.RecordMessage("reading their id")
_, err = conn.Read(buf)
if err != nil {
return err
}
runenv.RecordMessage("returning their id")
_, err = conn.Write(buf)
if err != nil {
return err
}
runenv.RecordMessage("reading my id")
// read our sequence number
_, err = conn.Read(buf)
if err != nil {
return err
}
runenv.RecordMessage("done")
// stop
end := time.Now()
// check the sequence number.
if buf[0] != byte(seq) {
return fmt.Errorf("read unexpected value")
}
// check the RTT
rtt := end.Sub(start)
if rtt < rttMin || rtt > rttMax {
return fmt.Errorf("expected an RTT between %s and %s, got %s", rttMin, rttMax, rtt)
}
runenv.RecordMessage("ping RTT was %s [%s, %s]", rtt, rttMin, rttMax)
// Don't reconfigure the network until we're done with the first test.
state := sync.State("ping-pong-" + test)
client.MustSignalAndWait(ctx, state, runenv.TestInstanceCount)
return nil
}
err = pingPong("200", 200*time.Millisecond, 215*time.Millisecond)
if err != nil {
return err
}
config.Default.Latency = 10 * time.Millisecond
config.CallbackState = "latency-reduced"
netclient.MustConfigureNetwork(ctx, config)
runenv.RecordMessage("ping pong")
err = pingPong("10", 20*time.Millisecond, 35*time.Millisecond)
if err != nil {
return err
}
return nil
}
func sameAddrs(a, b []net.Addr) bool {
if len(a) != len(b) {
return false
}
aset := make(map[string]bool, len(a))
for _, addr := range a {
aset[addr.String()] = true
}
for _, addr := range b {
if !aset[addr.String()] {
return false
}
}
return true
}
参考
https://docs.testground.ai/
扫一扫
667
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
