本文目录
通过 iPerf3 -P参数详细图文分析我们知道,在以下二种情况下,我们有可能会使用到-P参数来优化测试参数以达到最准确的网络测试结果。
- 测试:“客户端和服务端之间多连接”的测试场景
- TCP接收窗口和发送窗口太小,以至于无法充分利用网络带宽
第一种场景,是我们使用-P参数的标准场景,而第二种场景,我们通常需要通过-W参数来调整TCP接收窗口和发送窗口的大小,来充分利用网络带宽。本文描述的是iperf3如何在客户端和接收端管理与处理“iperf3 -P多连接流并行测试”场景下的多个连接的收发包测试过程 。
从以下文章中,我们已经分析了状态机转换和代码调用的主要过程。
- 七,iperf3源代码分析:状态机及状态转换过程—>运行正向TCP单向测试时的服务端代码
- 八,iperf3源代码分析:状态机及状态转换过程—>运行正向TCP单向测试时的客户端代码
- 九,iperf3源代码分析:正向TCP单向测试时服务端和客户端的交互过程详解
1、总体代码结构和业务流程
总体来说:
1.1、对于服务端:
-
创建过程:
在进入CREATE_STREAM状态后,iperf3服务端在收到客户段发过来的测试TCP连接建立的请求后,会先调用iperf_accept()—>iperf_tcp_accept()来建立测试链接test stream,然后调用iperf_new_stream()—>iperf_init_stream()—>iperf_add_stream()创建并添加当前测试连接test stream到test实例下,注意这个测试链接test stream依赖于-P参数的设置,可以有一条或者多条。 -
管理与使用过程:
在进入TEST_RUNNING状态后,iperf3服务端会用select轮询所有测试连接test stream,并通过所有有收到测试数据的连接的est stream的socket上调用iperf_recv()—>iperf_tcp_recv()来收取测试数据,并做测试结果统计。直到收到客户端发过来的TEST_END指令,进入TEST_END状态为止。 -
释放过程:
在进入IPERF_DONE状态后,iperf3服务端会调用iperf_reset_test()—>iperf_free_stream() 遍历所有测试连接test stream,并挨个释放测试连接test stream,直到释放完全部test stream。
1.2、对于客户端:
-
创建过程:
在收到服务端发过来的的CREATE_STREAM指令后,进入CREATE_STREAM状态,iperf3客户端会主动发起TCP连接建立的请求,并调用iperf_new_stream—iperf_init_stream—iperf_add_stream直到所有测试连接test stream建立成功,并挂到到test实例下,注意这个测试链接test stream依赖于-P参数的设置,可以有一条或者多条。 -
管理与使用过程:
在收到服务端发过来的的TEST_RUNNING指令后,在进入TEST_RUNNING状态后,iperf3客户端会轮询所有测试连接test stream,并调用iperf_send—iperf_tcp_send来发送每个stream上的测试数据,并做测试结果统计。直到完成本次测试为止。 -
释放过程:
在进入IPERF_DONE状态后,iperf3客户端会调用iperf_free_test—iperf_free_stream 遍历所有测试连接test stream,并挨个释放测试连接test stream,直到释放完全部test stream。
2、数据结构分析:
test steam是test这个结构下的了一个指针,这个指针指向一个单向列表,管理当前test实例下的一个或者多个test stream, 每个test stream 对应一个TCP测试连接。
struct iperf_test
{
char role; /* 'c' lient or 's' erver */
enum iperf_mode mode;
...... //此处省略很多代码
SLIST_HEAD(slisthead, iperf_stream) streams;//这里通过SLIST_HEAD这个宏定义了指向test->streams的单列表指针
SLIST_HEAD(plisthead, protocol) protocols;
...... //此处省略很多代码
};
这时通过SLIST_HEAD这个宏展开后struct slisthead{struct iperf_stream *slh_first;} streams;就是定义了一个类型为struct slisthead的变量streams, 这个变量的成员只有一个,就是类型为struct iperf_stream的指针slh_first。
可以通过streams.slh_first来进行访问。
3、重要的几个管理用的函数及期功能分析:
3.1、对于服务端:
-
创建过程:
iperf_run_server()—>iperf_tcp_accept()
iperf_run_server()—>iperf_new_stream()—>iperf_init_stream()—>iperf_add_stream()
接受客户端发起的TCP连接,创始并保存每个TCP连接实例。 -
管理与使用过程:
iperf_run_server()—>iperf_recv()—>iperf_tcp_recv(),查询每一条TCP连接,有数据接收的则调用iperf_tcp_recv()接收该连接上的所有数据: -
释放过程:
run()—>iperf_reset_test()—>iperf_free_stream() ,reset测试对象,释放所有测试连接,等待下一次测试开始
3.2、对于客户端:
-
创建过程:
iperf_create_streams()—>create_socket()—>iperf_new_stream()—>iperf_init_stream()—>iperf_add_stream()
调用iperf_create_streams后,循环调用create_socket向服务端发起TCPI测试连接,并创建TCP测试连接实例,每个TCP测试连接对应一个实例,并且以单列表的方式保存在test->streams.slh_first中。 -
管理与使用过程:
iperf_run_client()—>iperf_send()—>iperf_tcp_send(),循环发送每一个TCP测试连接里的数据,直到本次测试结束。 -
释放过程:
main()—>iperf_free_test()—>iperf_free_stream(), 完成本次测试后,返回到main函数,开始释放。
4、使用到的管理列表用的宏:
列表管理的宏都放在queue.h这个文件里,主要是对test->streams.slh_first这个单位列表进行,初始化,增加,删除,释放,查找,遍历等操作。以下摘录一部分代码做了一些注释,可以供阅读代码参考使用。
/*
* Singly-linked List definitions.
*/
// 定义一个单列表变量指针
#define SLIST_HEAD(name, type) \
struct name { \
struct type *slh_first; /* first element */ \
}
#define SLIST_HEAD_INITIALIZER(head) \
{ NULL }
#define SLIST_ENTRY(type) \
struct { \
struct type *sle_next; /* next element */ \
}
/*
* Singly-linked List access methods.
*/
//查找单列表头结点
#define SLIST_FIRST(head) ((head)->slh_first)
#define SLIST_END(head) NULL
#define SLIST_EMPTY(head) (SLIST_FIRST(head) == SLIST_END(head))
#define SLIST_NEXT(elm, field) ((elm)->field.sle_next)
//遍历单列表
#define SLIST_FOREACH(var, head, field) \
for((var) = SLIST_FIRST(head); \
(var) != SLIST_END(head); \
(var) = SLIST_NEXT(var, field))
#define SLIST_FOREACH_PREVPTR(var, varp, head, field) \
for ((varp) = &SLIST_FIRST((head)); \
((var) = *(varp)) != SLIST_END(head); \
(varp) = &SLIST_NEXT((var), field))
/*
* Singly-linked List functions.
*/
//初始化一个列表
#define SLIST_INIT(head) { \
SLIST_FIRST(head) = SLIST_END(head); \
}
5、运行过程分析
5.1、对于服务端:
- 创建过程:
iperf_run_server()—>iperf_tcp_accept()
iperf_run_server()—>iperf_new_stream()—>iperf_init_stream()—>iperf_add_stream()
接受客户端发起的TCP连接,创始并保存每个TCP连接实例。调用Log如下所示:
iperf3 -s
......
debug out: func = iperf_run_server ,line = 625, file = iperf_server_api.c
debug out: func = iperf_tcp_accept ,line = 123, file = iperf_tcp.c
debug out: func = iperf_run_server ,line = 727, file = iperf_server_api.c
debug out: func = iperf_new_stream ,line = 4247, file = iperf_api.c
debug out: func = iperf_init_stream ,line = 4398, file = iperf_api.c
debug out: func = iperf_add_stream ,line = 4461, file = iperf_api.c
debug out: add stream 1
[ 5] local 127.0.0.1 port 5201 connected to 127.0.0.1 port 56288
---------------------------------------------------------------------------------------
debug out: func = iperf_run_server ,line = 625, file = iperf_server_api.c
debug out: func = iperf_tcp_accept ,line = 123, file = iperf_tcp.c
debug out: func = iperf_run_server ,line = 727, file = iperf_server_api.c
debug out: func = iperf_new_stream ,line = 4247, file = iperf_api.c
debug out: func = iperf_init_stream ,line = 4398, file = iperf_api.c
debug out: func = iperf_add_stream ,line = 4461, file = iperf_api.c
debug out: search stream 1, i = 3
debug out: add stream 3, i = 3
[ 8] local 127.0.0.1 port 5201 connected to 127.0.0.1 port 56292
---------------------------------------------------------------------------------------
debug out: func = iperf_run_server ,line = 625, file = iperf_server_api.c
debug out: func = iperf_tcp_accept ,line = 123, file = iperf_tcp.c
debug out: func = iperf_run_server ,line = 727, file = iperf_server_api.c
debug out: func = iperf_new_stream ,line = 4247, file = iperf_api.c
debug out: func = iperf_init_stream ,line = 4398, file = iperf_api.c
debug out: func = iperf_add_stream ,line = 4461, file = iperf_api.c
debug out: search stream 1, i = 3
debug out: search stream 3, i = 4
debug out: add stream 4, i = 4
[ 10] local 127.0.0.1 port 5201 connected to 127.0.0.1 port 56304
---------------------------------------------------------------------------------------
debug out: func = iperf_run_server ,line = 625, file = iperf_server_api.c
debug out: func = iperf_tcp_accept ,line = 123, file = iperf_tcp.c
debug out: func = iperf_run_server ,line = 727, file = iperf_server_api.c
debug out: func = iperf_new_stream ,line = 4247, file = iperf_api.c
debug out: func = iperf_init_stream ,line = 4398, file = iperf_api.c
debug out: func = iperf_add_stream ,line = 4461, file = iperf_api.c
debug out: search stream 1, i = 3
debug out: search stream 3, i = 4
debug out: search stream 4, i = 5
debug out: add stream 5, i = 5
[ 12] local 127.0.0.1 port 5201 connected to 127.0.0.1 port 56314
---------------------------------------------------------------------------------------
- 管理与使用过程:
iperf_run_server()—>iperf_recv()—>iperf_tcp_recv()
调用Log如下所示,查询每一条TCP连接,有数据接收的则调用iperf_tcp_recv()接收该连接上的所有数据:
iperf3 -s
......
debug out: func = iperf_run_server ,line = 848, file = iperf_server_api.c
debug out: func = iperf_recv ,line = 1951, file = iperf_api.c
debug out: current sp->id =1
debug out: current sp->id =3
debug out: current sp->id =4
debug out: current sp->id =5
---------------------------------------------------------------------------------------
debug out: func = iperf_run_server ,line = 848, file = iperf_server_api.c
debug out: func = iperf_recv ,line = 1951, file = iperf_api.c
debug out: current sp->id =1
debug out: receiving at sp->id =1
debug out: func = iperf_tcp_recv ,line = 59, file = iperf_tcp.c
debug out: current sp->id =3
debug out: current sp->id =4
debug out: current sp->id =5
---------------------------------------------------------------------------------------
debug out: func = iperf_run_server ,line = 848, file = iperf_server_api.c
debug out: func = iperf_recv ,line = 1951, file = iperf_api.c
debug out: current sp->id =1
debug out: current sp->id =3
debug out: receiving at sp->id =3
debug out: func = iperf_tcp_recv ,line = 59, file = iperf_tcp.c
debug out: current sp->id =4
debug out: receiving at sp->id =4
debug out: func = iperf_tcp_recv ,line = 59, file = iperf_tcp.c
debug out: current sp->id =5
---------------------------------------------------------------------------------------
debug out: func = iperf_run_server ,line = 848, file = iperf_server_api.c
debug out: func = iperf_recv ,line = 1951, file = iperf_api.c
debug out: current sp->id =1
debug out: receiving at sp->id =1
debug out: func = iperf_tcp_recv ,line = 59, file = iperf_tcp.c
debug out: current sp->id =3
debug out: receiving at sp->id =3
debug out: func = iperf_tcp_recv ,line = 59, file = iperf_tcp.c
debug out: current sp->id =4
debug out: current sp->id =5
debug out: receiving at sp->id =5
debug out: func = iperf_tcp_recv ,line = 59, file = iperf_tcp.c
---------------------------------------------------------------------------------------
debug out: func = iperf_run_server ,line = 848, file = iperf_server_api.c
debug out: func = iperf_recv ,line = 1951, file = iperf_api.c
debug out: current sp->id =1
debug out: current sp->id =3
debug out: current sp->id =4
debug out: receiving at sp->id =4
debug out: func = iperf_tcp_recv ,line = 59, file = iperf_tcp.c
debug out: current sp->id =5
debug out: receiving at sp->id =5
debug out: func = iperf_tcp_recv ,line = 59, file = iperf_tcp.c
---------------------------------------------------------------------------------------
- 释放过程:
run()—>iperf_reset_test()—>iperf_free_stream()
debug out: func = run ,line = 179, file = main.c
debug out: func = iperf_reset_test ,line = 3072, file = iperf_api.c
debug out: func = iperf_free_stream ,line = 4222, file = iperf_api.c
debug out: free stream 1
debug out: func = iperf_free_stream ,line = 4222, file = iperf_api.c
debug out: free stream 3
debug out: func = iperf_free_stream ,line = 4222, file = iperf_api.c
debug out: free stream 4
debug out: func = iperf_free_stream ,line = 4222, file = iperf_api.c
debug out: free stream 5
5.2、对于客户端:
- 创建过程:
iperf_create_streams()—>create_socket()—>iperf_new_stream()—>iperf_init_stream()—>iperf_add_stream()
调用iperf_create_streams后,循环调用create_socket向服务端发起TCPI测试连接,并创建TCP测试连接实例,每个TCP测试连接对应一个实例,并且以单列表的方式保存在test->streams.slh_first中。
debug out: func = iperf_create_streams ,line = 69, file = iperf_client_api.c
debug out: func = create_socket ,line = 129, file = net.c
debug out: func = iperf_new_stream ,line = 4247, file = iperf_api.c
debug out: func = iperf_init_stream ,line = 4398, file = iperf_api.c
debug out: func = iperf_add_stream ,line = 4461, file = iperf_api.c
debug out: add stream 1
[ 5] local 127.0.0.1 port 56288 connected to 127.0.0.1 port 5201
debug out: func = create_socket ,line = 129, file = net.c
debug out: func = iperf_new_stream ,line = 4247, file = iperf_api.c
debug out: func = iperf_init_stream ,line = 4398, file = iperf_api.c
debug out: func = iperf_add_stream ,line = 4461, file = iperf_api.c
debug out: search stream 1, i = 3
debug out: add stream 3, i = 3
[ 7] local 127.0.0.1 port 56292 connected to 127.0.0.1 port 5201
debug out: func = create_socket ,line = 129, file = net.c
debug out: func = iperf_new_stream ,line = 4247, file = iperf_api.c
debug out: func = iperf_init_stream ,line = 4398, file = iperf_api.c
debug out: func = iperf_add_stream ,line = 4461, file = iperf_api.c
debug out: search stream 1, i = 3
debug out: search stream 3, i = 4
debug out: add stream 4, i = 4
[ 9] local 127.0.0.1 port 56304 connected to 127.0.0.1 port 5201
debug out: func = create_socket ,line = 129, file = net.c
debug out: func = iperf_new_stream ,line = 4247, file = iperf_api.c
debug out: func = iperf_init_stream ,line = 4398, file = iperf_api.c
debug out: func = iperf_add_stream ,line = 4461, file = iperf_api.c
debug out: search stream 1, i = 3
debug out: search stream 3, i = 4
debug out: search stream 4, i = 5
debug out: add stream 5, i = 5
[ 11] local 127.0.0.1 port 56314 connected to 127.0.0.1 port 5201
- 管理与使用过程:
iperf_run_client()—>iperf_send()—>iperf_tcp_send()
循环发送每一个TCP测试连接里的数据,直到本次测试结束。
debug out: func = iperf_run_client ,line = 633, file = iperf_client_api.c
debug out: func = iperf_send ,line = 1890, file = iperf_api.c
debug out: func = iperf_send ,line = 1907, file = iperf_api.c
debug out: func = iperf_tcp_send ,line = 87, file = iperf_tcp.c
debug out: sending at sp->id = 1
debug out: func = iperf_send ,line = 1907, file = iperf_api.c
debug out: func = iperf_tcp_send ,line = 87, file = iperf_tcp.c
debug out: sending at sp->id = 3
debug out: func = iperf_send ,line = 1907, file = iperf_api.c
debug out: func = iperf_tcp_send ,line = 87, file = iperf_tcp.c
debug out: sending at sp->id = 4
debug out: func = iperf_send ,line = 1907, file = iperf_api.c
debug out: func = iperf_tcp_send ,line = 87, file = iperf_tcp.c
debug out: sending at sp->id = 5
debug out: func = iperf_send ,line = 1907, file = iperf_api.c
debug out: func = iperf_tcp_send ,line = 87, file = iperf_tcp.c
debug out: sending at sp->id = 1
debug out: func = iperf_send ,line = 1907, file = iperf_api.c
debug out: func = iperf_tcp_send ,line = 87, file = iperf_tcp.c
debug out: sending at sp->id = 3
debug out: func = iperf_send ,line = 1907, file = iperf_api.c
debug out: func = iperf_tcp_send ,line = 87, file = iperf_tcp.c
debug out: sending at sp->id = 4
debug out: func = iperf_send ,line = 1907, file = iperf_api.c
debug out: func = iperf_tcp_send ,line = 87, file = iperf_tcp.c
debug out: sending at sp->id = 5
- 释放过程:
main()—>iperf_free_test()—>iperf_free_stream(), 完成本次测试后,返回到main函数,开始释放。
iperf Done.
debug out: func = main ,line = 121, file = main.c
debug out: func = iperf_free_test ,line = 2940, file = iperf_api.c
debug out: func = iperf_free_stream ,line = 4222, file = iperf_api.c
debug out: free stream 1
debug out: func = iperf_free_stream ,line = 4222, file = iperf_api.c
debug out: free stream 3
debug out: func = iperf_free_stream ,line = 4222, file = iperf_api.c
debug out: free stream 4
debug out: func = iperf_free_stream ,line = 4222, file = iperf_api.c
debug out: free stream 5