scheduler performance testing

Scheduler Performance Test Manual

主机测试

1. 并行任务

time make -jX  #监测并行任务的runtime
perf stat make -jX #统计并行任务的数据

以并行任务数量X为横轴，测得的runtime（Y）为纵轴，绘制曲线图。

举例：

通过make -jX来跑并行任务。我这里通过编译qemu-5.1.0来测试性能。

cd qemu-5.1.0
mkdir build
cd build
../configure --target-list=x86_64-softmmu
perf stat make -jX

2. IPC之管道通信

基于pipe()测试进程间IPC性能。

Ingo Molar的pipe-test-1m.c程序如下：

#include<unistd.h>
#include<stdio.h>
#include<stdlib.h>
#include<signal.h>
#include<sys/wait.h>
#include<linux/unistd.h>

#define LOOPS 1000000

int main(int argc, char *argv[])
{
    unsigned long long t0, t1;
    int pipe_1[2], pipe_2[2];
    int m = 0, i;

    pipe(pipe_1);
    pipe(pipe_2);

    if(!fork()){
        for(i = 0; i < LOOPS; i++){
            read(pipe_1[0], &m, sizeof(int));
            write(pipe_2[1], &m, sizeof(int));
        }
    }else{
        for(i = 0; i < LOOPS; i++){
            write(pipe_1[1], &m, sizeof(int));
            read(pipe_2[0], &m, sizeof(int));
        }
    }

    return 0;
}

现在perf中以及基于该pipe-test-1m.c程序集成了pipe()性能测试工具。

perf bench sched pipe -l 10000  #在两个进程之间执行10000个pipe通信操作

但光有pipe()函数的测试还不够，还需要并行执行该pipe()测试

#!/bin/sh

pairs=$1
loops=1000000

file="test-"$pairs"-"$loops
cat /dev/null > $file

for((i=1; i<=${pairs}; i++));
do
    (echo "$(date +%H:%M:%S) | $(perf bench sched pipe -l $loops | grep "Total time" | awk '{print $3}') | $(date +%H:%M:%S)" >> $file) &
done

可以利用上面的脚本，生成测试文件，然后读取文件中最早时间戳和最晚时间戳之差"与"执行最长时间"两个值较大值。

对"test-$pairs-1000000"的处理脚本如下：

#!/bin/sh

cat /dev/null > fifo_pipe.txt

for((i=1; i<=50; i++))
do
    (echo -n "$i " >> fifo_pipe.txt && echo $(cat "test-$i-1000000" | cut -d'|' -f2 | sort -nr | sed -n '1p') >> fifo_pipe.txt)
done

绘图：X轴是并行pairs任务数量，Y轴是runtime执行时间。

3. 调度性能及IPC机制

perf中有基于hackbench的相关测试。

Suite for evaluating performance of scheduler and IPC mechanisms. Based on hackbench by Rusty Russell.

perf bench sched messaging -g 10       #400个threads,  10 groups == 400 processes

perf bench sched messaging -t -g 15        #600个threads	

perf bench sched messaging -t -g 20        #800个threads, 20 groups == 800 threads

perf bench sched messaging -t -g 25        #1000个threads

测试脚本：

#!/bin/sh

if [ $1 -gt 25 ]; then
    echo "error: too many groups"
    exit 0
fi

groups=${1:-10}

for((i=1; i<=${groups}; i++))
do
    echo -n "$i "
    perf bench sched messaging -t -g $i | grep "Total time:" | awk {'print $3'}
done

绘图：X轴是groups数量，Y轴是runtime执行时间。

4. benchmark压力测试

这里使用sysbench工具，可以测试 文件I/O性能、CPU性能、内存性能、线程子系统性能、以及mutex性能测试。

举例

cpu测试举例

[root@localhost sysbench]# sysbench cpu --threads=20 run
sysbench 1.0.17 (using system LuaJIT 2.0.4)

Running the test with following options:
Number of threads: 20
Initializing random number generator from current time


Prime numbers limit: 10000

Initializing worker threads...

Threads started!

CPU speed:
    events per second: 16461.50

General statistics:
    total time:                          10.0012s
    total number of events:              164704

Latency (ms):
         min:                                    1.15
         avg:                                    1.21
         max:                                    3.03
         95th percentile:                        1.39
         sum:                               199951.66

Threads fairness:
    events (avg/stddev):           8235.2000/124.07
    execution time (avg/stddev):   9.9976/0.00

mutex调度程序性能举例

[root@localhost sysbench]# sysbench mutex --threads=48 --mutex-num=1024 --mutex-locks=50000 --mutex-loops=100000 run
sysbench 1.0.17 (using system LuaJIT 2.0.4)

Running the test with following options:
Number of threads: 48
Initializing random number generator from current time


Initializing worker threads...

Threads started!


General statistics:
    total time:                          5.1528s
    total number of events:              48

Latency (ms):
         min:                                 1847.95
         avg:                                 3603.47
         max:                                 5148.72
         95th percentile:                     3773.42
         sum:                               172966.62

Threads fairness:
    events (avg/stddev):           1.0000/0.00
    execution time (avg/stddev):   3.6035/0.41

qemu虚机环境测试

1. latency延迟

可以用perf sched latency来测试qemu虚机的延迟数据。

2. load balance

可以自编工具来监测qemu进程的CPU迁移热力图。

3. perf kvm

perf kvm模块可以测试虚拟机环境下的性能。