1. blktrace安装
#yum install blktrace
#mount a /sys/kernel/debug -t debugfs
2. blktrace使用
1.通过blkiomon,可以告诉你IO size,完成花费时间,吞吐量, 还可以统计出这次关键参数的分布
#blktrace /dev/sde -a issue -a complete -w 3600 -o - | blkiomon -I 10 -h -
time: Fri Sep 13 09:50:23 2013
device: 8,64
sizes read (bytes): num 0, min -1, max 0, sum 0, squ 0, avg -nan, var -nan
sizes write (bytes): num 156, min 4096, max 524288, sum 80764928, squ 42331633876992, avg 517723.9, var 3318593440.4
d2c read (usec): num 0, min -1, max 0, sum 0, squ 0, avg -nan, var -nan
d2c write (usec): num 156, min 127, max 84010, sum 6410332, squ 330367299862, avg 41091.9, var 429197174.1
throughput read (bytes/msec): num 0, min -1, max 0, sum 0, squ 0, avg -nan, var -nan
throughput write (bytes/msec): num 156, min 6240, max 189068, sum 3417217, squ 164794315173, avg 21905.2, var 576534399.3
sizes histogram (bytes):
0: 0 1024: 0 2048: 0 4096: 1
8192: 0 16384: 0 32768: 1 65536: 0
131072: 0 262144: 0 524288: 154 1048576: 0
2097152: 0 4194304: 0 8388608: 0 > 8388608: 0
d2c histogram (usec):
0: 0 8: 0 16: 0 32: 0
64: 0 128: 1 256: 0 512: 1
1024: 0 2048: 0 4096: 1 8192: 7
16384: 24 32768: 12 65536: 102 131072: 8
262144: 0 524288: 0 1048576: 0 2097152: 0
4194304: 0 8388608: 0 16777216: 0 33554432: 0
>33554432: 0
bidirectional requests: 0
sizes write(bytes):
num:表示写IO次数
min:最小Write IO 4096Bytes
Max:最大Write IO 524288Bytes
sum:总共Write IO 80764928Bytes
avg:平均IO 517723Bytes
d2c write(usec)
num:表示写IO次数
min:最小IO响应时间127微秒
max:最大io响应时间84010微秒
sum:IO响应总时间6410332微秒
avg:平均IO响应时间41091微秒
d2c histogram (usec)柱状图说明:
响应时间128微秒以内的IO 1次
响应时间512微秒以内的IO 1次
响应时间4096微秒以内的IO 1次
响应时间8192微秒以内的IO 7次
响应时间16384微秒以内的IO 24次
响应时间32768微秒以内的IO 12次
响应时间65536微秒以内的IO 102次
响应时间131072微秒以内的IO 8次
blkparse使用
# blktrace /dev/sde -o - | blkparse -i -
8,64 0 1 0.000000000 2480 Q W 11853824 + 1024 [dd]
8,64 0 2 0.000011603 2480 G W 11853824 + 1024 [dd]
8,64 0 3 0.000016037 2480 P N [dd]
8,64 0 4 0.000092645 2480 Q W 11854848 + 1024 [dd]
8,64 0 5 0.000095456 2480 G W 11854848 + 1024 [dd]
8,64 0 6 0.000102259 2480 I W 11853824 + 1024 [dd]
8,64 0 0 0.000108432 0 m N cfq34A / insert_request
8,64 0 0 0.000110791 0 m N cfq34A / add_to_rr
8,64 0 7 0.000113614 2480 I W 11854848 + 1024 [dd]
8,64 0 0 0.000114744 0 m N cfq34A / insert_request
8,64 0 8 0.000115696 2480 U N [dd] 2
8,64 0 0 0.000118660 0 m N cfq workload slice:60
8,64 0 0 0.000120308 0 m N cfq34A / set_active wl_class:0 wl_type:0
8,64 0 0 0.000123098 0 m N cfq34A / fifo= (null)
8,64 0 0 0.000124405 0 m N cfq34A / dispatch_insert
8,64 0 0 0.000125979 0 m N cfq34A / dispatched a request
8,64 0 0 0.000127375 0 m N cfq34A / activate rq, drv=1
8,64 0 9 0.000127803 2480 D W 11853824 + 1024 [dd]
8,64 0 0 0.000153214 0 m N cfq34A / dispatch_insert
8,64 0 0 0.000154664 0 m N cfq34A / dispatched a request
8,64 0 0 0.000155569 0 m N cfq34A / activate rq, drv=2
8,64 0 10 0.000155730 2480 D W 11854848 + 1024 [dd]
8,64 0 0 0.002036492 0 m N cfq34A / slice expired t=0
8,64 0 0 0.002039138 0 m N / served: vt=444186624 min_vt=444178432
8,64 0 0 0.002041404 0 m N cfq34A / sl_used=2 disp=2 charge=2 iops=1 sect=2048
8,64 0 0 0.002042463 0 m N cfq34A / del_from_rr
8,64 0 0 0.002043051 0 m N / del_from_rr group
8,64 0 11 0.002073197 2436 C W 11853824 + 1024 [0]
8,64 0 0 0.002125838 0 m N cfq34A / complete rqnoidle 0
8,64 0 12 0.003306086 2288 C W 11854848 + 1024 [0]
8,64 0 0 0.003358069 0 m N cfq34A / complete rqnoidle 0
CPU0 (8,64):
Reads Queued: 0, 0KiB Writes Queued: 2, 1,024KiB
Read Dispatches: 0, 0KiB Write Dispatches: 2, 1,024KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 0, 0KiB Writes Completed: 2, 1,024KiB
Read Merges: 0, 0KiB Write Merges: 0, 0KiB
Read depth: 0 Write depth: 2
IO unplugs: 1 Timer unplugs: 0
Throughput (R/W): 0KiB/s / 341,333KiB/s
Events (8,64): 31 entries
Skips: 6 forward (51 - 62.2%)
blktrace是个很强大的IO监控工具,还往后的工作中会经常用到,需要更加详细的了解应用。