理解磁盘访问I/O性能||IO Workload Profiles

• Overview

  关于数据库的性能，对于column-oriented database，很多时候性能取决于计算机硬件的磁盘访问I/O性能。

• 一、Workload Profiles

• Workload Profiles

  In the most generic sense, a workload profile describes the behavior of an application, or portion of an application with respect to how it performs work;

  Workload profiles for storage systems are typically descibed and grouped by key “Workload Characteristics”;

• IO workload characteristics

  There are many characteristics of an IO workload that affect performance:

  1. IO Size

     IO size: the size of the Input or Output request.

     The smaller an IO, the less time it takes to transmit; the larger an IO, the longer it takes to transmit;

     It is the compromise that most applications need to trade-off moving data efficiently in the right size containers with as few round trips as possible.

  2. IOPS

     IOPS measures the number of Input/Output operations Per Second, this is sometimes referred to as requests per second.

     The number of vehicles that pass through the boll in one hour is analogous to the number of bytes that pass through an interface each percond - IOPS;

  3. Throughput (bandwidth)

     Throughput measures the amount of work performed in a given time period;

     Returing to the toll road example, the throughout measures the amount of cargo that passes through the toll tooths in a given period of time;
     $$IOPS\ast IOSize=Throughput$$

  4. Response Time

     The time required to send, process, and receive acknowledgement (ACK) for a request is known as Response Time (RT).

     A read request involves sending a request for data, processing time to retrieve the data, and returning the data payload;

     A write request involves sending a data payload, processing time to store the data, and returing confirmation (ACK) that the write has completed;

  5. Read/Write Ratio

     How many requests are reads compared to writes;

  6. Random vs. Sequential

     Sequential IO is a sequence of requests accessing data blocks that are adjacent to each other:

• Other notions

• Bits vs. Bytes

  Bits are referenced with a lower-case ‘b’ and Bytes are referenced by an upper-case ‘B’;
  $$\begin{gathered} 8b=1B \\ 4b=1nybble \end{gathered}$$

• Decimal vs. Binary

  The storage industry uses a decimal system for raw capacity using multiples of 1000;

  The computer systems use binary, and the units of measure are based on powers of 1024;
  $$\begin{gathered} 1TB=1000GB=1000^{4}Bytes \\ 1TiB=1024GiB=1024^{4}Bytes \end{gathered}$$

• Workload Variations

  Sustained workload; Averages; Peaks and Bursts;

• 二、有几种I/O

• Buffer IO 缓存I/O（标准I/O）

  缓存I/O又被称作标准I/O

  Buffered I/O means the data for the I/O gets buffered or stored in some fast temporary storage, and gathered there for a while, before the I/O is actually done on the storage device;

• Direct IO 直接I/O

  Direct I/O means the data gets written out to the storage device as soon as your program issues the call to do so.

• text I/O, binary I/O and raw I/O

  From [3], The io module provides Python’s main facilities for dealing with various types of I/O.

  There are three main types of I/O:

  1. text I/O
  2. binary I/O
  3. raw I/O

• IO检测工具

• iostat

  理解iostat in Linux

  Sar is part of the sysstat package. According to the package, it includes the following system performance tools:

  • sar: collects and reports system activity information
  • iostat: report CPU utilization and disk I/O statistics;
  • mpstat: reports global and per-processor statistics;
  • pidstat: reports statistics for Linux tasks (processes);
  • sadf: displays data collected bby sar in various formats;

  Sar by Jack Jonn

• iotop

  sudo apt update
  sudo apt install iotop
  

  How to install iotop in Ubuntu 18.04

  iotop homepage

• blktrace

  IO神器blktrace使用介绍

  利用BLKTRACE分析IO性能

  Linux blktrace

  blktrace分析IO

  How to Install blktrace in Ubuntu 18.04

• Summary

  iostat 是统计工具，查看具体某个进程的io问题需要iotop或blktrace

• References

1. 磁盘IO瓶颈
2. 美团技术团队：磁盘I/O那些事
3. Python Standard Library: io - Core tools for working with streams
4. SQL Server disk performance metrics – Part 1 – the most important disk performance metrics
5. MicroSoft: How To: Troubleshooting SQL Server I/O bottlenecks
6. SqlMonitor: Resolving SQL Server Disk IO bottlenecks
7. Undertanding how i/o workload profiles relate to performance
8. Oracle 8: Designing and Tuning for Performance Release 2
9. What’s the difference between buffered I/O and unbuffered I/O?