Linux那些事儿之我是Block层(10)scsi命令的前世今生(四)

当然 ,while 循环结束也可能是因为 1453 行的这两个判断 . 首先 req 如果没有了 , 另一个得看 scsi_dev_queue_ready() 的返回值 , 如果返回值为 0, 那么 break 也会被执行 , 从而结束循环 .

   1270 /*

   1271  * s csi_dev_queue_ready: if we can send requests to sdev, return 1 else

   1272  * return 0.

   1273  *

   1274  * Called with the queue_lock held.

   1275  */

   1276 static inline int scsi_dev_queue_ready(struct request_queue *q,

   1277                                   struct scsi_device *sdev)

   1278 {

   1279         if (sdev->device_busy >= sdev->queue_depth)

   1280                  return 0;

   1281         if (sdev->device_busy == 0 && sdev->device_blocked) {

   1282                 /*

   1283                  * unblock after device_blocked iterates to zero

   1284                  */

   1285                 if (--sdev->device_blocked == 0) {

   1286                         SCSI_LOG_MLQUEUE(3,

   1287                                    sdev_printk(KERN_INFO, sdev,

   1288                                    "unblocking device at zero depth/n"));

   1289                 } else {

   1290                         blk_plug_device(q);

   1291                         return 0;

   1292                 }

   1293         }

   1294         if (sdev->device_blocked)

   1295                 return 0;

   1296

   1297         return 1;

   1298 }

这里需要判断的是 device_busy. 这个 flag 如果设置了 , 说明命令正在执行中 , 或者说命令已经传递到了底层驱动 . 因此 , 我们在调用 scsi_dispatch_cmd 之前先增加 device_busy, 即 1469 行 .

另一个 flag 是 device_blocked. 这个 flag 是告诉世人这个设备不能再接收新的命令了 , 因为它十有八九是正在处理命令 . 正常情况下这个 flag 的值为 0. 除非你调用了 scsi_queue_insert() 函数 . 友情提示一下 ,scsi 设备的这个 flag 是提供了 sysfs 的接口的 , 因此我们可以通过 sysfs 的接口看一下设备的这个值 , 下面列举了两个 scsi 设备的这个变量的值 , 可以看到都是 0, 应该说这是它的常态 .

[root@localhost ~]# ls /sys/bus/scsi/devices/

0:0:8:0/ 0:2:0:0/ 1:0:0:0/ 2:0:0:0/

[root@localhost ~]# ls /sys/bus/scsi/devices/2/:0/:0/:0/

block:sdb/   iocounterbits        modalias              rev                  subsystem/ bus/   iodone_cnt          model                scsi_device:2:0:0:0/ timeout delete  ioerr_cnt   queue_depth          scsi_disk:2:0:0:0/   type device_blocked       iorequest_cnt        queue_type           scsi_level           uevent driver/   max_sectors     rescan           state                vendor

[root@localhost ~]# cat /sys/bus/scsi/devices/2/:0/:0/:0/device_blocked

0

[root@localhost ~]# cat /sys/bus/scsi/devices/0/:0/:8/:0/device_blocked

0

所以正常情况下 ,scsi_dev_queue_ready() 函数的返回值就是 1, 这一点正如其注释里说的那样 . 但是所谓的常态 , 指的是单独执行一个命令 , 如果要执行多个命令 , 或者说我们提交了多个 request, 那么 device_busy 就会一次次的在 1469 行增加 , 从而使得 device_busy 有可能将超过 queue_depth, 这样子 scsi_dev_queue_ready() 就会返回 0, 从而 scsi_request_fn() 就有可能结束 , 这之后 ,__generic_unplug_device 也将返回 , 之后 blk_execute_rq_nowait() 返回 , 回到 blk_execute_rq() 中 , 执行 wait_for_completion(), 于是就睡眠了 , 等待了 , 按照游戏规则 , 我们应该能找到一条 complete() 语句来唤醒它 , 那么这条语句在哪里呢 ? 答案是 blk_end_sync_rq.

网友 ” 宁失身不失眠 ” 非常好奇我是怎么知道的 . 说来话长 , 还记得我们当时在 usb-storage 中说的那个 scsi_done 么 ? 命令执行完了就会 call scsi_done. 而 scsi_done 来自 drivers/scsi/scsi.c, 很显然这个函数是我们的突破口 , 我们找到了这个函数就好比国民党找到了甫志高 , 就好比王佳芝找到了易先生 :

    608 /**

    609  * scsi_done - Enqueue the finished SCSI command into the done queue.

    610  * @cmd: The SCSI Command for which a low-level device driver (LLDD) gives

    611  * ownership back to SCSI Core -- i.e. the LLDD has finished with it.

    612  *

    613  * This function is the mid-level's (SCSI Core) interrupt routine, which

    614  * regains ownership of the SCSI command (de facto) from a LLDD, and enqueues

    615  * the command to the done queue for further processing.

    616  *

    617  * This is the producer of the done queue who enqueues at the tail.

    618  *

    619  * This function is interrupt context safe.

    620  */

    621 static void scsi_done(struct scsi_cmnd *cmd)

    622 {

    623         /*

    624          * We don't have to worry about this one timing out any more.

    625          * If we are unable to remove the timer, then the command

    626          * has already timed out.  In which case, we have no choice but to

    627          * let the timeout function run, as we have no idea where in fact

    628          * that function could really be.  It might be on another processor,

    629          * etc, etc.

    630          */

    631         if (!scsi_delete_timer(cmd))

    632                 return;

    633         __scsi_done(cmd);

    634 }

躲躲闪闪的是来自同一文件的 __scsi_done,

    636 /* Private entry to scsi_done() to complete a command when the timer

    637  * isn't running --- used by scsi_times_out */

    638 void __scsi_done(struct scsi_cmnd *cmd)

    639 {

    640         struct request *rq = cmd->request;

    641

    642         /*

    643          * Set the serial numbers back to zero

    644          */

    645         cmd->serial_number = 0;

    646

    647          atomic_inc(&cmd->device->iodone_cnt);

    648         if (cmd->result)

    649                 atomic_inc(&cmd->device->ioerr_cnt);

    650

    651         BUG_ON(!rq);

    652

    653         /*

    654          * The uptodate/nbytes values don't matter, as we allow partial

    655          * completes and thus will check this in the softirq callback

    656          */

    657         rq->completion_data = cmd;

    658         blk_complete_request(rq);

    659 }

别的我们都不关心 , 就关心最后这个 blk_complete_request().

   3588 /**

   3589  * blk_complete_request - end I/O on a request

   3590  * @req:      the request being processed

   3591  *

   3592  * Description:

   3593  *     Ends all I/O on a request. It does not handle partial completions,

   3594  *     unless the driver actually implements this in its completion callback

   3595  *     through requeueing. Theh actual completion happens out-of-order,

   3596  *     through a softirq handler. The user must have registered a completion

   3597  *     callback through blk_queue_softirq_done().

   3598  **/

   3599

   3600 void blk_complete_request(struct request *req)

   3601 {

   3602         struct list_head *cpu_list;

   3603         unsigned long flags;

   3604

   3605         BUG_ON(!req->q->softirq_done_fn);

   3606

   3607         local_irq_save(flags);

   3608

   3609         cpu_list = &__get_cpu_var(blk_cpu_done);

   3610         list_add_tail(&req->donelist, cpu_list);

   3611         raise_softirq_irqoff(BLOCK_SOFTIRQ);

   3612

   3613         local_irq_restore(flags);

   3614 }

其它的咱们不管 , 就管一管这个 raise_softirq_irqoff(). 在很久很久以前 , 有一个函数 , 它的名字叫做 blk_dev_init(). 它是我们这个故事的起源 . 在这个函数中我们曾经见过这么一行 ,

   3720         open_softirq(BLOCK_SOFTIRQ, blk_done_softirq, NULL);

当时咱们就说过 , 它所做的就是初始化了一个 softirq, 即 BLOCK_SOFTIRQ. 并且绑定了 softirq 函数 blk_done_softirq, 而要触发这个软中断 , 咱们当时也说了 , 只要调用 raise_softirq_irqoff() 即可 . 所以现在我们也就这样做了 . 这也就意味着 ,blk_done_softirq 会被调用 .

   3542 /*

   3543  * splice the completion data to a local structure and hand off to

   3544  * process_completion_queue() to complete the requests

    3545  */

   3546 static void blk_done_softirq(struct softirq_action *h)

   3547 {

   3548         struct list_head *cpu_list, local_list;

   3549

   3550         local_irq_disable();

   3551         cpu_list = &__get_cpu_var(blk_cpu_done);

   3552          list_replace_init(cpu_list, &local_list);

   3553         local_irq_enable();

   3554

   3555         while (!list_empty(&local_list)) {

   3556                 struct request *rq = list_entry(local_list.next, struct request, donelist);

   3557

   3558                  list_del_init(&rq->donelist);

   3559                 rq->q->softirq_done_fn(rq);

   3560         }

   3561 }

而这个 softirq_done_fn 是什么呢 ? 不要说你不知道 , 其实我们也讲过 . 不过忘记了也不要紧 , 人最大的烦恼便是记忆太好 , 健忘的人容易快乐 . 在 scsi_alloc_queue 中 , 我们调用 blk_queue_softirq_done 把 scsi_softirq_done 赋给了 q->softirq_done_fn, 所以到了这里 , 被调用的就是 scsi_softirq_done.

   1376 static void scsi_softirq_done(struct request *rq)

   1377 {

   1378         struct scsi_cmnd *cmd = rq->completion_data;

   1379         unsigned long wait_for = (cmd->allowed + 1) * cmd->timeout_per_command;

   1380         int disposition;

   1381

   1382         INIT_LIST_HEAD(&cmd->eh_entry);

   1383

   1384         disposition = scsi_decide_disposition(cmd);

   1385         if (disposition != SUCCESS &&

   1386             time_before(cmd->jiffies_at_alloc + wait_for, jiffies)) {

   1387                 sdev_printk(KERN_ERR, cmd->device,

   1388                             "timing out command, waited %lus/n",

   1389                             wait_for/HZ);

   1390                 disposition = SUCCESS;

   1391         }

   1392

   1393         scsi_log_completion(cmd, disposition);

   1394

   1395         switch (disposition) {

   1396                 case SUCCESS:

   1397                         scsi_finish_command(cmd);

   1398                          break;

   1399                 case NEEDS_RETRY:

   1400                         scsi_queue_insert(cmd, SCSI_MLQUEUE_EH_RETRY);

   1401                         break;

   1402                 case ADD_TO_MLQUEUE:

   1403                          scsi_queue_insert(cmd, SCSI_MLQUEUE_DEVICE_BUSY);

   1404                         break;

   1405                 default:

   1406                         if (!scsi_eh_scmd_add(cmd, 0))

   1407                                 scsi_finish_command(cmd);

    1408         }

   1409 }

不用我多说 , 你也知道 ,scsi_softirq_done 会调用 scsi_finish_command, 来自 drivers/scsi/scsi.c:

    661 /*

    662  * Function:    scsi_finish_command

    663  *

    664  * Purpose:     Pass command off to upper layer for finishing of I/O

    665  *              request, waking processes that are waiting on results,

    666  *              etc.

    667  */

    668 void scsi_finish_command(struct scsi_cmnd *cmd)

    669 {

    670         struct scsi_device *sdev = cmd->device;

    671         struct Scsi_Host *shost = sdev->host;

    672

    673         scsi_device_unbusy(sdev);

    674

    675         /*

    676          * Clear the flags which say that the device/host is no longer

    677          * capable of accepting new commands.  These are set in scsi_queue.c

    678           * for both the queue full condition on a device, and for a

    679          * host full condition on the host.

    680          *

    681          * XXX(hch): What about locking?

    682          */

    683         shost->host_blocked = 0;

    684          sdev->device_blocked = 0;

    685

    686         /*

    687          * If we have valid sense information, then some kind of recovery

    688          * must have taken place.  Make a note of this.

    689          */

    690         if (SCSI_SENSE_VALID(cmd))

    691                 cmd->result |= (DRIVER_SENSE << 24);

    692

    693         SCSI_LOG_MLCOMPLETE(4, sdev_printk(KERN_INFO, sdev,

    694                                 "Notifying upper driver of completion "

    695                                  "(result %x)/n", cmd->result));

    696

    697         cmd->done(cmd);

    698 }

也就是说 ,cmd->done 会被调用 , 从而真正的幕后工作者 scsi_blk_pc_done 会被调用 . 因为 , 当初在 scsi_setup_blk_pc_cmnd() 中有这么一行 ,

   1135         cmd->done = scsi_blk_pc_done;

而 scsi_blk_pc_done 来自 drivers/scsi/scsi_lib.c:

   1078 static void scsi_blk_pc_done(struct scsi_cmnd *cmd)

   1079 {

   1080          BUG_ON(!blk_pc_request(cmd->request));

   1081         /*

   1082          * This will complete the whole command with uptodate=1 so

   1083          * as far as the block layer is concerned the command completed

   1084          * successfully. Since this is a REQ_BLOCK_PC command the

   1085          * caller should check the request's errors value

   1086          */

   1087         scsi_io_completion(cmd, cmd->request_bufflen);

   1088 }

来自 drivers/scsi/scsi_lib.c:

    789 /*

    790  * Function:    scsi_io_completion()

    791  *

    792  * Purpose:     Completion processing for block device I/O requests.

    793  *

    794  * Arguments:   cmd   - command that is finished.

    795  *

    796  * Lock status: Assumed that no lock is held upon entry.

    797  *

    798  * Returns:     Nothing

    799  *

    800  * Notes:       This function is matched in terms of capabilities to

    801  *              the function that created the scatter-gather list.

    802  *              In other words, if there are no bounce buffers

    803  *              (the normal case for most drivers), we don't need

    804  *              the logic to deal with cleaning up afterwards.

    805  *

    806  *               We must do one of several things here:

    807  *

    808  *              a) Call scsi_end_request.  This will finish off the

    809  *                 specified number of sectors.  If we are done, the

    810  *                 command block will be released, and the queue

    811  *                 function will be goosed.  If we are not done, then

    812  *                 scsi_end_request will directly goose the queue.

    813  *

    814  *              b) We can just use scsi_requeue_command() here.  This would

    815  *                 be used if we just wanted to retry, for example.

    816  */

    817 void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes)

    818 {

    819         int result = cmd->result;

    820         int this_count = cmd->request_bufflen;

    821         request_queue_t *q = cmd->device->request_queue;

    822         struct request *req = cmd->request;

    823         int clear_errors = 1;

    824         struct scsi_sense_hdr sshdr;

    825         int sense_valid = 0;

    826         int sense_deferred = 0;

    827

    828         scsi_release_buffers(cmd);

    829

    830         if (result) {

    831                 sense_valid = scsi_command_normalize_sense(cmd, &sshdr);

    832                 if (sense_valid)

    833                         sense_deferred = scsi_sense_is_deferred(&sshdr);

    834         }

    835

    836         if (blk_pc_request(req)) { /* SG_IO ioctl from block level */

    837                 req->errors = result;

    838                  if (result) {

    839                         clear_errors = 0;

    840                         if (sense_valid && req->sense) {

    841                                 /*

    842                                  * SG_IO wants current and deferred errors

    843                                  */

    844                                 int len = 8 + cmd->sense_buffer[7];

    845

    846                                 if (len > SCSI_SENSE_BUFFERSIZE)

    847                                          len = SCSI_SENSE_BUFFERSIZE;

    848                                 memcpy(req->sense, cmd->sense_buffer,  len);

    849                                 req->sense_len = len;

    850                         }

    851                 }

    852                  req->data_len = cmd->resid;

    853         }

    854

    855         /*

    856          * Next deal with any sectors which we were able to correctly

    857          * handle.

    858          */

    859         SCSI_LOG_HLCOMPLETE(1, printk("%ld sectors total, "

    860                                       "%d bytes done./n",

    861                                       req->nr_sectors, good_bytes));

    862         SCSI_LOG_HLCOMPLETE(1, printk("use_sg is %d/n", cmd->use_sg));

    863

    864          if (clear_errors)

    865                 req->errors = 0;

    866

    867         /* A number of bytes were successfully read.  If there

    868          * are leftovers and there is some kind of error

    869          * (result != 0), retry the rest.

    870          */

871         if (scsi_end_request(cmd, 1, good_bytes, result == 0) == NULL)

    872                 return;

    873

    874         /* good_bytes = 0, or (inclusive) there were leftovers and

    875          * result = 0, so scsi_end_request couldn't retry.

    876          */

    877         if (sense_valid && !sense_deferred) {

    878                 switch (sshdr.sense_key) {

    879                 case UNIT_ATTENTION:

    880                         if (cmd->device->removable) {

    881                                 /* Detected disc change.  Set a bit

    882                                  * and quietly refuse further access.

    883                                  */

    884                                 cmd->device->changed = 1;

    885                                 scsi_end_request(cmd, 0, this_count, 1);

    886                                 return;

    887                         } else {

    888                                 /* Must have been a power glitch, or a

    889                                  * bus reset.  Could not have been a

    890                                  * media change, so we just retry the

    891                                  * request and see what happens.

    892                                   */

    893                                 scsi_requeue_command(q, cmd);

    894                                 return;

    895                         }

    896                         break;

    897                 case ILLEGAL_REQUEST:

     898                         /* If we had an ILLEGAL REQUEST returned, then

    899                          * we may have performed an unsupported

    900                          * command.  The only thing this should be

    901                           * would be a ten byte read where only a six

    902                          * byte read was supported.  Also, on a system

    903                          * where READ CAPACITY failed, we may have

    904                          * read past the end of the disk.

    905                          */

    906                         if ((cmd->device->use_10_for_rw &&

    907                             sshdr.asc == 0x20 && sshdr.ascq == 0x00) &&

    908                             (cmd->cmnd[0] == READ_10 ||

    909                              cmd->cmnd[0] == WRITE_10)) {

    910                                 cmd->device->use_10_for_rw = 0;

    911                                 /* This will cause a retry with a

912                                  * 6-byte command.

    913                                  */

    914                                 scsi_requeue_command(q, cmd);

    915                                 return;

    916                         } else {

    917                                 scsi_end_request(cmd, 0, this_count, 1);

    918                                 return;

    919                         }

    920                         break;

    921                 case NOT_READY:

    922                         /* If the device is in the process of becoming

    923                          * ready, or has a temporary blockage, retry.

    924                          */

    925                         if (sshdr.asc == 0x04) {

    926                                 switch (sshdr.ascq) {

    927                                 case 0x01: /* becoming ready */

    928                                 case 0x04: /* format in progress */

    929                                 case 0x05: /* rebuild in progress */

    930                                  case 0x06: /* recalculation in progress */

    931                                 case 0x07: /* operation in progress */

    932                                 case 0x08: /* Long write in progress */

    933                                 case 0x09: /* self test in progress */

    934                                         scsi_requeue_command(q, cmd);

    935                                         return;

    936                                 default:

    937                                          break;

    938                                 }

    939                         }

    940                         if (!(req->cmd_flags & REQ_QUIET)) {

    941                                 scmd_printk(KERN_INFO, cmd,

    942                                              "Device not ready: ");

    943                                 scsi_print_sense_hdr("", &sshdr);

    944                         }

    945                         scsi_end_request(cmd, 0, this_count, 1);

    946                          return;

    947                 case VOLUME_OVERFLOW:

    948                         if (!(req->cmd_flags & REQ_QUIET)) {

    949                                 scmd_printk(KERN_INFO, cmd,

    950                                             "Volume overflow, CDB: ");

    951                                 __scsi_print_command(cmd->cmnd);

    952                                 scsi_print_sense("", cmd);

    953                         }

    954                         /* See SSC3rXX or current. */

    955                         scsi_end_request(cmd, 0, this_count, 1);

    956                         return;

    957                 default:

    958                         break;

    959                 }

    960         }

    961         if (host_byte(result) == DID_RESET) {

    962                 /* Third party bus reset or reset for error recovery

    963                  * reasons.  Just retry the request and see what

    964                  * happens.

    965                  */

    966                  scsi_requeue_command(q, cmd);

    967                 return;

    968         }

    969         if (result) {

    970                 if (!(req->cmd_flags & REQ_QUIET)) {

    971                         scsi_print_result(cmd);

    972                          if (driver_byte(result) & DRIVER_SENSE)

    973                                 scsi_print_sense("", cmd);

    974                 }

    975         }

    976         scsi_end_request(cmd, 0, this_count, !result);

    977 }

又是一个令人发指的函数 . 但我什么都不想多说了 . 直接跳到最后一行 ,scsi_end_request(). 来自 drivers/scsi_lib.c:

    632 /*

    633  * Function:    scsi_end_request()

    634  *

    635  * Purpose:     Post-processing of completed commands (usually invoked at end

    636  *              of upper level post-processing and scsi_io_completion).

    637  *

    638  * Arguments:   cmd      - command that is complete.

    639  *              uptodate - 1 if I/O indicates success, <= 0 for I/O error.

    640  *              bytes    - number of bytes of completed I/O

    641  *              requeue  - indicates whether we should requeue leftovers.

    642  *

    643  * Lock status: Assumed that lock is not held upon entry.

    644  *

    645  * Returns:     cmd if requeue required, NULL otherwise.

    646  *

    647  * Notes:       This is called for block device requests in order to

    648  *              mark some number of sectors as complete.

    649  *

    650  *              We are guaranteeing that the request queue will be goosed

    651  *               at some point during this call.

    652  * Notes:       If cmd was requeued, upon return it will be a stale pointer.

    653  */

    654 static struct scsi_cmnd *scsi_end_request(struct scsi_cmnd *cmd, int uptodate,

    655                                            int bytes, int requeue)

    656 {

    657         request_queue_t *q = cmd->device->request_queue;

    658         struct request *req = cmd->request;

    659         unsigned long flags;

    660

    661         /*

    662          * If there are blocks left over at the end, set up the command

    663          * to queue the remainder of them.

    664          */

    665         if (end_that_request_chunk(req, uptodate, bytes)) {

    666                 int leftover = (req->hard_nr_sectors << 9);

    667

    668                 if (blk_pc_request(req))

    669                         leftover = req->data_len;

    670

    671                 /* kill remainder if no retrys */

    672                 if (!uptodate && blk_noretry_request(req))

673                         end_that_request_chunk(req, 0, leftover);

    674                 else {

    675                         if (requeue) {

    676                                 /*

    677                                  * Bleah.  Leftovers again.  Stick the

    678                                  * leftovers in the front of the

    679                                  * queue, and goose the queue again.

    680                                  */

    681                                 scsi_requeue_command(q, cmd);

    682                                 cmd = NULL;

    683                         }

    684                         return cmd;

    685                 }

    686         }

    687

    688         add_disk_randomness(req->rq_disk);

     689

    690         spin_lock_irqsave(q->queue_lock, flags);

    691         if (blk_rq_tagged(req))

    692                 blk_queue_end_tag(q, req);

    693         end_that_request_last(req, uptodate);

    694         spin_unlock_irqrestore(q->queue_lock, flags);

    695

    696         /*

    697          * This will goose the queue request function at the end, so we don't

    698          * need to worry about launching another command.

    699          */

    700         scsi_next_command(cmd);

     701         return NULL;

    702 }

而我们最需要关心的 , 是 693 行 end_that_request_last.

   3618 /*

   3619  * queue lock must be held

   3620  */

   3621 void end_that_request_last(struct request *req, int uptodate)

   3622 {

   3623         struct gendisk *disk = req->rq_disk;

   3624         int error;

   3625

   3626         /*

   3627          * extend uptodate bool to allow < 0 value to be direct io error

   3628          */

   3629         error = 0;

   3630         if (end_io_error(uptodate))

   3631                  error = !uptodate ? -EIO : uptodate;

   3632

   3633         if (unlikely(laptop_mode) && blk_fs_request(req))

   3634                 laptop_io_completion();

   3635

   3636         /*

   3637          * Account IO completion.  bar_rq isn't accounted as a normal

   3638          * IO on queueing nor completion.  Accounting the containing

   3639          * request is enough.

   3640          */

   3641         if (disk && blk_fs_request(req) && req != &req->q->bar_rq) {

   3642                 unsigned long duration = jiffies - req->start_time;

   3643                 const int rw = rq_data_dir(req);

   3644

   3645                 __disk_stat_inc(disk, ios[rw]);

   3646                 __disk_stat_add(disk, ticks[rw], duration);

   3647                  disk_round_stats(disk);

   3648                 disk->in_flight--;

   3649         }

   3650         if (req->end_io)

   3651                 req->end_io(req, error);

   3652         else

   3653                 __blk_put_request(req->q, req);

   3654 }

好了 ,3651 行这个 end_io 是最关键的代码 . 也许你早已忘记我们曾经见过 end_io, 但是不要紧 , 有我在 . 在 blk_execute_rq_nowait() 中 , 曾经有一行

   2596         rq->end_io = done;

而 done 是这个函数的第四个参数 . 当初我们在调用这个函数的时候 , 在 blk_execute_rq 中 , 我们是这样写的 :

   2636         blk_execute_rq_nowait(q, bd_disk, rq, at_head, blk_end_sync_rq);

也就是说 ,rq->end_io 被赋上了 blk_end_sync_rq.

   2786 /**

   2787  * blk_end_sync_rq - executes a completion event on a request

   2788  * @rq: request to complete

   2789  * @error: end io status of the request

   2790  */

   2791 void blk_end_sync_rq(struct request *rq, int error)

   2792 {

   2793         struct completion *waiting = rq->end_io_data;

   2794

   2795         rq->end_io_data = NULL;

   2796         __blk_put_request(rq->q, rq);

   2797

   2798         /*

   2799          * complete last, if this is a stack request the process (and thus

   2800          * the rq pointer) could be invalid right after this complete()

   2801          */

   2802         complete(waiting);

   2803 }

终于我们找到了亲爱的可爱的相爱的深爱的最爱的 complete(). 那么如何确定此 waiting 就是彼 wait 呢 ? 对照一下这个 waiting, 当时在 blk_execute_rq 中我们有 :

   2635         rq->end_io_data = &wait;

而眼下我们又有 :

   2793         struct completion *waiting = rq->end_io_data;

由此可知我们没有搞错对象 , 毕竟我们深知 , 接吻可以搞错对象 , 发脾气则不可以 , 写代码则更加不可以 .

至此 ,blk_execute_rq 被唤醒 , 然后迅速返回 . 紧随其后的是 scsi_execute 的返回和 scsi_execute_req 的返回 . 这一刻 , 一个 scsi 命令终于从无到有最终到有 , 它经历了 scsi 命令到 request 的蜕变 , 也经历了 request 到 scsi 命令的历练 . 最终它完成了它的使命 . 对它来说 , 生命是一场幻觉 , 别离或者死亡是唯一的结局 .