Documentation/scsi/scsi_eh.txt
scmd为scsi cmd的简称。
scsi_eh_scmd_add()作用:将发生error的scmd加入到eh中。
发生error有两种情况:
1).scmd执行完成了,但是结果为error。
2).scmd执行超时了,即一直没有返回,发生了timeout。
scsi_eh_scmd_add()在两处被调用:
1).scsi_times_out函数 => scmd超时处理函数,将timeout的scmd加入eh2).scsi_softirp_done函数 => scmd命令结果处理函数,将结果为error的scmd加入eh
scsi_eh_scmd_add()完成以下工作:
1. Turns on scmd->eh_eflags as requested. It's 0 for error
completions and SCSI_EH_CANCEL_CMD for timeouts.
2. Links scmd->eh_entry to shost->eh_cmd_q
3. Sets SHOST_RECOVERY bit in shost->shost_state
4. Increments shost->host_failed
5. Wakes up SCSI EH thread if shost->host_busy == shost->host_failed
scsi-high层
中间层完成命令时通过调用scmd->done()通知scsi-high层SCSI命令处理完毕。
HLD completion callback - sd:sd_rw_intr, sr:rw_intr, st:st_intr.
scsi-mid层
1). LLDD顺利完成SCSI命令 => 调用scsi-mid层提供的回调函数scsi_done()。
2). LLDD没能顺利完成SCSI命令 => scsi-mid层将该scmd time out,调用scsi_times_out()处理该命令。
LLDD层
SCSI EH如何工作
LLDD可以通过以下两种方式实现SCSI EH
- 实现EH回调函数
int (* eh_abort_handler)(struct scsi_cmnd *);
int (* eh_device_reset_handler)(struct scsi_cmnd *);
int (* eh_bus_reset_handler)(struct scsi_cmnd *);
int (* eh_host_reset_handler)(struct scsi_cmnd *);
供scsi_unjam_host调用完成SCSI EH的全部工作。
- 实现eh_strategy_handler()回调函数
transportt->eh_strategy_handler()函数完成全部的EH工作。
英文版
How SCSI EH works
LLDD's can implement SCSI EH actions in one of the following two
ways.
- Fine-grained EH callbacks
LLDD can implement fine-grained EH callbacks and let SCSI
midlayer drive error handling and call appropriate callbacks.
This will be discussed further in [2-1].
- eh_strategy_handler() callback
This is one big callback which should perform whole error
handling. As such, it should do all choirs SCSI midlayer
performs during recovery. This will be discussed in [2-2].
/**
* scsi_error_handler - SCSI error handler thread
* @data: Host for which we are running.
*
* Notes:
* This is the main error handling loop. This is run as a kernel thread
* for every SCSI host and handles all error handling activity.
*/
int scsi_error_handler(void *data)
{
...
/* 如果eh_strategy_handler在LLDD中有实现,则调用该函数处理,
否则调用通用处理函数scsi_unjam_host进行处理,scsi_unjam_host
会通过调用LLDD定义的EH回调函数们来完成工作。
*/
if (shost->transportt->eh_strategy_handler)
shost->transportt->eh_strategy_handler(shost);
else
scsi_unjam_host(shost);
...
}
/**
* scsi_host_alloc - register a scsi host adapter instance.
* @sht: pointer to scsi host template
* @privsize: extra bytes to allocate for driver
*
* Note:
* Allocate a new Scsi_Host and perform basic initialization.
* The host is not published to the scsi midlayer until scsi_add_host
* is called.
*
* Return value:
* Pointer to a new Scsi_Host
**/
struct Scsi_Host *scsi_host_alloc(struct scsi_host_template *sht, int privsize)
{
...
shost->ehandler = kthread_run(scsi_error_handler, shost, /* scsi_error_handler在scsi_host_alloc函数中被指定 */
"scsi_eh_%d", shost->host_no);
...
}
static void scsi_unjam_host(struct Scsi_Host *shost)
{
unsigned long flags;
LIST_HEAD(eh_work_q);
LIST_HEAD(eh_done_q);
spin_lock_irqsave(shost->host_lock, flags);
list_splice_init(&shost->eh_cmd_q, &eh_work_q);
spin_unlock_irqrestore(shost->host_lock, flags);
SCSI_LOG_ERROR_RECOVERY(1, scsi_eh_prt_fail_stats(shost, &eh_work_q));
/* scsi_eh_get_sense函数最后调用list_empty(eh_work_q),如果为空,则返回1,否则返回0,
继续执行更高级别的EH。即如果 scsi_eh_get_sense函数能解决掉所有eh_work_q中的scmd,
则不许要下一步调用,否则,继续进行更高级别的EH。
*/
if (!scsi_eh_get_sense(&eh_work_q, &eh_done_q))
if (!scsi_eh_abort_cmds(&eh_work_q, &eh_done_q))
scsi_eh_ready_devs(shost, &eh_work_q, &eh_done_q);
scsi_eh_flush_done_q(&eh_done_q);
}
对于error-completed (completed but result is error) scmd,调用scsi_eh_get_sense将它们结束
对于timed-out scmd,
a).调用scsi_eh_abort_cmds让LLDD忽略这个scmd
b).调用scsi_eh_stu 对每个device,执行一遍START_UNIT命令,
1). First call scsi_eh_get_sense
<<scsi_eh_get_sense>> /** 处理error-completed scmd,将它们连同sence data返回给scsi-high层处理 **/
该函数是为那些error-completed(complted,但是结果有问题)的scmd准备的。
如果所有scmd都是error-completed的,且成功获得了sense data,scsi_eh_finish_cmd()被调用将该scmd
移到eh_done_q,不需要进一步操作,EH完成;
否则经过scsi_eh_get_sense函数之后,eh_work_q链表仍然非空(即还有无法处理的scmd)则进行更高级别的EH。
标记为 SCSI_EH_CANCEL_CMD的scmd是因为time out而进入EH链表的,不是scsi_eh_get_sense函数处理对象。
2). If !list_empty(&eh_work_q), invoke scsi_eh_abort_cmds().
<<scsi_eh_abort_cmds>>/** 处理timed-out scmd,让底层LLDD abort(终止)这些scmd **/
该函数是为那些timed-out的scmd准备的。
调用hostt->eh_abort_handler()函数依次处理每一个scmd,如果该函数成功地让LLDD和相关的硬件忽略了这个scmd,
则成功返回SUCCESS,。
如果一个timed-out scmd被成功地终止掉了,那么这个scmd对应的device就应该处于offline或者ready状态。
根据scsi_try_to_abort_cmd函数的返回值移动scmd
返回值:
a).FAST_IO_FAIL,则调用scsi_eh_finish_cmd()将该scmd 移到eh_done_q;
b).SUCCESS,则进行检查,如果scmd对应的device是offline或者ready状态,则调用scsi_eh_finish_cmd()
将它移到eh_done_q,否则不移动。
c).否则不移动scmd,它仍然保留在eh_work_q中
如果有不成功的scmd仍然留在eh_work_q链表中,则调用更高级别的EH。
3). If !list_empty(&eh_work_q), invoke scsi_eh_ready_devs()
<<scsi_eh_ready_devs>> /** **/
void scsi_eh_ready_devs(struct Scsi_Host *shost,
struct list_head *work_q,
struct list_head *done_q)
{
if (!scsi_eh_stu(shost, work_q, done_q))
if (!scsi_eh_bus_device_reset(shost, work_q, done_q))
if (!scsi_eh_target_reset(shost, work_q, done_q))
if (!scsi_eh_bus_reset(shost, work_q, done_q))
if (!scsi_eh_host_reset(work_q, done_q))
scsi_eh_offline_sdevs(work_q,
done_q);
}
<<scsi_eh_stu>>
每个device,执行一遍START_UNIT命令,然后检查执行结果。
如果device是offline或者ready状态,则调用scsi_eh_finish_cmd()将它关联的scmd都移动到eh_done_q,否则不移动。
<<scsi_eh_test_devices>>
测试device状态:offline、online、online && ready
/**
* scsi_eh_test_devices - check if devices are responding from error recovery.
* @cmd_list: scsi commands in error recovery.
* @work_q: queue for commands which still need more error recovery
* @done_q: queue for commands which are finished
* @try_stu: boolean on if a STU command should be tried in addition to TUR.
*
* Decription:
* Tests if devices are in a working state. Commands to devices now in
* a working state are sent to the done_q while commands to devices which
* are still failing to respond are returned to the work_q for more
* processing.
**/
static int scsi_eh_test_devices(struct list_head *cmd_list,
struct list_head *work_q,
struct list_head *done_q, int try_stu)
{
struct scsi_cmnd *scmd, *next;
struct scsi_device *sdev;
int finish_cmds;
while (!list_empty(cmd_list)) {
scmd = list_entry(cmd_list->next, struct scsi_cmnd, eh_entry);
sdev = scmd->device;
/* 1.if scsi_device_online() return value is 0, means the device is offline, then the things
after || don't need to be done.
2.if scsi_device_online() return value is 1, means the device is online, then the things
after || need to be done to juge if the device is in a ready state.
3.the way to test if device is in ready state is to call function scsi_eh_tur() to
send TEST_UNIT_READY cmd to the device and see the return value.
*/
finish_cmds = !scsi_device_online(scmd->device) ||
(try_stu && !scsi_eh_try_stu(scmd) &&
!scsi_eh_tur(scmd)) ||
!scsi_eh_tur(scmd);
list_for_each_entry_safe(scmd, next, cmd_list, eh_entry)
if (scmd->device == sdev) {
if (finish_cmds)
scsi_eh_finish_cmd(scmd, done_q);
else
list_move_tail(&scmd->eh_entry, work_q);
}
}
return list_empty(work_q);
}
static void scsi_unjam_host(struct Scsi_Host *shost)
{
unsigned long flags;
LIST_HEAD(eh_work_q);
LIST_HEAD(eh_done_q);
spin_lock_irqsave(shost->host_lock, flags);
list_splice_init(&shost->eh_cmd_q, &eh_work_q); /*将eh_cmd_q中的scmd复制到eh_work_q中*/
spin_unlock_irqrestore(shost->host_lock, flags);
SCSI_LOG_ERROR_RECOVERY(1, scsi_eh_prt_fail_stats(shost, &eh_work_q));
/*eh_work_q中的scmd分为两类:
1.scmd执行完成了,但是执行结果为error的。即errored scmd。
2.scmd执行了就没有返回,知道超时发生time out,即timed-out scmd。
*/
/*1.调用scsi_eh_get_sense()从errored scmd的目标设备处获取sense信息。
2.sense信息用途:作为上层驱动判断scmd发生error的原因的依据。
3.然后将errored scmd处理掉(即加入eh_done_q并调用finish函数将scmd返回上层)。
4.eh_done_q中剩余的全部都是timed-out scmd。
*/
if (!scsi_eh_get_sense(&eh_work_q, &eh_done_q))
/*1.对eh_work_q中剩余的timed-out scmd,执行abort操作(abort,放弃执行),
2.如果abort之后,eh_work_q还不为空,即还有没处理掉的timed-out scmd,
调用scsi_eh_ready_devs开始逐步升级的reset操作,直到eh_work_q中的scmd处理光。
*/
if (!scsi_eh_abort_cmds(&eh_work_q, &eh_done_q))
scsi_eh_ready_devs(shost, &eh_work_q, &eh_done_q);
scsi_eh_flush_done_q(&eh_done_q);
}
static int scsi_eh_abort_cmds(struct list_head *work_q,
struct list_head *done_q)
{
struct scsi_cmnd *scmd, *next;
LIST_HEAD(check_list);
int rtn;
/*循环对每个scmd执行一遍abort,放弃该命令的执行*/
list_for_each_entry_safe(scmd, next, work_q, eh_entry) {
if (!(scmd->eh_eflags & SCSI_EH_CANCEL_CMD))
continue;
SCSI_LOG_ERROR_RECOVERY(3, printk("%s: aborting cmd:"
"0x%p\n", current->comm,
scmd));
/* 执行对当前scmd的abort */
rtn = scsi_try_to_abort_cmd(scmd->device->host->hostt, scmd);
/* 如果执行结果为SUCCESS,或者期望通过将IO设置为FAIL快速返回,则进入 */
if (rtn == SUCCESS || rtn == FAST_IO_FAIL) {
scmd->eh_eflags &= ~SCSI_EH_CANCEL_CMD;
if (rtn == FAST_IO_FAIL)
scsi_eh_finish_cmd(scmd, done_q);/*将返回FAST_IO_FAIL的scmd移动到done_q从而finish掉*/
else
list_move_tail(&scmd->eh_entry, &check_list); /*将返回SUCCESS的scmd加入check_list*/
} else
SCSI_LOG_ERROR_RECOVERY(3, printk("%s: aborting"
" cmd failed:"
"0x%p\n",
current->comm,
scmd));
}
/*调用scsi_eh_test_devices()函数,检查返回SUCCESS的scmd们(都保存在check_list中)的目标设备的状态。
如果一个scmd的目标设备的状态要么为offline,要么为ready,则说明对这个scmd的abort操作成功了,
可以将这个scmd移动到done_q中finish掉这个scmd了。
*/
return scsi_eh_test_devices(&check_list, work_q, done_q, 0);
}
static int scsi_eh_test_devices(struct list_head *cmd_list,
struct list_head *work_q,
struct list_head *done_q, int try_stu)
{ /*该函数的改进在于如果一个scmd的目标设备处于offline或者ready,则把check_list中的其他具有该目标设备
的scmd也移动到done_q中finish掉。而不是每个scmd都检测一遍它的目标设备,提高了效率。
即以sdev为单位处理check_list中的scmd,而不是以scmd为单位挨个执行,做很多重复工作。
*/
struct scsi_cmnd *scmd, *next;
struct scsi_device *sdev;
int finish_cmds;
while (!list_empty(cmd_list)) {
scmd = list_entry(cmd_list->next, struct scsi_cmnd, eh_entry);
sdev = scmd->device;
/* 1.if scsi_device_online return value is 0, means the device is offline, then the things
after || don't need to be done.
2.if scsi_device_online return value is 1, means the device is online, then the things
after || need to be done to juge if the device is in a ready state.
3.the way to test if device is in ready state is to call function scsi_eh_tur to
send TEST_UNIT_READY cmd to the device and see the return value.
*/
finish_cmds = !scsi_device_online(scmd->device) ||
(try_stu && !scsi_eh_try_stu(scmd) &&
!scsi_eh_tur(scmd)) ||
!scsi_eh_tur(scmd);
/*到check_list中检测目标设备是上面的sdev的scmd,将它们都处理掉,提高效率。*/
list_for_each_entry_safe(scmd, next, cmd_list, eh_entry)
if (scmd->device == sdev) {
if (finish_cmds)
scsi_eh_finish_cmd(scmd, done_q);
else
list_move_tail(&scmd->eh_entry, work_q);
}
}
return list_empty(work_q);
}