调试遗漏IoStartNextPacket引起的阻塞

最新推荐文章于 2023-06-04 11:45:14 发布

Eugene800

最新推荐文章于 2023-06-04 11:45:14 发布

阅读量694

点赞数

分类专栏： win驱动调试设备驱动器之卷文章标签： windows 调试

本文链接：https://blog.csdn.net/lixiangminghate/article/details/53606453

版权

调试同时被 3 个专栏收录

122 篇文章 7 订阅

订阅专栏

器之卷

68 篇文章 2 订阅

订阅专栏

win驱动

45 篇文章 7 订阅

订阅专栏

前面 driver verifier检测驱动死锁一文中本想检测一下驱动中潜在的死锁来解决驱动无响应的bug，然而并没有实质性的进展。后来通过一系列的调试终于找到了根源所在，本文用于记录查找问题的过程。

下面是测试程序，主线程循环读驱动，子线程循环写驱动，由于是异步操作，每次读写结束后，线程都要调用WaitForSingleObject等待驱动完成IRP。可是由于驱动处理有误，ReadFile调用2次后，主线程就彻底卡死在Wait函数上了:

	hDev = CreateFileA(interfaceBuff, 
				GENERIC_ALL, 
				FILE_SHARE_READ | FILE_SHARE_WRITE,
				NULL, OPEN_EXISTING,
				FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED, 
				NULL);

	InitializeCriticalSection(&cs);

	CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)ThreadProc, NULL, 0, &threadId);

	while (1)
	{
		memset(readBuff, 0, 4096);
		BOOL bRead = ReadFile(hDev, readBuff, 4096, &readLen, &overlapRd); 1)
		WaitForSingleObject(overlapRd.hEvent, INFINITE);                   2)

		if (readLen)
		{
			EnterCriticalSection(&cs);
			printf("_tmain read:%s\n",readBuff);
			LeaveCriticalSection(&cs);
		}
			
	}

	CloseHandle(hDev);
	return 0;
}

DWORD ThreadProc(void* param)
{
	OVERLAPPED overlapRd = { 0 },overlapWr = {0};
	overlapRd.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
	overlapWr.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);

	char writeBuff[4096] = { "_ThreadProc" }, readBuff[4096] = { 0 };
	DWORD len = 0, writeLen, readLen;

	while (1)
	{
		WriteFile(hDev, writeBuff, strlen(writeBuff), &writeLen, &overlapWr);
		WaitForSingleObject(overlapWr.hEvent, INFINITE);

	}
}

对于异步IRP，调用ReadFile会传入OVERLAPPED结构，之后WaitForSingleObject会在OVERLAPPED!HEVENT上等待。当驱动中完成读请求(调用IoCompleteRequest)时，会调用KeSetEvnet(irp->UserEvent);使得用户层的等待函数返回。基于上述分析，可以推知，ReadFile的IRP请求被挂起后一直没有被完成(悬而未决的IRP请求)。为了验证我的猜测，可以查看一下线程的IRP请求:

1.首先查看R3下线程ID，当前进程有3个线程，0/1分别对应读写线程，其ID号分别为:75c.3a8/75c.828;2号线程可能是windbg调试线程

0:002> ~*kb ;注意这里的0:002> 暗示当前是在R3下调试

   0  Id: 75c.3a8 Suspend: 1 Teb: 7ffdf000 Unfrozen
ChildEBP RetAddr  Args to Child              
0042d53c 77165e6c 754f179c 0000007c 00000000 ntdll!KiFastSystemCallRet
0042d540 754f179c 0000007c 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
0042f7d0 001b400a 00000001 005f96d0 005f9738 onlyForWrite!main+0x22b [c:\studio\samplechar\onlyforwrite\onlyforwrite\testsamplechar.cpp @ 173]

   1  Id: 75c.828 Suspend: 1 Teb: 7ffde000 Unfrozen
ChildEBP RetAddr  Args to Child              
01c7db38 77165e6c 754f179c 00000090 00000000 ntdll!KiFastSystemCallRet
01c7db3c 754f179c 00000090 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
01c7fd20 76ef1174 00000000 01c7fd6c 7717b3f5 onlyForWrite!ThreadProc+0x139 [c:\studio\samplechar\onlyforwrite\onlyforwrite\testsamplechar.cpp @ 200]

#  2  Id: 75c.8dc Suspend: 1 Teb: 7ffdd000 Unfrozen
ChildEBP RetAddr  Args to Child              
007efa94 771bd279 7764a5c0 00000000 00000000 ntdll!DbgBreakPoint
007efac4 76ef1174 00000000 007efb10 7717b3f5 ntdll!DbgUiRemoteBreakin+0x3c

2.利用上面的线程号，在R0 windbg上查看线程挂起的IRP:

kd> !process 0 0 ;查找所有进程  注意这里调试开头是kd>
**** NT ACTIVE PROCESS DUMP ****
PROCESS 89a4bd40  SessionId: 1  Cid: 075c    Peb: 7ffd4000  ParentCid: 0f8c
    DirBase: 7ff7e480  ObjectTable: 8a2a16d0  HandleCount:  57.
    Image: onlyForWrite.exe ;找到onlyForWrite.exe进程的信息


kd> !process 89a4bd40 6 ;查找进程onlyForWrite.exe下所有线程
PROCESS 89a4bd40  SessionId: 1  Cid: 075c    Peb: 7ffd4000  ParentCid: 0f8c
    DirBase: 7ff7e480  ObjectTable: 8a2a16d0  HandleCount:  57.
    Image: onlyForWrite.exe

        THREAD 8a927d48  Cid 075c.03a8  Teb: 7ffdf000 Win32Thread: fd3c7dd8 WAIT: (Suspended) KernelMode Non-Alertable
SuspendCount 1
FreezeCount 1
            8a927f10  Semaphore Limit 0x2
        IRP List: ;这个线程有IRP队列
            8773eef8: (0006,0100) Flags: 00060970  Mdl: 00000000
        

        THREAD 8b3a37d8  Cid 075c.0828  Teb: 7ffde000 Win32Thread: 00000000 WAIT: (Suspended) KernelMode Non-Alertable
SuspendCount 1
FreezeCount 1
            8b3a39a0  Semaphore Limit 0x2
        Not impersonating
        DeviceMap                 946136a0
        Owning Process            89a4bd40       Image:         onlyForWrite.exe
        Attached Process          N/A            Image:         N/A
        

        THREAD 8b328cd8  Cid 075c.08dc  Teb: 7ffdd000 Win32Thread: 00000000 WAIT: (Executive) KernelMode Non-Alertable
SuspendCount 1
            92ddd6e0  SynchronizationEvent
        Not impersonating
        DeviceMap                 946136a0
        Owning Process            89a4bd40       Image:         onlyForWrite.exe
        Attached Process          N/A            Image:         N/A

对比windbg输出的3个线程信息，发现唯独Cid==075c.03a8线程有IRP队列。结合R3 windbg中查到的读线程ID:0x3a8，可以确定读线程的确出了什么问题。再进一步确认IRP是不是应用层发出的:

kd> !irp 8773eef8 ；8773ee8从前面THREAD 8a927d48的IRP List中获取
Irp is active with 2 stacks 2 is current (= 0x8773ef8c)
 No Mdl: System buffer=8776e000: Thread 8a927d48:  Irp stack trace.  
     cmd  flg cl Device   File     Completion-Context
 [N/A(0), N/A(0)]
            0  0 00000000 00000000 00000000-00000000    

			Args: 00000000 00000000 00000000 00000000
>[IRP_MJ_READ(3), N/A(0)]
            0  1 8a7762b8 876c7988 00000000-00000000    pending -->irp状态是挂起的 
	       \Driver\SampleChar
			Args: 00001000 00000000 00000000 00000000
kd> da 8776e000
8776e000  "................................"
8776e020  "................................"

果不其然，这是程序发出的IRP请求未被处理。至少可以肯定应该是驱动中StartIo函数有错误了。

重启电脑后，再次运行测试程序，在驱动程序读派遣函数和StartIo函数中下断点:

#pragma code_seg()
void SampleStartIo(PDEVICE_OBJECT devObj, PIRP irp)
{
	KEVENT workEvt, completeEvt;
	KIRQL origIrql;
	NTSTATUS status = STATUS_SUCCESS;
	unsigned long readLen;
	SampleCharDevContext* devCtx = (SampleCharDevContext*)devObj->DeviceExtension;
	IO_STACK_LOCATION* curStack = IoGetCurrentIrpStackLocation(irp);

	KeInitializeEvent(&workEvt,SynchronizationEvent,FALSE);
	KeInitializeEvent(&completeEvt, NotificationEvent, FALSE);

	if (curStack->Parameters.Read.Length > 4096)
	{
		status = irp->IoStatus.Status = STATUS_BUFFER_OVERFLOW;
		irp->IoStatus.Information = 0;

		IoCompleteRequest(irp, IO_NO_INCREMENT);
		IoStartNextPacket(devObj, FALSE);
		return;
	}

	KeAcquireSpinLock(&devCtx->devSpinLock, &origIrql);
	
	if(devCtx->buffPos != 0x00UL)
	{
		readLen = devCtx->buffPos >= curStack->Parameters.Read.Length ? curStack->Parameters.Read.Length : devCtx->buffPos;
		RtlCopyMemory(irp->AssociatedIrp.SystemBuffer,
			devCtx->SampleBuff,
			readLen);
		devCtx->buffRemained += readLen;
		devCtx->buffPos -= readLen;
	}
	else
	{
		KeReleaseSpinLock(&devCtx->devSpinLock, origIrql);
		
		irp->IoStatus.Status = STATUS_SUCCESS;
		irp->IoStatus.Information = 0x00UL;
		IoCompleteRequest(irp, IO_NO_INCREMENT); ->1)
		//maybe blocked at here,if without this
		//IoStartNextPacket(devObj, FALSE);  ->4)
		return; ->2)
	}

	KeReleaseSpinLock(&devCtx->devSpinLock, origIrql);

	IoCopyCurrentIrpStackLocationToNext(irp);
	IoSetCompletionRoutine(irp, IrpAsyncReadCompleteRoutine, &completeEvt, TRUE, TRUE, TRUE);
	
	status = IoCallDriver(devCtx->lowerDev,irp);
	if (status == STATUS_PENDING)
	{
		KeWaitForSingleObject(&completeEvt, Executive, KernelMode, FALSE, NULL);
	}
	
	irp->IoStatus.Status = STATUS_SUCCESS;
	irp->IoStatus.Information = readLen;
	IoCompleteRequest(irp, IO_NO_INCREMENT);

	IoStartNextPacket(devObj,FALSE);
}

NTSTATUS SampleCharReadAsync(PDEVICE_OBJECT devObj, PIRP irp)
{
	IoMarkIrpPending(irp);
	IoStartPacket(devObj,irp,NULL,NULL); ->3)
	return STATUS_PENDING;
}

第一次调用ReadFile时，驱动依次经过代码中标注3)->1)->2)的顺序处理IRP；当第二次调用ReadFile时，程序只经历了步骤3)，之后StartIo就再也没有被调用。给我的感觉是设备默认的IRP队列出问题了。检查一下设备队列的情况:

kd> !devobj 8a7762b8 -------------->8a7762b8是前面!irp 命令输出的irp相关的devobj
Device object (8a7762b8) is for:
 SampleChar \Driver\SampleChar DriverObject 8774cb18
Current Irp 8773eef8 RefCount 0 Type 00000022 Flags 0000204c
Dacl 8fb049c0 DevExt 8a776370 DevObjExt 8a7763f0 
ExtensionFlags (0x30000000)  DOE_RAW_FDO
                             Unknown flags 0x10000000
Characteristics (0x00000100)  FILE_DEVICE_SECURE_OPEN
AttachedTo (Lower) 8761a400 \Driver\PnpManager
Device queue is busy - Queue flink = blink  ------->windbg显示设备队列忙

结合wrk源码，只有当设备队列为空闲时，Io管理器会将设备队列中的IRP出队交给StartIo处理。而调用IoStartPacket会置设备队列为Busy；调用IoStartNextPacket会置设备队列为Free。这么分析下来，就是说我的代码中少做一次IoStartNextPacket，进一步说，前一次处理IRP后，漏调用IoStartNextPacket....基于这样的猜测，错误就明朗了，只要把代码中标注为4)的位置前的注释符去掉，重新编译即可。再次验证，果然解决了驱动没有响应的情况。