https://www.freesion.com/article/9518152212/
了解一下什么叫总线模型(这是Linux下的概念,但是感觉和windows有相通之处),总线负责连接驱动和设备,通俗的来说就是总线提供接口给驱动,所谓接口就是设备的一些基本操作,驱动负责实现设备逻辑。比如:驱动读取设备数据时,先发地址再发命令,就可以调用总线提供的接口来实现这一简单逻辑。
Windows下提供接口的功能叫框架,当然框架比总线功能更强大,框架不仅提供接口,还规范了驱动编写的逻辑,框架会自动调用驱动里的特定函数。
1, Windows下有两种驱动框架,本文只涉及KWDF框架。
驱动入口函数 NTSTATUS DriverEntry(IN PDRIVER_OBJECT driverObject, IN PUNICODE_STRING registryPath);
入口函数完成两个任务:
1) 创建驱动对象(WdfDriverCreate())
2) 配置EvtDriverDeviceAdd函数(WDF_DRIVER_CONFIG_INIT())
当驱动被加载到操作系统时会执行入口函数,当windows枚举硬件设备时会调用驱动注册的EvtDriverDeviceAdd函数.
所谓硬件枚举,是系统识别硬件的过程,识别过程中需要按照相应通讯协议获取设备描述符,来确定是否为可识别设备.
2, EvtDeviceAdd函数完成一系列的设备回调函数设置.
EvtDeviceAdd函数主要任务:
1) 设备识别后进行硬件初始化(即注册EvtDevicePrepareHardware回调函数)
2) windows下应用层打开设备函数为,CreateFile(类似于Linux下的open),所以要注册相应驱动回调函数(EvtDeviceFileCreate)
3) 创建设备对象(即打开哪个设备)
4) 设置设备接口也就是应用程序找到此驱动的路径
5) 应用层调用ReadFile/WriteFile函数读取/写入数据,所有要注册相应的回调函数(EvtIoRead/EvtIoWrite)
6) 创建请求队列(即读写设备的请求需要先挂接到队列,然后KWDF调用队列的回调函数处理请求)
看XDMA设备的驱动源码:
NTSTATUS EvtDeviceAdd(IN WDFDRIVER Driver, IN PWDFDEVICE_INIT DeviceInit) {
NTSTATUS status = STATUS_SUCCESS;
PAGED_CODE();
TraceVerbose(DBG_INIT, "(Driver=0x%p)", Driver);
// We prefer Direct I/O
// Direct I/O only works with deferred buffer retrieval No guarantee that Direct I/O is
// actually used Direct I/O is only used for buffers that are full pages Buffered I/O is used
// for other parts of the transfer
WdfDeviceInitSetIoType(DeviceInit, WdfDeviceIoDirect);
//设置设备是否独占
WdfDeviceInitSetExclusive(DeviceInit,TRUE);
// Set call-backs for any of the functions we are interested in. If no call-back is set, the
// framework will take the default action by itself.
//设置即插即用基本例程(硬件映射)
//也就是设备枚举完成之后就会调用EvtDevicePrepareHardware函数
WDF_PNPPOWER_EVENT_CALLBACKS PnpPowerCallbacks;
WDF_PNPPOWER_EVENT_CALLBACKS_INIT(&PnpPowerCallbacks);
PnpPowerCallbacks.EvtDevicePrepareHardware = EvtDevicePrepareHardware;
PnpPowerCallbacks.EvtDeviceReleaseHardware = EvtDeviceReleaseHardware;
WdfDeviceInitSetPnpPowerEventCallbacks(DeviceInit, &PnpPowerCallbacks);
//电源管理回调函数,这里并没有设置回调函数
WDF_POWER_POLICY_EVENT_CALLBACKS powerPolicyCallbacks;
WDF_POWER_POLICY_EVENT_CALLBACKS_INIT(&powerPolicyCallbacks);
WdfDeviceInitSetPowerPolicyEventCallbacks(DeviceInit, &powerPolicyCallbacks);
// Register file object call-backs
//EvtDeviceFileCreate 函数处理应用层的CreateFile函数
WDF_OBJECT_ATTRIBUTES fileAttributes;
WDF_FILEOBJECT_CONFIG fileConfig;
WDF_FILEOBJECT_CONFIG_INIT(&fileConfig, EvtDeviceFileCreate, EvtFileClose, EvtFileCleanup);
WDF_OBJECT_ATTRIBUTES_INIT(&fileAttributes);
fileAttributes.SynchronizationScope = WdfSynchronizationScopeNone;
WDF_OBJECT_ATTRIBUTES_SET_CONTEXT_TYPE(&fileAttributes, FILE_CONTEXT); //为fileAttributes文件对象注册一个上下文,相当于一个私有变量类型
WdfDeviceInitSetFileObjectConfig(DeviceInit, &fileConfig, &fileAttributes);
// Specify the context type and size for the device we are about to create.
WDF_OBJECT_ATTRIBUTES deviceAttributes;
WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&deviceAttributes, DeviceContext);
// ContextCleanup will be called by the framework when it deletes the device. So you can defer
// freeing any resources allocated to Cleanup callback in the event EvtDeviceAdd returns any
// error after the device is created.
deviceAttributes.EvtCleanupCallback = EvtDeviceCleanup;
WDFDEVICE device;
status = WdfDeviceCreate(&DeviceInit, &deviceAttributes, &device);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "WdfDeviceCreate failed: %!STATUS!", status);
return status;
}
// Create a user-space device interface
status = WdfDeviceCreateDeviceInterface(device, (LPGUID)&GUID_DEVINTERFACE_XDMA, NULL);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "WdfDeviceCreateDeviceInterface failed %!STATUS!", status);
return status;
}
// create the default queue upon all I/O requests arrive
// accept multiple I/O request to run in parallel, they are sequentialized later
WDF_IO_QUEUE_CONFIG queueConfig;
WDF_IO_QUEUE_CONFIG_INIT_DEFAULT_QUEUE(&queueConfig, WdfIoQueueDispatchParallel);
//分别用来处理应用层的ReadFile,WriteFile,DeviceIoControl函数
queueConfig.EvtIoDeviceControl = EvtIoDeviceControl; // callback handler for control requests
queueConfig.EvtIoRead = EvtIoRead; // callback handler for read requests
queueConfig.EvtIoWrite = EvtIoWrite; // callback handler for write requests
WDFQUEUE entryQueue;
status = WdfIoQueueCreate(device, &queueConfig, WDF_NO_OBJECT_ATTRIBUTES, &entryQueue);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "WdfIoQueueCreate failed: %!STATUS!", status);
return status;
}
TraceVerbose(DBG_INIT, "returns %!STATUS!", status);
return status;
}
3. EvtDevicePrepareHardware硬件映射函数
硬件映射函数完成硬件初始化,要了解PCIe协议并参考硬件手册pg195-pcie-dma.pdf
XDMA设备C2H和H2C都有最多4个通道,每个通道成为一个engine,跟着源码来分析.
NTSTATUS EvtDevicePrepareHardware(IN WDFDEVICE device, IN WDFCMRESLIST Resources,
IN WDFCMRESLIST ResourcesTranslated) {
PAGED_CODE();
UNREFERENCED_PARAMETER(Resources);
TraceVerbose(DBG_INIT, "-->Entry");
//这个就是在EvtDeviceAdd函数里注册的上下文
DeviceContext* ctx = GetDeviceContext(device);
PXDMA_DEVICE xdma = &(ctx->xdma);
NTSTATUS status = XDMA_DeviceOpen(device, xdma, Resources, ResourcesTranslated);//这里传递的都是指针类型
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "XDMA_DeviceOpen failed: %!STATUS!", status);
return status;
}
// get poll mode parameter and configure engines as poll mode if needed
ULONG pollMode = 0;
status = GetPollModeParameter(&pollMode);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "GetPollModeParameter failed: %!STATUS!", status);
return status;
}
for (UINT dir = H2C; dir < 2; dir++) { // 0=H2C, 1=C2H
for (ULONG ch = 0; ch < XDMA_MAX_NUM_CHANNELS; ch++) {
XDMA_ENGINE* engine = &(xdma->engines[ch][dir]);
XDMA_EngineSetPollMode(engine, (BOOLEAN)pollMode);
}
}
// create a queue for each engine
for (UINT dir = H2C; dir < 2; dir++) { // 0=H2C, 1=C2H
for (ULONG ch = 0; ch < XDMA_MAX_NUM_CHANNELS; ch++) {
XDMA_ENGINE* engine = &(xdma->engines[ch][dir]);
if (engine->enabled == TRUE) {
//为每一个engine创建一个请求队列,
//应用程序的读/写请求在EvtIoRead函数,会将请求分配给相应队列
status = EngineCreateQueue(device, engine, &(ctx->engineQueue[dir][ch]));
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "EngineCreateQueue() failed: %!STATUS!", status);
return status;
}
}
}
}
for (UINT i = 0; i < XDMA_MAX_USER_IRQ; ++i) {
KeInitializeEvent(&ctx->eventSignals[i], NotificationEvent, FALSE);
XDMA_UserIsrRegister(xdma, i, HandleUserEvent, &ctx->eventSignals[i]);
}
TraceVerbose(DBG_INIT, "<--Exit returning %!STATUS!", status);
return status;
}
//硬件相关的初始化,ResourcesTranslated为操作系统分配给XDMA设备的资源表
NTSTATUS XDMA_DeviceOpen(WDFDEVICE wdfDevice,
PXDMA_DEVICE xdma,
WDFCMRESLIST ResourcesRaw,
WDFCMRESLIST ResourcesTranslated) {
NTSTATUS status = STATUS_INTERNAL_ERROR;
DeviceDefaultInitialize(xdma);
xdma->wdfDevice = wdfDevice;
// map PCIe BARs to host memory
//需查看PCIe协议里关于BAR空间的讲解
//映射BAR空间的物理地址到内存xdma->bar[],数量为xdma->numBars,
status = MapBARs(xdma, ResourcesTranslated);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "MapBARs() failed! %!STATUS!", status);
return status;
}
// identify BAR configuration - user(optional), config, bypass(optional)
// 这个需要FPGA去设置,通常设置BAR0为user_bar; BAR1为config_BAR,只用这两个BAR
//
status = IdentifyBars(xdma);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "IdentifyBars() failed! %!STATUS!", status);
return status;
}
// get the module offsets in config BAR
// 看手册29页关于BAR1地址空间的介绍:
// 将配置BAR1的Config、IRQ、SGDMA Common空间地址保存
GetRegisterModules(xdma);
// Confirm XDMA IP core version matches this driver
UINT version = GetVersion(xdma);
if (version != v2017_1) {
TraceWarning(DBG_INIT, "Version mismatch! Expected 2017.1 (0x%x) but got (0x%x)",
v2017_1, version);
}
//创建中断对象
//我们公司只用到了Legend中断模式,关于MSI/MSIx中断模式本文并不涉及
status = SetupInterrupts(xdma, ResourcesRaw, ResourcesTranslated);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "SetupInterrupts failed: %!STATUS!", status);
return status;
}
// WDF DMA Enabler - at least 8 bytes alignment
//地址对齐,(8 - 1)是掩码,地址对齐原理查看Linux源码里的ALIGN宏
WdfDeviceSetAlignmentRequirement(xdma->wdfDevice, 8 - 1); // TODO - choose correct value
//创建一个DMA适配器,它标明一个DMA通道的特性和提供串行化访问的服务
WDF_DMA_ENABLER_CONFIG dmaConfig;
WDF_DMA_ENABLER_CONFIG_INIT(&dmaConfig, WdfDmaProfileScatterGather64Duplex, XDMA_MAX_TRANSFER_SIZE);
status = WdfDmaEnablerCreate(xdma->wdfDevice, &dmaConfig, WDF_NO_OBJECT_ATTRIBUTES, &xdma->dmaEnabler);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, " WdfDmaEnablerCreate() failed: %!STATUS!", status);
return status;
}
// Detect and initialize engines configured in HW IP
status = ProbeEngines(xdma);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "ProbeEngines failed: %!STATUS!", status);
return status;
}
return status;
}
NTSTATUS ProbeEngines(IN PXDMA_DEVICE xdma) {
PAGED_CODE();
ULONG engineIndex = 0;
// iterate over H2C (FPGA performs PCIe reads towards FPGA),
// then C2H (FPGA performs PCIe writes from FPGA)
for (UINT dir = H2C; dir < 2; dir++) { // 0=H2C, 1=C2H
for (ULONG ch = 0; ch < XDMA_MAX_NUM_CHANNELS; ch++) {
if (EngineExists(xdma, dir, ch)) {
XDMA_ENGINE* engine = &(xdma->engines[ch][dir]);
NTSTATUS status = EngineCreate(xdma, engine, dir, ch, engineIndex);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "EngineCreate failed! %!STATUS!", status);
return status;
}
engineIndex++;
TraceInfo(DBG_INIT, "%s_%u engine created (AXI-%s)",
DirectionToString(dir), ch, engine->type == EngineType_ST ? "ST" : "MM");
} else { // skip inactive engines
TraceInfo(DBG_INIT, "Skipping non-existing engine %s_%u",
DirectionToString(dir), ch);
}
}
}
return STATUS_SUCCESS;
}
static NTSTATUS EngineCreate(PXDMA_DEVICE xdma, XDMA_ENGINE* engine, DirToDev dir, ULONG channel,
ULONG engineIndex) {
NTSTATUS status;
engine->parentDevice = xdma;
engine->channel = channel;
engine->dir = dir;
const ULONG offset = (dir * BLOCK_OFFSET) + (channel * ENGINE_OFFSET); //相应地址空间相应通道的偏移地址
PUCHAR configBarAddr = (PUCHAR)xdma->bar[xdma->configBarIdx];
engine->regs = (XDMA_ENGINE_REGS*)(configBarAddr + offset);
engine->sgdma = (XDMA_SGDMA_REGS*)(configBarAddr + offset + SGDMA_BLOCK_OFFSET);
// AXI-MM or AXI-ST? 0 = MM, 1 = ST
//读取C2H地址空间中的config寄存器的第15位,我们公司用的ST模式(数据流的形式,数据对时间匹配要求高),
engine->type = (engine->regs->identifier & XDMA_ID_ST_BIT) != 0;
// Incremental or Non-Incremental address mode? 0 = inc, 1=non-inc
engine->addressMode = (engine->regs->control & XDMA_CTRL_NON_INCR_ADDR) != 0;
// set interrupt sources
// 使能engine所有的中断源
EngineConfigureInterrupt(engine, engineIndex);
// create common buffer for poll mode descriptor write back - if used
//poll为挂起模式,也就是采用轮训模式来接收数据,一般采用另一种中断DMA模式传输,
status = EngineCreatePollWriteBackBuffer(engine);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "EngineCreatePollWriteBackBuffer() failed: %!STATUS!", status);
return status;
}
// capture alignment requirements
EngineGetAlignments(engine);
// create and bind dma desciptor buffer to hardware
// 创建描述符,描述符保存一块内存地址,将此描述符发送给XDMA设备,开启DMA传输之后XDMA设备会将数据发送到指定内存
// 当然传输数据较多时,创建多个描述符,描述符组成一个环形链表,这样就得到一个环形缓冲区
status = EngineCreateDescriptorBuffer(engine);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "EngineCreateDescriptorBuffer() failed: %!STATUS!",
status);
return status;
}
// allocate wdf dma transaction object
// 创建一个DMA传输对象,用来控制DMA传输
status = WdfDmaTransactionCreate(xdma->dmaEnabler, WDF_NO_OBJECT_ATTRIBUTES,
&engine->dmaTransaction);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "WdfDmaTransactionCreate() failed: %!STATUS!", status);
return status;
}
if ((engine->type == EngineType_ST) && (engine->dir == C2H)) {
engine->work = EngineProcessRing;
//创建XDMA_RING_NUM_BLOCKS * XDMA_RING_BLOCK_SIZE大的MDL环形缓冲区,将缓冲区分为XDMA_RING_NUM_BLOCKS块,
//为每块缓冲区分配MDL描述符,保存在engine->ring.mdl[]数组中,
//所谓MDL内存,即一个描述符,描述符包含一块内存首地址,内存大小,下一个MDL地址等
status = EngineCreateRingBuffer(engine);
if (!NT_SUCCESS(status)) {
TraceError(DBG_INIT, "EngineCreateStreamBuffers() failed: %!STATUS!", status);
return status;
}
engine->parentDevice->sgdmaRegs->creditModeEnableW1S = BIT_N(engine->channel) << 16;
TraceInfo(DBG_INIT, "creditModeEnable=0x%x", engine->parentDevice->sgdmaRegs->creditModeEnable);
} else {
engine->work = EngineProcessTransfer;
}
engine->enabled = TRUE; //使能engine
return status;
}
这是初始化过程,下一篇博客解析read过程.