【verbs】ibv_req_notify_cq()

ibv_req_notify_cq() - RDMAmojo RDMAmojo

描述

ibv_req_notify_cq() 在完成队列 (CQ) 上请求Completion Notification（完成通知）。
ibv_req_notify_cq() 请求(驱动）： 当下一个请求的WC 被添加到CQ，生成一个（能被ibv_get_cq_event() 读取的）通知。（bv_req_notify_cq() requests a notification when the next Work Completion of a requested type is added to the CQ.）

在调用 ibv_req_notify_cq() 之前，存在于 CQ 中的任何WC(工作完成)都不会创建将要用ibv_get_cq_event() 读取的“完成通知”。

（个人理解：调用ibv_req_notify_cq()检查CQ上是否有WC，如果有就生成event（通知）。后面调用ibv_get_cq_event()就能读出event，就知道cq有WC，再调用ibv_poll_cq（）从cq中获取WC）

可以请求两种类型的完成事件：
Solicited Completion Event （被动完成事件） - 当对面的send 或 带立即数且设置了Solicited Event 指示（即对端send_flags设置为IBV_SEND_SOLICITED ） 的RDMA write 到来产生一个 “接收成功” or “接收失败”的WC加入到CQ。

Unsolicited Completion Event  （主动完成事件） - 当任何WC(工作完成)添加到 CQ 时发生，无论它是发送还是接收WC(工作完成)，以及它是成功还是不成功的工作完成。

(Unsolicited :不请自来的，主动的）

如果一个请求的Completion Notification（完成通知）处于pending（待处理）：即调用了 ibv_req_notify_cq() 但没有Completion Notification发生，则继续调用 ibv_req_notify_cq() 对同一个 CQ 请求相同的Completion Notification类型将无效；只会生成一个Completion Notification（完成通知）。只有在“完成通知”发生后，调用 ibv_req_notify_cq() 才会生效。

next completion event的“请求完成通知”（Request Completion Notification）优先于同一 CQ 的 solicited event的“请求完成通知”。

如果对同一个 CQ 多次调用 ibv_req_notify_cq() 并且至少有一个请求将类型设置为下一个完成，则在将下一个“完成”添加到该 CQ 时（即不是为下一个Solicited Completion）生成一个“”完成通知”。

一旦一个“完成通知”发生，如果想要获得更多的“完成通知”，他必须再次调用ibv_req_notify_cq()。

原文：ibv_req_notify_cq() - RDMAmojo RDMAmojo

ibv_req_notify_cq() requests a Completion Notification on a Completion Queue (CQ).

ibv_req_notify_cq() requests a notification when the next Work Completion of a requested type is added to the CQ. Any Work Completions that existed in the CQ before calling ibv_req_notify_cq() will not result in creating a Completion Notification. The Completion Notification will be read using ibv_get_cq_event().

There are two types of Completion Events that can be requested:

• Solicited Completion Event - Occurs when an incoming Send or RDMA Write with Immediate Data message with the Solicited Event indicator set (i.e. the remote side posted a Send Request with IBV_SEND_SOLICITED set in send_flags) causes a generation of a successful Receive Work Completion or any unsuccessful (either Send of Receive) Work Completion to be added to the CQ.
• Unsolicited Completion Event - occurs when any Work Completion is added to the CQ, whether it is a Send or Receive Work Completion and whether is it successful or unsuccessful Work Completion.

If a Request Completion Notification is pending, i.e. ibv_req_notify_cq() was called and no Completion Notification occurred, subsequent calls to ibv_req_notify_cq() with the same CQ requesting the same Completion Event type will have no effect; only one Completion Notification will be generated. Calling ibv_req_notify_cq() will have an effect only after the Completion Notification will occur.

A Request Completion Notification for the next completion event takes precedence over a Request Completion Notification for a solicited event completion for the same CQ.

If multiple calls to ibv_req_notify_cq() have been made for the same CQ and at least one of the requests set the type to the next completion, a Completion Notification will be generated when the next Completion is added to that CQ (i.e. not for the next Solicited Completion).

Once a Completion Notification occurred, if one wishes to get more Completions Notification, he has to call ibv_req_notify_cq() again.

参数

Name	Direction	Description
cq	in	CQ that was returned from ibv_create_cq()
solicited_only	in	Type of the requested Completion Notification. 0 The next Completion whether it is Solicited or Unsolicited otherwise the next Completion is Solicited only or for any unsuccessful Work Completion

 返回值

Value	Description
0	On success
errno	On failure.
EINVAL	Invalid CQ handle

 例子

1) 请求下一次Completion的通知：

struct ibv_cq *cq;
 
if (ibv_req_notify_cq(cq, 0)) {
	fprintf(stderr, "Error, ibv_req_notify_cq() failed when requested a "
		"notification for the next completion\n");
	return -1;
}

2) 请求下一次Solicited Completion的通知：

struct ibv_cq *cq;
 
if (ibv_req_notify_cq(cq, 1)) {
	fprintf(stderr, "Error, ibv_req_notify_cq() failed when requested a "
		"notification for the next solicited completion\n");
	return -1;
}

常见问题

我可以使用 ibv_poll_cq() 读取WC(工作完成)，为什么我要使用 ibv_req_notify_cq()？

使用events 处理WC将减少应用程序的 CPU 消耗；您的线程将休眠，直到新的WC将添加到 CQ。

我可以对每个 CQ 调用 ibv_req_notify_cq() 吗？

是的。尽管请记住，请求Solicited Completions的通知 仅对与接收队列相关联的 CQ 有意义。

当 CQ 中有WC（是我已经ask过的type）时，我调用了 ibv_req_notify_cq() 我会收到完成通知吗？
不，你不会。A Completion Notification will be generated for the *next* Completion of the type that you've asked (Solicited or next) that will be added to the CQ *after* you called

我在收到完成通知之前调用了 ibv_req_notify_cq() 几次，这是什么效果？

If all calls to ibv_req_notify_cq() were done when requesting the same Completion type, you will get only one Completion Notification, according to the quested type. If at least one of them requested a Completion Notification for the next Completion, you will get a notification for the next Completion.

我可以要求在将 N 个“完成”添加到 CQ 后收到通知吗
不，这不受支持。

可以让每个添加到 CQ 的“完成”自动获取通知（而不是每次再次调用 ibv_req_notify_cq()）？

不，这不受支持。收到通知后，如果想再收到通知，需要再次调用 ibv_req_notify_cq() 。