ATH9K Driver Learning Part III: Data packet transmission

在第二篇文章中，我们发现 function ath_tx_start() 会根据 packet 的种类不同而选择不同的发送方式。根据实验结果，我发现只有在 AP-STATION CONNECTION INITIALIZATION 过程中 ath_tx_start() 会触发 ath_tx_send_normal()，即此时的 packet typ 并不是 data。一旦 CONNECTION 被成功建立后，AP 与 STATION 之间就只会去发 type = data 的 packets，例如 DHCP request 或者是 application layer packets。因此本篇文章就针对发送 data type packets 的过程来进行研究。

ath_tx_start(struct ieee80211_hw *hw, struct sk_buff *skb, struct ath_tx_control *txctl)

由于该 function 的源代码已经在第二篇 blog 中贴出来了，这里就不再贴了，而是对一些关键语句进行一些补充。

第67行：ath_tx_queue_tid(txq, tid);
这个 function 把 tid 放入 ac-> tid_q，然后把该 ac 放入 txq -> axq_acq 里面。这样子的话内核只需要 txq 就可以定位到储存 packets 的 tid。
第70行: ath_txq_schedule(sc, txq);
有了构建好的 txq, 该 function 将确保每一个 tid 的 packets 都被成功地发送出去。

ath_txq_schedule( struct ath_softc *sc, struct ath_txq *txq)

第49行：if (ath_tx_sched_aggr(sc, txq, tid, &stop))
这一个判断语句十分的关键。只有当 ath_tx_sched_aggr() 返回 FAULSE 时，bool sent 才不会被设置为 TRUE 从而当 ac==last_ac ( &txq->axq_acq 中的 ac 全部被发送完毕）的时候内核可以退出循环，否则就要重新发。所以说 ath_txq_schedule() 的关键就在于 function: ath_tx_sched_aggr()。

ath_tx_sched_aggr(struct ath_softc *sc, struct ath_txq *txq, struct ath_atx_tid *tid, bool *stop)

从第二篇文章可知，该 function 检查了是否使用 EDMA 来发送 packets。如果是，就要调用 func: ath_tx_form_aggr()。反之就调用 func: ath_tx_form_burst。根据实验后得到的 syslog，在 default setting 下 EDMA 功能是不打开的，都是使用 ath_tx_form_burst 来创建一个链表头结构 bf_q，它指向的链表就储存着 packet descriptors。有了这个链表头结构才可以使用 func: ath_tx_txqaddbuf()。

ath_tx_form_burst(struct ath_softc *sc, struct ath_txq *txq, struct ath_atx_tid *tid, struct list_head *bf_q, struct ath_buf *bf_first, struct sk_buff_head *tid_q)

这一个 function 并没有在第二篇文章中详细说明。它的目的是给输入进来的空链表头结构 bf_q 所指向的链表中加入两个 packet descriptors。
源代码如下：

// RT-WiFi: /drivers/net/wireless/ath/ath9k/xmit.c
static void
ath_tx_form_burst(struct ath_softc *sc, struct ath_txq *txq,
		  struct ath_atx_tid *tid, struct list_head *bf_q,
		  struct ath_buf *bf_first, struct sk_buff_head *tid_q)
{
	struct ath_buf *bf = bf_first, *bf_prev = NULL;
	struct sk_buff *skb;
	int nframes = 0;

	do {
		struct ieee80211_tx_info *tx_info;
		skb = bf->bf_mpdu;

		nframes++;
		__skb_unlink(skb, tid_q);					// Unlink the skb from the tid_q
		list_add_tail(&bf->list, bf_q);				// Add the &bf->list into the bf_q. This is the head of the list
													// storing bf 

		if (bf_prev)								// When nframes==1, set the bf_prev->bf_next as the second bf
			bf_prev->bf_next = bf;
		bf_prev = bf;

		if (nframes >= 2)							// Here is the most important CTL information:
													// If the number of nframes is higher than 2, break
			break;

		bf = ath_tx_get_tid_subframe(sc, txq, tid, &tid_q);	// Get the new bf
		if (!bf)
			break;

		tx_info = IEEE80211_SKB_CB(bf->bf_mpdu);
		if (tx_info->flags & IEEE80211_TX_CTL_AMPDU)
			break;

		ath_set_rates(tid->an->vif, tid->an->sta, bf);
	} while (1);
}

该 function 的逻辑并不复杂。进入循环结构后，首先根据输入进来的 descriptor: bf 来拿到它所对应的 packet 。然后使用 function: _skb_unlink(skb, tid_q) 来解除该 packet 与 tid_q 的绑定 (此时 tid_q 指向 tid->buf_q or tid->retry_q)，从而得以把该 descriptor: bf 给放入bf_q 所指向的 list 中。一旦这个 descriptor: bf 保存完毕后，就使用 function: ath_tx_get_tid_subframe() 来构建一个新的 descriptor: bf。再完成一系列 settings 之后，这个新的 bf 也被放入 bf_q 所指向的 list 中。注意这个循环只能重复两次，即 bf_q 所指向的 list 长度只能为2，之后就会跳出循坏结束该 function。

一旦 function: ath_tx_sched_aggr() 从 function: ath_tx_form_burst() 拿到了完成好的链表头结构 bf_q，就会使用 ath_tx_txqaddbuf() 把 packets 发送出去。

ath_tx_txqaddbuf(struct ath_softc *sc, struct ath_txq *txq,struct list_head *head, bool internal)

第二篇文章关注的是当 RT-WiFi 未启动的时候 function: ath_tx_txqaddbuf() 的工作逻辑。当 RT-WiFi 启动时，code 跳到第120行：else if (sc->rt_wifi_enable == 1)。然后使用 function: list_for_each_entry(bf_itr, head, list) 来遍历链表头 head 所对应的链表 list 中的所有 components，并把这些 components 存入 kfifo 结构：&sc->rt_wifi_fifo 中。储存完毕后，便会 enable interruptions 然后退出函数。这样一来，一旦进入了在 RT-WiFi 中定义的发送 packet 的 time slot，function：ath_rt_wifi_tasklet() 便会被 interrupt 触发，从 rt_wifi_fifo 中拿出数据发包。

ath_rt_wifi_tasklet(struct ath_softc *sc)

The definition of tasklet: Tasklets are built on top of softirqs to allow dynamic creation of deferrable functions. [link]由此可见，ath_rt_wifi_takslet() 是可以被 interrupts 触发的 function。其源代码如下：

// RT-WiFi: /drivers/net/wireless/ath/ath9k/rtwifi.c
void ath_rt_wifi_tasklet(struct ath_softc *sc)/*Define the method to access												 data for different data types*/
{
	int sched_offset;
	struct ath_buf *new_buf = NULL;
	u64 cur_hw_tsf;

	if (sc->rt_wifi_enable == 0) {
		if (sc->sc_ah->opmode == NL80211_IFTYPE_AP) {//located in the NL80211 file
			sc->rt_wifi_enable = 1;
		} else {
			RT_WIFI_DEBUG("RT_WIFI: not enable\n");
			return;
		}
	}

	/* House keeping */
	sc->rt_wifi_asn++;	

	cur_hw_tsf = ath9k_hw_gettsf64(sc->sc_ah);
	cur_hw_tsf = cur_hw_tsf - sc->rt_wifi_virt_start_tsf + (RT_WIFI_TIMER_OFFSET * 2);
	sc->rt_wifi_asn = cur_hw_tsf >> ilog2((sc->rt_wifi_slot_len));

	sched_offset = sc->rt_wifi_asn % sc->rt_wifi_superframe_size;
	if (sc->sc_ah->opmode == NL80211_IFTYPE_AP) {									// If you are AP
		if (sc->rt_wifi_superframe[sched_offset].type == RT_WIFI_RX) {				// If the working type at this time slot is RT_WIFI_RX
			RT_WIFI_DEBUG("RT_WIFI_RX(%d)\n", sched_offset);						
			new_buf = ath_rt_wifi_get_buf_ap_tx(sc,									// Get the packet (decribed by new_buf) going to be transmitted.
					sc->rt_wifi_superframe[sched_offset].sta_id);
		} else if (sc->rt_wifi_superframe[sched_offset].type == RT_WIFI_SHARED) {	// If the working type at this time slot is RT_WIFI_SHARED
			RT_WIFI_DEBUG("RT_WIFI_SHARED(%d)\n", sched_offset);
			new_buf = ath_rt_wifi_get_buf_ap_shared(sc);
		}
	} else if (sc->sc_ah->opmode == NL80211_IFTYPE_STATION) {						// If you are Station
		if (sc->rt_wifi_superframe[sched_offset].type == RT_WIFI_TX					// If the working type at this time slot is RT_WIFI_TX
		&& sc->rt_wifi_superframe[sched_offset].sta_id == RT_WIFI_LOCAL_ID) {
			RT_WIFI_DEBUG("RT_WIFI_TX(%d)\n", sched_offset);
			new_buf = ath_rt_wifi_get_buf_sta(sc);
		}
	}

	ath_rt_wifi_tx(sc, new_buf);
}

这个 function 的逻辑也很简单。首先先判断设备是不是 AP，如果是 AP 并且当前 time slot 是 RT_WIFI_RX 的话，就使用 function: ath_rt_wifi_get_buf_ap_tx() 从 kfifo 中拿要发送给对应 station 的包。如果当前 time slot 是 RT_WIFI_SHARED, 就使用 function: ath_rt_wifi_get_buf_ap_shared() 拿包。如果设备是 station, 就查看该 time slot 是不是 RT_WIFI_TX ，自己可不可以在这个 slot 上发包。如果都满足，就使用 function: ath_rt_wifi_get_buf_sta() 从 kfifo 里面拿包。最后，使用 function: ath_rt_wifi_tx() 来发送 packet。注意，ath_rt_wifi_tx() 只接受一个 decriptor, 不再要求输入一个链表结构了。

ath_rt_wifi_tx(struct ath_softc *sc, struct ath_buf *new_buf)

这个 function 可以说就是 COPY 嘛，话不多说上源代码：

// RT-WiFi: /drivers/net/wireless/ath/ath9k/rtwifi.c
void ath_rt_wifi_tx(struct ath_softc *sc, struct ath_buf *new_buf)
{
	struct ath_hw *ah = sc->sc_ah;
	struct ath_common *common = ath9k_hw_common(ah);
	struct ath_buf *bf, *bf_last;
	bool puttxbuf = false;
	bool edma;//enhanced direct memeory access

	/* rt-wifi added */
	bool internal = false;
	struct ath_txq *txq;
	struct list_head head_tmp, *head;//tmp: temporary folder

	if(new_buf == NULL)
		return;
	REG_SET_BIT(ah, AR_DIAG_SW, AR_DIAG_RX_DIS | AR_DIAG_RX_ABORT);//Register level operation!

	/* mainpulate list to fit original driver code. */
	head = &head_tmp;

	/*IMPORTANT: We create a new LIST structure with the head as the initial pointer*/
	INIT_LIST_HEAD(head);/*Initialize the list header*/


	list_add_tail(&new_buf->list, head);//Z: list is the head of the new_buf. Insert it before the head of the LIST for building a queue

	txq = &(sc->tx.txq[new_buf->qnum]);//txq belongs to struct ath_txq

	/* original dirver code */
	edma = !!(ah->caps.hw_caps & ATH9K_HW_CAP_EDMA);

	/*bf means buffer. The following tow sentences here are used to get the address 
	of the pointer to the header of the structure containing MPDU*/
	bf = list_first_entry(head, struct ath_buf, list);//bf: buffer Get the first element in the LIST structure with pointer: head,
													//the catched target is the MPDU(the head of the MPDU is the list)

	bf_last = list_entry(head->prev, struct ath_buf, list);//Get the MPDU pointed by the list from the previous node
	//bf_last=list_last_entry(head, struct ath_buf, list)

	ath_dbg(common, QUEUE, "qnum: %d, txq depth: %d\n",
			txq->axq_qnum, txq->axq_depth);

/*The following part is used to associate the MPDU with the txq;
  If edma is enabled and the fifo is empty, then the MPDU is associated with txq_fifo
  If fifo is not empty at the same time edma is not enabled, then the MPDU vitual address is linked with
  txq_axq_link and the MPDU itself is associated with the axq_q
  
  For any other cases except the above two, puttxbuf=false, no transmission */
	if (edma && list_empty(&txq->txq_fifo[txq->txq_headidx])) {//edma is related with fifo???????????????? 6/30
		list_splice_tail_init(head, &txq->txq_fifo[txq->txq_headidx]);//join two lists and reinitialise the emptied list
		//list_splice_tail_init: Insert the content of the head structure into txq_headidx structure
		// and then clear the head structure. Since txq_headidx is empty, it just like cut head and paste in
		// the empty txq_headidx 
		INCR(txq->txq_headidx, ATH_TXFIFO_DEPTH);
		puttxbuf = true;//Determine whether the information have been put into buffer
	} 
	else {//fifo is not empty
		list_splice_tail_init(head, &txq->axq_q);//axq means ath9k hardware queue. If not edma nor fifo is empty, 
												// merge the head of the queue structure with 
												// axq_q

		if (txq->axq_link) {
			ath9k_hw_set_desc_link(ah, txq->axq_link, bf->bf_daddr);//axq: ath9k hardware queue
																	
			ath_dbg(common, XMIT, "link[%u] (%p)=%llx (%p)\n",
					txq->axq_qnum, txq->axq_link,
					ito64(bf->bf_daddr), bf->bf_desc);
		} else if (!edma)//txq->axq_link is false and we do not use edma
			puttxbuf = true;

		txq->axq_link = bf_last->bf_desc;// bf_desc: virtual addr of desc of the last packet
	}
//???????????????????????????????? 7/1  cannot fully understand axq_link. axq_link is a void, how could it be
							    //  related with the virtual address of both newest MPDU and latest MPDU??
//
	if (puttxbuf) {
		TX_STAT_INC(txq->axq_qnum, puttxbuf);
		ath9k_hw_puttxbuf(ah, txq->axq_qnum, bf->bf_daddr);//u32 axq_qnum: ath9k hardware queue number 
														   //This function builds up a link between
														   //the hardware queue and the physical address
														   //of the packet descriptor
		ath_dbg(common, XMIT, "TXDP[%u] = %llx (%p)\n",
				txq->axq_qnum, ito64(bf->bf_daddr), bf->bf_desc);
	}

	/* rt-wifi, disable carrier sense random backoff */
	#if RT_WIFI_ENABLE_COEX == 0
	REG_SET_BIT(ah, AR_DIAG_SW, AR_DIAG_FORCE_CH_IDLE_HIGH);
	REG_SET_BIT(ah, AR_D_GBL_IFS_MISC, AR_D_GBL_IFS_MISC_IGNORE_BACKOFF);
	#endif
	REG_SET_BIT(ah, AR_DIAG_SW, AR_DIAG_IGNORE_VIRT_CS);
	/*When the edma is not activated, how to transmit*/
	if (!edma || sc->tx99_state) {
		TX_STAT_INC(txq->axq_qnum, txstart);
		ath9k_hw_txstart(ah, txq->axq_qnum);
	}

	if (!internal) {
		while (bf) {
			txq->axq_depth++;
			if (bf_is_ampdu_not_probing(bf))
				txq->axq_ampdu_depth++;

			bf_last = bf->bf_lastbf;
			bf = bf_last->bf_next;
			bf_last->bf_next = NULL;
		}
	}

	REG_CLR_BIT(ah, AR_DIAG_SW, AR_DIAG_RX_DIS | AR_DIAG_RX_ABORT);
}

嗯里面的 comment 有一些乱了，但还是抓重点。其实这个函数和原版的 function: ath_tx_txqaddbuf 是一样的，只不过这里用的是单个 bf 而不是一个链表结构了。所以在 RT-WiFi 中，packets 的 decriptors 先放进了 kfifo 里面，然后在使用中断程序让 kernel 在特定的时间点上进行发包。从而实现 TDMA 功能。