转载：https://www.freebsdchina.org/forum/viewtopic.php?t=37913
queue和list是freebsd内核中很基础的数据结构，在内核代码的很多地方都用到，下面是小弟的一点心得，不对的地方希望大家指教，谢~~.
queue和list的结构定义和操作都在'sys/queue.h'中完成，内容并不多，主要定义了下面四种数据结构：
单向列表(single-linked lists)、单向尾队列(single-linked tail queue)、列表(lists)、尾队列(tail queues)。
一、数据结构定义
1.SLIST单向列表(single-linked lists)
整个列表由如下类型的列表头标识：
140 #define SLIST_HEAD(name, type) \
141 struct name { \
142 struct type *slh_first; /* first element */ \
143 }
列表中的用来勾勒链接的指针项的结构如下：
148 #define SLIST_ENTRY(type) \
149 struct { \
150 struct type *sle_next; /* next element */ \
151 }
其实上述定义并没有定义一个具体的list，应该说是提供了定义一个list的template，为什么这么说呢？
因为这里的type是不确定的，可以是内核的其他很多地方就用它们来定义一个“具体”的列表，例如：
在(sys/sysctl.h)中就有：
147 SLIST_HEAD(sysctl_oid_list, sysctl_oid);
和
153 struct sysctl_oid {
154 struct sysctl_oid_list *oid_parent;
155 SLIST_ENTRY(sysctl_oid) oid_link;
156 int oid_number;
157 u_int oid_kind;
158 void *oid_arg1;
159 int oid_arg2;
160 const char *oid_name;
161 int (*oid_handler)(SYSCTL_HANDLER_ARGS);
162 const char *oid_fmt;
163 int oid_refcnt;
164 const char *oid_descr;
165 };
这里就将"type"实例化为"sysctl_oid"。也就是定义了“某种”具体的列表：sysctl_oid列表。
该列表类型由类型为"sysctl_oid_list"的变量来标识，列表中每个元素的类型为"sysctl_oid"。
到此还是在定义数据结构，还没到“具体”的一个一个列表，那么整个kernel中“sysctl_oid列表”型的
列表都有哪些呢？通过lxr查找，我们可以看到：
74 struct sysctl_oid_list sysctl__children; /* root list */
也就是说freebsd内核维护着一条由变量"sysctl__children"为头的列表，列表中各元素的类型为"sysctl_oid"，
整个列表由元素中"oil_link"这个entry链接起来。
SLIST其实现了什么结构，queue.h中的注释说得很清楚了，下面就直接引用吧，呵呵；其他三个类型类同。
42 * A singly-linked list is headed by a single forward pointer. The elements
43 * are singly linked for minimum space and pointer manipulation overhead at
44 * the expense of O(n) removal for arbitrary elements. New elements can be
45 * added to the list after an existing element or at the head of the list.
46 * Elements being removed from the head of the list should use the explicit
47 * macro for this purpose for optimum efficiency. A singly-linked list may
48 * only be traversed in the forward direction. Singly-linked lists are ideal
49 * for applications with large datasets and few or no removals or for
50 * implementing a LIFO queue.
2.STAILQ单向尾队列(single-linked tail queue)
STAILQ实现了什么呢？先看看注释吧
52 * A singly-linked tail queue is headed by a pair of pointers, one to the
53 * head of the list and the other to the tail of the list. The elements are
54 * singly linked for minimum space and pointer manipulation overhead at the
55 * expense of O(n) removal for arbitrary elements. New elements can be added
56 * to the list after an existing element, at the head of the list, or at the
57 * end of the list. Elements being removed from the head of the tail queue
58 * should use the explicit macro for this purpose for optimum efficiency.
59 * A singly-linked tail queue may only be traversed in the forward direction.
60 * Singly-linked tail queues are ideal for applications with large datasets
61 * and few or no removals or for implementing a FIFO queue.
再看看对应的类型定义，STAILQ的类型“template”定义如下：
211 #define STAILQ_HEAD(name, type) \
212 struct name { \
213 struct type *stqh_first;/* first element */ \
214 struct type **stqh_last;/* addr of last next element */ \
215 }
以及
220 #define STAILQ_ENTRY(type) \
221 struct { \
222 struct type *stqe_next; /* next element */ \
223 }
这里有意思的是"STAILQ_HEAD"中stqh_last的定义和注释，首先注意到stqh_last是指向"type"类型元素的
二级指针。那么，这个stqh_last到底指向哪里呢？该项的注释说明是"addr of last next element"，而不是
"addr of last element"。以三个元素的STAILQ为例，其指针之间的关系可以由下图表示。
stqh_head:
+----------|---------+
|stqh_first|sqth_last|
+-+--------|-------+-+
| |
| +-----------------------------------------+
| |
| |
| +----+---------+ +----+---------+ +----+----V----+
+->| |stqe_next+------>| |stqe_next+------>| |stqe_next+--->NULL
+----+---------+ +----+---------+ +----+---------+
关于"addr of last next element"的值可以从"STAILQ_INSERT_TAIL"的定义得到验证。
3. LIST列表(lists)
LIST就是我们常说的双向链表，还是看看注释吧。
63 * A list is headed by a single forward pointer (or an array of forward
64 * pointers for a hash table header). The elements are doubly linked
65 * so that an arbitrary element can be removed without a need to
66 * traverse the list. New elements can be added to the list before
67 * or after an existing element or at the head of the list. A list
68 * may only be traversed in the forward direction.
LIST的"template"定义如下：
310 #define LIST_HEAD(name, type) \
311 struct name { \
312 struct type *lh_first; /* first element */ \
313 }
以及
318 #define LIST_ENTRY(type) \
319 struct { \
320 struct type *le_next; /* next element */ \
321 struct type **le_prev; /* address of previous next element */ \
322 }
HEAD的定义和SLIST一模一样，不同的是ENTRY的定义，entry多了一项即"le_prev"。但是，注意到这个
le_preve的类型是"struct type **le_prev"，也就是说它也是一个二级指针，那么它指向哪里呢？
注释说明是“address of previous next element”。这点也和普通“数据结构”教材中的定义不一样。
我们还是用图示说明，比如相邻的LIST结点，其指针间的关系如下：
"previous" "this"
V V
+-------+ +-------+
| | | |
| | | |
+-------+ +-------+
... | . | .-+--> | . | .-+--> ...
^ +---+--^+ +-+-+---+
| | ^ | |
+-----+ | +-------+
|
|
"previous next element"
注意到，其中结点this的le_prev指针的值是结点previous的le_next的地址，它是"struct type **le_prev"类型的
变量。
这里，为什么偏要le_prev赋予类型"struct type **le_prev"，并赋值"address of previous
next element"呢？而不简简单单赋值"address of previous element"？怎么想也得不到合理的解释，
幸好别人讨论过这个问题，具体原因我们在文章的结尾讨论。
4.TAILQ尾队列(tail queues)
有了前面的分析，TAILQ容易理解了，这里还是把代码中的注释列出来。
70 * A tail queue is headed by a pair of pointers, one to the head of the
71 * list and the other to the tail of the list. The elements are doubly
72 * linked so that an arbitrary element can be removed without a need to
73 * traverse the list. New elements can be added to the list before or
74 * after an existing element, at the head of the list, or at the end of
75 * the list. A tail queue may be traversed in either direction.
再看定义，
392 #define TAILQ_HEAD(name, type) \
393 struct name { \
394 struct type *tqh_first; /* first element */ \
395 struct type **tqh_last; /* addr of last next element */ \
396 TRACEBUF \
397 }
402 #define TAILQ_ENTRY(type) \
403 struct { \
404 struct type *tqe_next; /* next element */ \
405 struct type **tqe_prev; /* address of previous next element */ \
406 TRACEBUF \
407 }
不难看出TAILQ就是STAILQ和LIST两者的特征结合。举一个由三个结点构成的TAILQ，如图所示。
tqh_head:
+---------|--------+
|tqh_first|tqh_last|
+-+----^--|-------++
| | +-------------------------------------+
| | one two three |
| | +-------+ +-------+ +-------+ |
| | | | | | | | |
| | | | | | | | |
| | +---+---+ +---+---+ +---+---+<NULL>stqh_last) - __offsetof(struct type, field))))
的确不一般，由于head->stqh_last并不是指到链表的最后一个元素，而是最后一个元素的stqe_next域的地址，因此，
这里只能由head->stqh_last的值“裸算”整个结点的起始地址了，呵呵。
3.TAILQ_LAST和TAILQ_PREV
TAILQ_LAST和TAILQ_PREV的定义如下：
497 #define TAILQ_LAST(head, headname) \
498 (*(((struct headname *)((head)->tqh_last))->tqh_last))
502 #define TAILQ_PREV(elm, headname, field) \
503 (*(((struct headname *)((elm)->field.tqe_prev))->tqh_last))
这两个操作的定义就更难理解了，特别是其中用到了"(struct headname *)"和"*->tqh_last"。这里是怎么计算
TAILQ_LAST和TAILQ_PREV的呢？我们在本文的第四部分讨论。
这里用到了一点trick：注意TAILQ_HEAD和TAILQ_ENTRY两个结构定义一样。
三、内核代码中的应用
SLIST、STAILQ、 LIST和TAILQ四种数据结构在FreeBSD内核代码中大量应用，例如我们上面说到的
"struct sysctl_oid_list"。
另外还有，
(sys/linker.h)
47 typedef TAILQ_HEAD(, linker_file) linker_file_list_t;
(ufs/ffs/softdep.h)
161 LIST_HEAD(dirremhd, dirrem);
162 LIST_HEAD(diraddhd, diradd);
163 LIST_HEAD(newblkhd, newblk);
164 LIST_HEAD(inodedephd, inodedep);
165 LIST_HEAD(allocindirhd, allocindir);
166 LIST_HEAD(allocdirecthd, allocdirect);
167 TAILQ_HEAD(allocdirectlst, allocdirect);
(sys/proc.h)
470 struct ksegrp {
471 struct proc *kg_proc; /* (*) Proc that contains this KSEG. */
472 TAILQ_ENTRY(ksegrp) kg_ksegrp; /* (*) Queue of KSEGs in kg_proc. */
473 TAILQ_HEAD(, thread) kg_threads;/* (td_kglist) All threads. */
474 TAILQ_HEAD(, thread) kg_runq; /* (td_runq) waiting RUNNABLE threads */
475 TAILQ_HEAD(, kse_upcall) kg_upcalls; /* All upcalls in the group. */
....，不胜枚举。
四、“特色”讨论
对于上述的“特色”，不光我们迷惑，幸好还有大牛们也迷惑，我们就看看人家是怎么解释这个疑团的。
先看几封邮件(http://archive.netbsd.se/?ml=freebsd-arch&a=2007-03&t=3301045)
Subject: <sys> bikeshed proposal
From: Poul-Henning Kamp
Date: 2007-03-13 10:01:54
It has always bothered me that some of the TAILQ macros need to
know the struct name of the header type.
Many times where I could have avoided naming that structure, I
had to, only to satisfy this weird craving of .
needs the struct name TAILQ_LAST and TAILQ_PREV casts,
what is potentially a pointer to an entry, to the header type,
before it dereferences the backwards pointer.
The validity of this cast depends critically on the layout of the
entry struct and the header struct being identical with respect to
the two pointers.
So, if we have already assumed, that they have the same layout, why
do the cast in the first place ?
The following proof of concept patch shows, that you can implement
these two macros without the cast to the header struct type.
If we did this, we could eliminate the "headname" argument to
TAILQ_FOREACH_REVERSE
TAILQ_FOREACH_REVERS_SAFE
TAILQ_LAST
TAILQ_PREV
Obviously this is bikeshed fodder, but given how big a help
is programming-wise, and given that those four macros are comparatively
seldomly used, I will propose to remove this wart from
under the banner of computer science in general and suffer the minor
backwards compatibility issues it will cause.
Poul-Henning
Index: queue.h
===================================================================
RCS file: /home/ncvs/src/sys/sys/queue.h,v
retrieving revision 1.68
diff -u -r1.68 queue.h
--- queue.h 24 Oct 2006 11:20:29 -0000 1.68
+++ queue.h 13 Mar 2007 08:51:51 -0000
@@ -546,12 +546,12 @@
} while (0)
#define TAILQ_LAST(head, headname) \
- (*(((struct headname *)((head)->tqh_last))->tqh_last))
+ (*((head)->tqh_last->tqe_prev)
#define TAILQ_NEXT(elm, field) ((elm)->field.tqe_next)
#define TAILQ_PREV(elm, headname, field) \
- (*(((struct headname *)((elm)->field.tqe_prev))->tqh_last))
+ (*((elm)->field.tqe_prev->tqe_prev))
#define TAILQ_REMOVE(head, elm, field) do { \
QMD_TAILQ_CHECK_NEXT(elm, field); \
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
<email> | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
freebsd-<email> mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-<email>"
------------------------------------------------------------------------------------
From: Yar Tikhiy
Date: 2007-03-15 14:43:00
On Tue, Mar 13, 2007 at 09:38:05AM +0000, Poul-Henning Kamp wrote:
> In message , Poul-Henning Kamp writes:
> >
> >It has always bothered me that some of the TAILQ macros need to
> >know the struct name of the header type.
Yeah, can present a challenge in understanding an
implementation of basic data structures and related algos.
You thought that tqe_prev points to the whole entry structure when
making the patch, didn't you?
Personally, I cannot explain to myself why in the double-linked
structs the prev member points to the next member in the previous
list element and not to the previous list element itself. Could
anybody with CS education explain merits of the current approach?
I can only see that now we have to go to the element before the
previous one for a pointer to the latter. I'm not going to dispute
the current way of things, just curious.
--
Yar
_______________________________________________
freebsd-<email> mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-<email>"
---------------------------------------------------------------------------------------
From: Bakul Shah
Date: 2007-03-15 18:59:24
> On Tue, Mar 13, 2007 at 09:38:05AM +0000, Poul-Henning Kamp wrote:
> > In message , Poul-Henning Kamp writes:
> > >
> > >It has always bothered me that some of the TAILQ macros need to
> > >know the struct name of the header type.
>
> Yeah, can present a challenge in understanding an
> implementation of basic data structures and related algos.
> You thought that tqe_prev points to the whole entry structure when
> making the patch, didn't you?
>
> Personally, I cannot explain to myself why in the double-linked
> structs the prev member points to the next member in the previous
> list element and not to the previous list element itself. Could
> anybody with CS education explain merits of the current approach?
> I can only see that now we have to go to the element before the
> previous one for a pointer to the latter. I'm not going to dispute
> the current way of things, just curious.
IIRC, the main reason is so that you don't have to know the
actual type of other things on the list. In particular the
list head can be just a next/prev ptr pair and not the entire
object. This is handy, for example, when you have a list
hanging off another object -- you can remove any list element
without detaching the entire list from the parent object.
The queue.h trick is quite clever and useful. To appreciate
it, try figuring out a solution using Lisp style lists!
IIRC, your typical CS education doesn't spend much time on
pragmatics.
_______________________________________________
freebsd-<email> mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-<email>"
From: Bakul Shah
Date: 2007-03-15 18:59:24
> On Tue, Mar 13, 2007 at 09:38:05AM +0000, Poul-Henning Kamp wrote:
> > In message , Poul-Henning Kamp writes:
> > >
> > >It has always bothered me that some of the TAILQ macros need to
> > >know the struct name of the header type.
>
> Yeah, can present a challenge in understanding an
> implementation of basic data structures and related algos.
> You thought that tqe_prev points to the whole entry structure when
> making the patch, didn't you?
>
> Personally, I cannot explain to myself why in the double-linked
> structs the prev member points to the next member in the previous
> list element and not to the previous list element itself. Could
> anybody with CS education explain merits of the current approach?
> I can only see that now we have to go to the element before the
> previous one for a pointer to the latter. I'm not going to dispute
> the current way of things, just curious.
IIRC, the main reason is so that you don't have to know the
actual type of other things on the list. In particular the
list head can be just a next/prev ptr pair and not the entire
object. This is handy, for example, when you have a list
hanging off another object -- you can remove any list element
without detaching the entire list from the parent object.
The queue.h trick is quite clever and useful. To appreciate
it, try figuring out a solution using Lisp style lists!
IIRC, your typical CS education doesn't spend much time on
pragmatics.
_______________________________________________
freebsd-<email> mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-<email>"
-------------------------------------------------------------------------------
From: Yar Tikhiy
Date: 2007-03-15 20:04:53
On Thu, Mar 15, 2007 at 10:59:24AM -0700, Bakul Shah wrote:
> > On Tue, Mar 13, 2007 at 09:38:05AM +0000, Poul-Henning Kamp wrote:
> > > In message , Poul-Henning Kamp writes:
> > > >
> > > >It has always bothered me that some of the TAILQ macros need to
> > > >know the struct name of the header type.
> >
> > Yeah, can present a challenge in understanding an
> > implementation of basic data structures and related algos.
> > You thought that tqe_prev points to the whole entry structure when
> > making the patch, didn't you?
> >
> > Personally, I cannot explain to myself why in the double-linked
> > structs the prev member points to the next member in the previous
> > list element and not to the previous list element itself. Could
> > anybody with CS education explain merits of the current approach?
> > I can only see that now we have to go to the element before the
> > previous one for a pointer to the latter. I'm not going to dispute
> > the current way of things, just curious.
>
> IIRC, the main reason is so that you don't have to know the
> actual type of other things on the list. In particular the
> list head can be just a next/prev ptr pair and not the entire
> object. This is handy, for example, when you have a list
> hanging off another object -- you can remove any list element
> without detaching the entire list from the parent object.
> The queue.h trick is quite clever and useful. To appreciate
> it, try figuring out a solution using Lisp style lists!
> IIRC, your typical CS education doesn't spend much time on
> pragmatics.
Oh, now I seem to understand. Thank you!
Here's how I see it:
Let's assume that the prev member points to the whole previous
element of the list or queue. E.g.:
struct queue_head {
type *first;
type *last;
};
struct queue_entry {
type *next;
type *prev;
}
In this case we can indicate the first element of the queue by prev
being NULL. But then we have no way to get from the first element
to the list head. This means that we have to special case such
operations as removal of the first element and insertion before it.
OTOH, the current approach doesn't suffer from this shortcoming.
BTW, the Linux folks took a different way: They keep pointers to
the list entry structure itself (they call it list_head):
struct list_head {
struct list_head *next, *prev;
};
Consequently, they need to do pointer math to get from an embedded
list_head to the containing structure itself:
#define list_entry(ptr, type, member) \
((type *)((char *)(ptr)-(unsigned long)(&((type *)0)->member)))
Linked lists are fun!
Thank you once more for pointing me in the right direction!
--
Yar
_______________________________________________
freebsd-<email> mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "freebsd-arch-<email>"
第一、二份邮件是两个“小牛”的抱怨，后面两个给出了答案。
480
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
