I/O复用 poll的内核源码剖析

一：I/O 复用技术

  I/O复用技术是：把我们关注的描述符组成一个描述符表（通常不止一个描述符），调用I/O复用函数（select/poll/epoll），当描述符表中有可进行非阻塞I/O操作的描述符时，复用函数返回；否则阻塞复用函数，直到描述符表中有可进行非阻塞I/O操作的描述符出现时，才能唤醒进程继续执行复用函数；当复用函数正常返回时，就可以知道哪些描述符可进行非阻塞I/O操作。

  I/O复用的描述符通常包括： 终端/伪终端，pipes，sockfd等。

二：I/O复用函数的主要过程

1.遍历描述符表，判断该描述符表中是否有描述符可进行非阻塞I/O（读，写，异常等）；

2.如果描述符表中有描述符可进行非阻塞I/O操作，I/O复用函数通知用户进程这些描述符；

3.如果描述符表中没有描述符可进行非阻塞I/O操作，那么I/O复用函数被阻塞，并将进程添加到描述符表中所有描述符的poll等待队列中；

4.当有描述符可进行非阻塞I/O操作时，内核唤醒该描述符poll等待队列中的阻塞进程；进程唤醒后继续执行I/O复用函数，I/O复用函数将进程从描述符表中所有描述符的poll等待队列中移除；然后重新遍历描述符表。

 

三：源代码分析

poll

  poll是I/O复用函数之一，其原型为：

int poll(struct pollfd fds[], nfds_t nfds, int timeout);

 39 /* Data structure describing a polling request.  */
 40 struct pollfd
 41   {
 42     int fd;                     /* File descriptor to poll.  */
 43     short int events;           /* Types of events poller cares about.  */
 44     short int revents;          /* Types of events that actually occurred.  */
 45   };

  输入参数：

fds：描述符及事件组成的pollfd数组

nfds：pollfd数组的大小

timeout：超时时间，单位为毫秒

  输出参数：

1.pollfd.events中发生的事件pollfd.revents

  返回值：

>0:pollfd.events事件发生的文件描述符的个数

0：超时，所有文件描述符中pollfd.events事件都未发生

-1：错误返回

poll和select功能相似，差异主要是：

1.poll的事件更为精确，select的事件只能为POLLIN_SET,POLLOUT_SET,POLLEX_SET,而这三个事件均是由poll事件组成的集合；

2.poll的描述符表没有大小限制；select描述符表最大不能超过FD_SET_SIZE（fd_set类型的比特位数）

3.poll的描述符及事件不用每次调用poll都重新赋值（输出为pollfd.events，输出为pollfd.revents），但是select每次调用都必须重新赋值（输入输出均为readfds,writefds,errnofds）;

数据结构

poll_list

694 struct poll_list {
695         struct poll_list *next;
696         int len;
697         struct pollfd entries[0];
698 };
808 #define N_STACK_PPS ((sizeof(stack_pps) - sizeof(struct poll_list))  / \
809                         sizeof(struct pollfd))

poll_list主要用于将pollfd数据存储在内核空间中，内存取自内核堆栈和slab；只有内核堆栈预分配给pollfd的内存使用完（pollfd数据大于N_STACK_PPS）后才会从SLAB(一种分配器)中获取额外的内存

poll_list之间的关系图

poll_wqueues

​
 33 typedef struct poll_table_struct {
 34         poll_queue_proc qproc;
 35         unsigned long key;
 36 } poll_table;
 
 50 struct poll_table_entry {
 51         struct file *filp;
 52         unsigned long key;
 53         wait_queue_t wait;
 54         wait_queue_head_t *wait_address;
 55 };
 56 
 57 /*
 58  * Structures and helpers for sys_poll/sys_poll
 59  */
 60 struct poll_wqueues {
 61         poll_table pt;
 62         struct poll_table_page *table;
 63         struct task_struct *polling_task;
 64         int triggered;
 65         int error;
 66         int inline_index;
 67         struct poll_table_entry inline_entries[N_INLINE_POLL_ENTRIES];
 68 };
​

poll_table：对每个文件进行poll操作时，判断是否能够非阻塞的进行key值（poll事件组成）标识的I/O操作；如果不能，调用回调函数pqroc将进程添加到文件的poll等待队列中。

poll_table_entry:用于阻塞进程并将进程添加到文件的poll等待队列中，一个文件对应一个poll_table_entry。

poll_wqueues：用于在select/poll时，如果需要阻塞进程，将进程添加到描述符表标识的所有文件的poll等待队列中，以便任意一个文件可进行非阻塞I/O操作时唤醒进程。

 

进程，打开文件，poll等待队列之间关系图

复用函数阻塞/唤醒

poll_wqueues的初始化

 44 static inline void init_poll_funcptr(poll_table *pt, poll_queue_proc qproc)
 45 {
 46         pt->qproc = qproc;
 47         pt->key   = ~0UL; /* all events enabled */
 48 }

 
116 void poll_initwait(struct poll_wqueues *pwq)
117 {
118         init_poll_funcptr(&pwq->pt, __pollwait);
119         pwq->polling_task = current;
120         pwq->triggered = 0;
121         pwq->error = 0;
122         pwq->table = NULL;
123         pwq->inline_index = 0;
124 }

1.将阻塞回调函数设置成 __pollwait

2.将阻塞进程设置成当前进程

文件poll阻塞

1.poll阻塞

当对单个文件执行poll操作时，如果文件不能非阻塞的进行key标识的I/O操作，会将当前进程添加到该文件的poll等待队列中

2.tcp阻塞f_op->poll:

socket_file_ops->sock_poll->inet_stream_ops->tcp_poll->sock_poll_wait->poll_wait

pipe阻塞f_op->poll: write_pipefifo_fops->pipe_poll->poll_wait

 38 static inline void poll_wait(struct file * filp, wait_queue_head_t * wait_address, poll_table *p)
 39 {
 40         if (p && wait_address)
 41                 p->qproc(filp, wait_address, p);
 42 }
 
216 static void __pollwait(struct file *filp, wait_queue_head_t *wait_address,
217                                 poll_table *p)
218 {
219         struct poll_wqueues *pwq = container_of(p, struct poll_wqueues, pt);
220         struct poll_table_entry *entry = poll_get_entry(pwq);
221         if (!entry)
222                 return;
223         get_file(filp);
224         entry->filp = filp;
225         entry->wait_address = wait_address;
226         entry->key = p->key;
227         init_waitqueue_func_entry(&entry->wait, pollwake);
228         entry->wait.private = pwq;
229         add_wait_queue(wait_address, &entry->wait);
230 }

a.poll_wait中的qproc在poll_initwait时设置成__pollwait；如果poll_table与wait_address非NULL，则调用__pollwait

b.将poll等待队列的waiter唤醒函数设置成pollwake

c.将poll_table_entry放入wait_address(socket为sock->sk_sleep,pipe为pipe_inode_info->wait)的等待队列中

3.poll_get_entry

 98 #define POLL_TABLE_FULL(table) \
 99         ((unsigned long)((table)->entry+1) > PAGE_SIZE + (unsigned long)(table))
 
155 static struct poll_table_entry *poll_get_entry(struct poll_wqueues *p)
156 {
157         struct poll_table_page *table = p->table;
158 
159         if (p->inline_index < N_INLINE_POLL_ENTRIES)//inline_index相当于索引
160                 return p->inline_entries + p->inline_index++;
161 
162         if (!table || POLL_TABLE_FULL(table)) {
163                 struct poll_table_page *new_table;
164 
165                 new_table = (struct poll_table_page *) __get_free_page(GFP_KERNEL);
166                 if (!new_table) {
167                         p->error = -ENOMEM;
168                         return NULL;
169                 }
170                 new_table->entry = new_table->entries;
171                 new_table->next = table;
172                 p->table = new_table;
173                 table = new_table;
174         }
175 
176         return table->entry++;
177 }

a.poll_get_entry用于获取poll_wqueues中的poll_table_entry

b.如果poll_wqueues的INLINE空间有空闲entry，则从INLINE空间中分配entry

c.如果INILINE空间没有空闲entry，则分配新页帧作为poll_table_page;新poll_table_page插入链表头，以便下次分配只查看首结点就能知道是否有空闲entry。

 

文件poll唤醒

当复用函数被阻塞后，如果有异步事件出现而是文件能非阻塞的进行key标识的I/O操作时，会调用wake_up_interruptible_sync_poll唤醒被阻塞的复用函数

tcp数据接收事件唤醒复用函数：

tcp_protocol->tcp_v4_rcv->tcp_v4_do_rcv->tcp_rcv_established->sk_data_ready->sock_def_readable->wake_up_interruptible_sync_poll

pipe写事件唤醒复用函数：pipe_write->wake_up_interruptible_sync_poll

164 #define wake_up_interruptible_sync(x)   __wake_up_sync((x), TASK_INTERRUPTIBLE, 1)

 
5897 /*
 5898  * The core wakeup function. Non-exclusive wakeups (nr_exclusive == 0) just
 5899  * wake everything up. If it's an exclusive wakeup (nr_exclusive == small +ve
 5900  * number) then we wake all the non-exclusive tasks and one exclusive task.
 5901  *
 5902  * There are circumstances in which we can try to wake a task which has already
 5903  * started to run but is not in state TASK_RUNNING. try_to_wake_up() returns
 5904  * zero in this (rare) case, and we handle it by continuing to scan the queue.
 5905  */
 5906 static void __wake_up_common(wait_queue_head_t *q, unsigned int mode,
 5907                         int nr_exclusive, int wake_flags, void *key)
 5908 {
 5909         wait_queue_t *curr, *next;
 5910 
 5911         list_for_each_entry_safe(curr, next, &q->task_list, task_list) {
 5912                 unsigned flags = curr->flags;
 5913 
 5914                 if (curr->func(curr, mode, wake_flags, key) &&
 5915                                 (flags & WQ_FLAG_EXCLUSIVE) && !--nr_exclusive)
 5916                         break;
 5917         }
 5918 }
 5954 /**
 5955  * __wake_up_sync_key - wake up threads blocked on a waitqueue.
 5956  * @q: the waitqueue
 5957  * @mode: which threads
 5958  * @nr_exclusive: how many wake-one or wake-many threads to wake up
 5959  * @key: opaque value to be passed to wakeup targets
 5960  *
 5961  * The sync wakeup differs that the waker knows that it will schedule
 5962  * away soon, so while the target thread will be woken up, it will not
 5963  * be migrated to another CPU - ie. the two threads are 'synchronized'
 5964  * with each other. This can prevent needless bouncing between CPUs.
 5965  *
 5966  * On UP it can prevent extra preemption.
 5967  *
 5968  * It may be assumed that this function implies a write memory barrier before
 5969  * changing the task state if and only if any tasks are woken up.
 5970  */
 5971 void __wake_up_sync_key(wait_queue_head_t *q, unsigned int mode,
 5972                         int nr_exclusive, void *key)
 5973 {
 5974         unsigned long flags;
 5975         int wake_flags = WF_SYNC;
 5976 
 5977         if (unlikely(!q))
 5978                 return;
 5979 
 5980         if (unlikely(!nr_exclusive))
 5981                 wake_flags = 0;
 5982 
 5983         spin_lock_irqsave(&q->lock, flags);
 5984         __wake_up_common(q, mode, nr_exclusive, wake_flags, key);
 5985         spin_unlock_irqrestore(&q->lock, flags);
 5986 }
 5987 EXPORT_SYMBOL_GPL(__wake_up_sync_key);
 5988 
 5989 /*
 5990  * __wake_up_sync - see __wake_up_sync_key()
 5991  */
 5992 void __wake_up_sync(wait_queue_head_t *q, unsigned int mode, int nr_exclusive)
 5993 {
 5994         __wake_up_sync_key(q, mode, nr_exclusive, NULL);
 5995 }
 5996 EXPORT_SYMBOL_GPL(__wake_up_sync);      /* For internal use only */

wake_up_interruotible_sync同步唤醒一个等待进程

__wake_up_common中的func是由__pollwait设置成的pollwake

179 static int __pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)
180 {
181         struct poll_wqueues *pwq = wait->private;
182         DECLARE_WAITQUEUE(dummy_wait, pwq->polling_task);
183 
184         /*
185          * Although this function is called under waitqueue lock, LOCK
186          * doesn't imply write barrier and the users expect write
187          * barrier semantics on wakeup functions.  The following
188          * smp_wmb() is equivalent to smp_wmb() in try_to_wake_up()
189          * and is paired with set_mb() in poll_schedule_timeout.
190          */
191         smp_wmb();
192         pwq->triggered = 1;
193 
194         /*
195          * Perform the default wake up operation using a dummy
196          * waitqueue.
197          *
198          * TODO: This is hacky but there currently is no interface to
199          * pass in @sync.  @sync is scheduled to be removed and once
200          * that happens, wake_up_process() can be used directly.
201          */
202         return default_wake_function(&dummy_wait, mode, sync, key);
203 }
204 
205 static int pollwake(wait_queue_t *wait, unsigned mode, int sync, void *key)
206 {
207         struct poll_table_entry *entry;
208 
209         entry = container_of(wait, struct poll_table_entry, wait);
210         if (key && !((unsigned long)key & entry->key))
211                 return 0;
212         return __pollwake(wait, mode, sync, key);
213 }

1.如果唤醒进程的事件不是复用函数所关心的事件，则不会去唤醒复用函数；如果是关心的时间，则调用__pollwake唤醒复用函数。

2.将triggered置为1；在遍历完复用函数所提供的文件描述符表后，如果没有满足的描述符时会阻塞进程；但是如果已经遍历的文件在遍历的过程中，有异步事件出现而使文件能非阻塞的进行key标识的I/O操作时，则不会去阻塞进程；阻塞进程时会检查triggered标识，如果是0才会去阻塞进程，否则不回去阻塞3.通过default_wake_function->try_to_wake_up唤醒复用函数的调用进程（当进程已经是TASK_RUNNING时，则直接返回）

 

poll_wquques释放

127 static void free_poll_entry(struct poll_table_entry *entry)
128 {
129         remove_wait_queue(entry->wait_address, &entry->wait);
130         fput(entry->filp);
131 }
132 
133 void poll_freewait(struct poll_wqueues *pwq)
134 {
135         struct poll_table_page * p = pwq->table;
136         int i;
137         for (i = 0; i < pwq->inline_index; i++)
138                 free_poll_entry(pwq->inline_entries + i);
139         while (p) {
140                 struct poll_table_entry * entry;
141                 struct poll_table_page *old;
142 
143                 entry = p->entry;
144                 do {
145                         entry--;
146                         free_poll_entry(entry);
147                 } while (entry > p->entries);
148                 old = p;
149                 p = p->next;
150                 free_page((unsigned long) old);
151         }
152 }

1.由于在遍历描述符表中文件的过程中，不知道未遍历到的文件能否非阻塞的进行key标识的I/O操作；所以当前文件不能非阻塞的进行key标识的I/O操作时，就会将进程添加到文件的poll等待队列中，以便后续文件不能非阻塞的进行I/O操作时不用再遍历描述符表去将进程添加到文件的poll等待队列中。

2.不管是阻塞被唤醒（进程添加到描述符表中所有文件的poll队列中）还是未阻塞（进程已经添加到描述符表中可进行非阻塞I/O操作文件之前的所有文件的poll队列中），在复用函数退出时，都会调用poll_freewait将poll_wqueues中所有的waiter从文件的等待队列中清空，及释放相应的文件及内存。

poll实现

poll

i. fs/select.c  :

​
898 SYSCALL_DEFINE3(poll, struct pollfd __user *, ufds, unsigned int, nfds,
899                 long, timeout_msecs)
900 {
901         struct timespec end_time, *to = NULL;
902         int ret;
903 
904         if (timeout_msecs >= 0) {
905                 to = &end_time;
906                 poll_select_set_timeout(to, timeout_msecs / MSEC_PER_SEC,
907                         NSEC_PER_MSEC * (timeout_msecs % MSEC_PER_SEC));
908         }
909 
910         ret = do_sys_poll(ufds, nfds, to);
911 
912         if (ret == -EINTR) {
913                 struct restart_block *restart_block;
914 
915                 restart_block = ¤t_thread_info()->restart_block;
916                 restart_block->fn = do_restart_poll;
917                 restart_block->poll.ufds = ufds;
918                 restart_block->poll.nfds = nfds;
919 
920                 if (timeout_msecs >= 0) {
921                         restart_block->poll.tv_sec = end_time.tv_sec;
922                         restart_block->poll.tv_nsec = end_time.tv_nsec;
923                         restart_block->poll.has_timeout = 1;
924                 } else
925                         restart_block->poll.has_timeout = 0;
926 
927                 ret = -ERESTART_RESTARTBLOCK;
928         }
929         return ret;
930 }
​

1.转换超时时间，由ms转化成timespec

2.调用do_sys_poll

3.如果在poll调用被阻塞时收到signal，do_sys_poll则产生EINR错误；此时返回ERESTART_RESTARTBLOCK,通知内核处理完信号后自动通过sys_restart_syscall重启poll调用（这个过程对用户进程而言是透明的）

ii.do_sys_poll

811 int do_sys_poll(struct pollfd __user *ufds, unsigned int nfds,
812                 struct timespec *end_time)
813 {
814         struct poll_wqueues table;
815         int err = -EFAULT, fdcount, len, size;
816         /* Allocate small arguments on the stack to save memory and be
817            faster - use long to make sure the buffer is aligned properly
818            on 64 bit archs to avoid unaligned access */
819         long stack_pps[POLL_STACK_ALLOC/sizeof(long)];
820         struct poll_list *const head = (struct poll_list *)stack_pps;
821         struct poll_list *walk = head;
822         unsigned long todo = nfds;
823 
824         if (nfds > current->signal->rlim[RLIMIT_NOFILE].rlim_cur)
825                 return -EINVAL;
826 
827         len = min_t(unsigned int, nfds, N_STACK_PPS);
828         for (;;) {
829                 walk->next = NULL;
830                 walk->len = len;
831                 if (!len)
832                         break;
833 
834                 if (copy_from_user(walk->entries, ufds + nfds-todo,
835                                         sizeof(struct pollfd) * walk->len))
836                         goto out_fds;
837 
838                 todo -= walk->len;
839                 if (!todo)
840                         break;
841 
842                 len = min(todo, POLLFD_PER_PAGE);
843                 size = sizeof(struct poll_list) + sizeof(struct pollfd) * len;
844                 walk = walk->next = kmalloc(size, GFP_KERNEL);
845                 if (!walk) {
846                         err = -ENOMEM;
847                         goto out_fds;
848                 }
849         }
850 
851         poll_initwait(&table);
852         fdcount = do_poll(nfds, head, &table, end_time);
853         poll_freewait(&table);
854 
855         for (walk = head; walk; walk = walk->next) {
856                 struct pollfd *fds = walk->entries;
857                 int j;
858 
859                 for (j = 0; j < walk->len; j++, ufds++)
860                         if (__put_user(fds[j].revents, &ufds->revents))
861                                 goto out_fds;
862         }
863 
864         err = fdcount;
865 out_fds:
866         walk = head->next
867         while (walk) {
868                 struct poll_list *pos = walk;
869                 walk = walk->next;
870                 kfree(pos);
871         }
872 
873         return err;
874 }

1.nfds检查，是否大于最大打开文件限制signal->rlim[RLIMIT_NOFILE].rlim_cur

2.将用户空间的pollfd数组复制到内核态的poll_list中

3.初始化poll_wqueues

4.调用do_poll

5.释放poll_wqueues

6.将返回事件从内核态的poll_list复制到用户空间的pollfd数组中

7.释放poll_list所占用的slab内存，内核堆栈上的内存会在do_sys_poll返回后自动释放。

iii. do_poll

740 static int do_poll(unsigned int nfds,  struct poll_list *list,
741                    struct poll_wqueues *wait, struct timespec *end_time)
742 {
743         poll_table* pt = &wait->pt;
744         ktime_t expire, *to = NULL;
745         int timed_out = 0, count = 0;
746         unsigned long slack = 0;
747 
748         /* Optimise the no-wait case */
749         if (end_time && !end_time->tv_sec && !end_time->tv_nsec) {
750                 pt = NULL;
751                 timed_out = 1;
752         }
753 
754         if (end_time && !timed_out)
755                 slack = estimate_accuracy(end_time);
756 
757         for (;;) {
758                 struct poll_list *walk;
759 
760                 for (walk = list; walk != NULL; walk = walk->next) {
761                         struct pollfd * pfd, * pfd_end;
762 
763                         pfd = walk->entries;
764                         pfd_end = pfd + walk->len;
765                         for (; pfd != pfd_end; pfd++) {
766                                 /*
767                                  * Fish for events. If we found one, record it
768                                  * and kill the poll_table, so we don't
769                                  * needlessly register any other waiters after
770                                  * this. They'll get immediately deregistered
771                                  * when we break out and return.
772                                  */
773                                 if (do_pollfd(pfd, pt)) {
774                                         count++;
775                                         pt = NULL;
776                                 }
777                         }
778                 }
779                 /*
780                  * All waiters have already been registered, so don't provide
781                  * a poll_table to them on the next loop iteration.
782                  */
783                 pt = NULL;
784                 if (!count) {
785                         count = wait->error;
786                         if (signal_pending(current))
787                                 count = -EINTR;
788                 }
789                 if (count || timed_out)
790                         break;
791 
792                 /*
793                  * If this is the first loop and we have a timeout
794                  * given, then we convert to ktime_t and set the to
795                  * pointer to the expiry value.
796                  */
797                 if (end_time && !to) {
798                         expire = timespec_to_ktime(*end_time);
799                         to = &expire;
800                 }
801 
802                 if (!poll_schedule_timeout(wait, TASK_INTERRUPTIBLE, to, slack))
803                         timed_out = 1;
804         }
805         return count;
806 }

1.遍历poll_list,对每个pollfd进行do_pollfd操作

  a.如果pollfd不能满足poll要求，poll会自动将进程添加到文件的poll等待队列中（见pol阻塞）；

  b.如果pollfd满足poll要求，将计数count加1，poll_table置为NULL，即后续的遍历不用再将进程添加到文件的poll等待队列中；即使该文件不能进行非阻塞的I/O操作也不用将进程添加到文件的等待队列中，因为复用函数只需要有一个可进行非阻塞I/O操作即可

2.遍历结束后

  A.如果poll_list中有pollfd满足poll，则返回满足poll的个数

  B.如果poll_list中没有pollfd满足poll

  a.如果有信号产生，则返回EINTR;内核会先处理信号，再重新恢复poll系统调用

  b.如果超时，则超时返回

  c.如果未超时，则超时阻塞poll系统调用；当有异步事件出现而满足文件poll（如tcp收到数据，满足读poll），唤醒进程，跳到1重新遍历poll_list

 

iv. do_pollfd

702 /*
703  * Fish for pollable events on the pollfd->fd file descriptor. We're only
704  * interested in events matching the pollfd->events mask, and the result
705  * matching that mask is both recorded in pollfd->revents and returned. The
706  * pwait poll_table will be used by the fd-provided poll handler for waiting,
707  * if non-NULL.
708  */
709 static inline unsigned int do_pollfd(struct pollfd *pollfd, poll_table *pwait)
710 {
711         unsigned int mask;
712         int fd;
713 
714         mask = 0;
715         fd = pollfd->fd;
716         if (fd >= 0) {
717                 int fput_needed;
718                 struct file * file;
719 
720                 file = fget_light(fd, &fput_needed);
721                 mask = POLLNVAL;
722                 if (file != NULL) {
723                         mask = DEFAULT_POLLMASK;
724                         if (file->f_op && file->f_op->poll) {
725                                 if (pwait)
726                                         pwait->key = pollfd->events |
727                                                         POLLERR | POLLHUP;
728                                 mask = file->f_op->poll(file, pwait);
729                         }
730                         /* Mask out unneeded events. */
731                         mask &= pollfd->events | POLLERR | POLLHUP;
732                         fput_light(file, fput_needed);
733                 }
734         }
735         pollfd->revents = mask;
736 
737         return mask;
738 }

1.根据pollfd中的文件描述符取出文件file对象

2.调用文件的poll方法；如果满足poll操作，则返回对应的poll事件；否则将进程添加到文件的poll等待队列中（具体过程见poll阻塞）

3.将返回事件取pollfd->events|POLLERR|POLLHUP子集后，放入pollfd->revents中