对于epoll实现原理的理解

Linux服务器开发

已于 2022-09-22 14:42:07 修改

阅读量264

点赞数

分类专栏： C++开发 Linux开发 epoll 文章标签： epoll 网络协议栈 Linux开发 C++开发 select

于 2022-09-22 14:36:06 首次发布

本文链接：https://blog.csdn.net/Linuxhus/article/details/126991123

版权

C++开发同时被 3 个专栏收录

139 篇文章 1 订阅

订阅专栏

Linux开发

68 篇文章 1 订阅

订阅专栏

epoll

19 篇文章 1 订阅

订阅专栏

本文详细探讨了epoll在Linux中的实现原理，包括epoll的数据结构、锁的机制、回调函数以及LT与ET的工作模式。通过对epoll的深入理解，阐述了其在处理IO多路复用时的优势和常见误区。

摘要由CSDN通过智能技术生成

Epoll是Linux IO的多路复用的机制，是select/poll的增强版本，在Linux内核fs/eventpoll.c中可以查看epoll的具体的实现。

一、epoll数据结构

　　学习任何组件，首先得知道它有什么数据结构或者数据类型，epoll主要有两个结构体：eventpoll和epitem。epitem是每一个IO对应的事件，比如EPOLL_CTL_ADD操作时，就需要创建一个epitem；eventpoll是每一个epoll所对应的，比如epoll_create就是创建一个eventpoll。

struct epitem {
    union {
        /* RB tree node links this structure to the eventpoll RB tree */
        struct rb_node rbn;
        /* Used to free the struct epitem */
        struct rcu_head rcu;
    };

    /* List header used to link this structure to the eventpoll ready list */
    struct list_head rdllink;

    /*
     * Works together "struct eventpoll"->ovflist in keeping the
     * single linked chain of items.
     */
    struct epitem *next;

    /* The file descriptor information this item refers to */
    struct epoll_filefd ffd;//sockfd

    /* List containing poll wait queues */
    struct eppoll_entry *pwqlist;

    /* The "container" of this item */
    struct eventpoll *ep;

    /* List header used to link this item to the "struct file" items list */
    struct hlist_node fllink;

    /* wakeup_source used when EPOLLWAKEUP is set */
    struct wakeup_source __rcu *ws;

    /* The structure that describe the interested events and the source fd */
    struct epoll_event event;
};

struct eventpoll {

    struct mutex mtx;

    /* Wait queue used by sys_epoll_wait() */
    wait_queue_head_t wq;

    /* Wait queue used by file->poll() */
    wait_queue_head_t poll_wait;

    /* List of ready file descriptors */
    struct list_head rdllist;

    /* Lock which protects rdllist and ovflist */
    rwlock_t lock;

    /* RB tree root used to store monitored fd structs */
    struct rb_root_cached rbr;

    /*
     * This is a single linked list that chains all the "struct epitem" that
     * happened while transferring ready events to userspace w/out
     * holding ->lock.
     */
    struct epitem *ovflist;

    /* wakeup_source used when ep_scan_ready_list is running */
    struct wakeup_source *ws;

    /* The user that created the eventpoll descriptor */
    struct user_struct *user;

    struct file *file;

    /* used to optimize loop detection check */
    u64 gen;
    struct hlist_head refs;

#ifdef CONFIG_NET_RX_BUSY_POLL
    /* used to track busy poll napi_id */
    unsigned int napi_id;
#endif

#ifdef CONFIG_DEBUG_LOCK_ALLOC
    /* tracks wakeup nests for lockdep validation */
    u8 nests;
#endif
};

　数据结构如下图所示。

　　list用来存储就绪的IO，rbtree用来存储所有IO，方便快速查找fd，这两种数据结构我们都从inster和remove来讨论。对于list，当内核IO准备就绪时，则执行epoll_event_callback的回调函数，将epitem添加到list中；当epoll_wait激活重新运行时，将list的epitem 逐一拷贝到events中，并删除list中被拷贝出来的epitem。

对于rbtree又该何时添加何时删除呢？当app执行epoll_ctl(EPOLL_CTL_ADD)操作，将epitem添加到rbtree中；当app执行epoll_ctl(EPOLL_CTL_DEL)操作，将对应的epitem从rbtree中删除。那么list和rbtree又如何做到线程安全呢？