GeekOS源代码学习(5)Main函数中Init_Scheduler

最新推荐文章于 2024-03-22 11:24:28 发布

wu5795175

最新推荐文章于 2024-03-22 11:24:28 发布

阅读量1.6k

点赞数

分类专栏： GeekOS 文章标签： thread struct function list pointers assembly

本文链接：https://blog.csdn.net/wu5795175/article/details/7455199

版权

GeekOS 专栏收录该内容

16 篇文章 6 订阅

订阅专栏

来看Main中的下一个函数Init_Scheduler

Init_Scheduler函数开启了系统的多任务机制。

位于./src/geekos/kthread.c

void Init_Scheduler(void)
{
    struct Kernel_Thread* mainThread = (struct Kernel_Thread *) KERN_THREAD_OBJ;

    /*
     * Create initial kernel thread context object and stack,
     * and make them current.
     */
    Init_Thread(mainThread, (void *) KERN_STACK, PRIORITY_NORMAL, true);
    g_currentThread = mainThread;
    Add_To_Back_Of_All_Thread_List(&s_allThreadList, mainThread);

    /*
     * Create the idle thread.
     */
    /*Print("starting idle thread\n");*/
    Start_Kernel_Thread(Idle, 0, PRIORITY_IDLE, true);

    /*
     * Create the reaper thread.
     */
    /*Print("starting reaper thread\n");*/
    Start_Kernel_Thread(Reaper, 0, PRIORITY_NORMAL, true);
}

粗粗的看一下，可以看到它初始化了内核线程mainThread，启动了一个Idle线程和Reaper线程。
内核线程mainThread表示的是现在运行的Main函数这个线程。

看一下struct Kernel_Thread内核线程结构
位于./include/geekos/kthread.h

/*
 * Kernel thread context data structure.
 * NOTE: there is assembly code in lowlevel.asm that depends
 * on the offsets of the fields in this struct, so if you change
 * the layout, make sure everything gets updated.
 */
struct Kernel_Thread {
    ulong_t esp;			 /* offset 0 */
    volatile ulong_t numTicks;		 /* offset 4 */
    int priority;
    DEFINE_LINK(Thread_Queue, Kernel_Thread);
    void* stackPage;
    struct User_Context* userContext;
    struct Kernel_Thread* owner;
    int refCount;

    /* These fields are used to implement the Join() function */
    bool alive;
    struct Thread_Queue joinQueue;
    int exitCode;

    /* The kernel thread id; also used as process id */
    int pid;

    /* Link fields for list of all threads in the system. */
    DEFINE_LINK(All_Thread_List, Kernel_Thread);

    /* Array of MAX_TLOCAL_KEYS pointers to thread-local data. */
#define MAX_TLOCAL_KEYS 128
    const void* tlocalData[MAX_TLOCAL_KEYS];

};

Kernel_Thread是一个内核线程的信息结构体，包括pid，优先级，堆栈指针，退出码，双向链表等。

看Init_Scheduler中的下一句Init_Thread
位于./src/geekos/kthread.c

/*
 * Initialize a new Kernel_Thread.
 */
static void Init_Thread(struct Kernel_Thread* kthread, void* stackPage,
	int priority, bool detached)
{
    static int nextFreePid = 1;

    struct Kernel_Thread* owner = detached ? (struct Kernel_Thread*)0 : g_currentThread;

    memset(kthread, '\0', sizeof(*kthread));
    kthread->stackPage = stackPage;
    kthread->esp = ((ulong_t) kthread->stackPage) + PAGE_SIZE;//指向进程堆栈底部的指针
    kthread->numTicks = 0;
    kthread->priority = priority;
    kthread->userContext = 0;
    kthread->owner = owner;

    /*
     * The thread has an implicit self-reference.
     * If the thread is not detached, then its owner
     * also has a reference to it.
     */
    kthread->refCount = detached ? 1 : 2;

    kthread->alive = true;
    Clear_Thread_Queue(&kthread->joinQueue);
    kthread->pid = nextFreePid++;

}

Init_Thread函数为线程分配了栈空间，并初始化优先级priority，线程运行滴答numticks等。

下一句
g_currentThread = mainThread
把mainThread赋给全局变量g_currentThread，表示此线程当前在运行
Add_To_Back_Of_All_Thread_List(&s_allThreadList, mainThread)将此结构体链入到结构体链表中

再看下一句
Start_Kernel_Thread(Idle, 0, PRIORITY_IDLE, true)

和Start_Kernel_Thread(Reaper, 0, PRIORITY_NORMAL, true)
启动了Idle和Reaper两个线程,这两个内核线程会在后面详细说明。

Start_Kernel_Thread位于./src/geekos/kthread.c

/*
 * Start a kernel-mode-only thread, using given function as its body
 * and passing given argument as its parameter.  Returns pointer
 * to the new thread if successful, null otherwise.
 *
 * startFunc - is the function to be called by the new thread
 * arg - is a paramter to pass to the new function
 * priority - the priority of this thread (use PRIORITY_NORMAL) for
 *    most things
 * detached - use false for kernel threads
 */
struct Kernel_Thread* Start_Kernel_Thread(
    Thread_Start_Func startFunc,
    ulong_t arg,
    int priority,
    bool detached
)
{
    struct Kernel_Thread* kthread = Create_Thread(priority, detached);
    if (kthread != 0) {
	/*
	 * Create the initial context for the thread to make
	 * it schedulable.
	 */
	Setup_Kernel_Thread(kthread, startFunc, arg);


	/* Atomically put the thread on the run queue. */
	Make_Runnable_Atomic(kthread);
    }

    return kthread;
}

Create_Thread用来为线程分配栈空间，初始化线程结构；Setup_Kernel_Thread用来将此线程的上下文压入堆栈中，压入的上下文是一些初始值；

Make_Runnable_Atomic用来将此线程链入到运行队列中。

先看Create_Thread函数

位于/src/geekos/kthread.c

/*
 * Create a new raw thread object.
 * Returns a null pointer if there isn't enough memory.
 */
static struct Kernel_Thread* Create_Thread(int priority, bool detached)
{
    struct Kernel_Thread* kthread;
    void* stackPage = 0;

    /*
     * For now, just allocate one page each for the thread context
     * object and the thread's stack.
     */
    kthread = Alloc_Page();
    if (kthread != 0)
        stackPage = Alloc_Page();    

    /* Make sure that the memory allocations succeeded. */
    if (kthread == 0)
	return 0;
    if (stackPage == 0) {
	Free_Page(kthread);
	return 0;
    }

    /*Print("New thread @ %x, stack @ %x\n", kthread, stackPage); */

    /*
     * Initialize the stack pointer of the new thread
     * and accounting info
     */
    Init_Thread(kthread, stackPage, priority, detached);

    /* Add to the list of all threads in the system. */
    Add_To_Back_Of_All_Thread_List(&s_allThreadList, kthread);

    return kthread;
}

Create_Thread函数首先用Alloc_Page分配了一个页的线程结构体和一个页的堆栈，这两个物理页就是一个线程的家当了。

Alloc_Page会在系统可用页链表中搜索一个可用页，然后返回这个页的首地址。

然后调用Init_Thread初始化线程，并将此线程结构链入到链表s_allThreadList中,s_allThreadList是系统中所有线程的链表，包括可运行的线程和休眠线程。

Init_Thread在前面初始化mainthread时已经看到了。

回到Start_Kernel_Thread函数中看下一句
Setup_Kernel_Thread将创建的线程的上下文压入到堆栈中。
位于./src/geekos/kthread.c

/*
 * Set up the initial context for a kernel-mode-only thread.
 */
static void Setup_Kernel_Thread(
    struct Kernel_Thread* kthread,
    Thread_Start_Func startFunc,
    ulong_t arg)
{
    /*
     * Push the argument to the thread start function, and the
     * return address (the Shutdown_Thread function, so the thread will
     * go away cleanly when the start function returns).
     */
    Push(kthread, arg);
    Push(kthread, (ulong_t) &Shutdown_Thread);

    /* Push the address of the start function. */
    Push(kthread, (ulong_t) startFunc);//用户指定的函数

    /*
     * To make the thread schedulable, we need to make it look
     * like it was suspended by an interrupt.  This means pushing
     * an "eflags, cs, eip" sequence onto the stack,
     * as well as int num, error code, saved registers, etc.
     */

    /*
     * The EFLAGS register will have all bits clear.
     * The important constraint is that we want to have the IF
     * bit clear, so that interrupts are disabled when the
     * thread starts.
     */
    Push(kthread, 0UL);  /* EFLAGS */

    /*
     * As the "return address" specifying where the new thread will
     * start executing, use the Launch_Thread() function.
     */
    Push(kthread, KERNEL_CS);
    Push(kthread, (ulong_t) &Launch_Thread);//线程被初次调度运行时，首先运行的是Launch_Thread函数

    /* Push fake error code and interrupt number. */
    Push(kthread, 0);
    Push(kthread, 0);

    /* Push initial values for general-purpose registers. */
    Push_General_Registers(kthread);//首次这里压入的通用寄存器值全为0

    /*
     * Push values for saved segment registers.
     * Only the ds and es registers will contain valid selectors.
     * The fs and gs registers are not used by any instruction
     * generated by gcc.
     */
    Push(kthread, KERNEL_DS);  /* ds */
    Push(kthread, KERNEL_DS);  /* es */
    Push(kthread, 0);  /* fs */
    Push(kthread, 0);  /* gs */
}

可以看到Setup_Kernel_Thread向堆栈中压入了线程的个方面的信息，包括线程参数，线程启动函数和退出函数，各寄存器信息等。

注意这里入栈的参数先后顺序是事先约定的。

Push(kthread,0UL)开始到最后这些压入的参数是用于还原现场的。

极其要注意首先入栈的那几个参数,这是理解线程执行的关键。

当切换到线程的时候，先还原上下文，然后跳到Launch_Thread去执行，Launch_Thread执行完后，最后一句ret指令，系统从栈中弹出函数地址startFunc继续执行，

startFunc执行完后，最后一句汇编ret再弹出函数Shutdown_Thread结束线程。

怎么理解这种用压入函数地址来调用函数的方式呢。

可以用如下代码来理解

...
call Shutdown_Thread
...



Shutdown_Thread:
		call startFunc
		...
		...
		ret

startFunc:
		call Launch_Thread
		...
		...
		ret

Launch_Thread
		...
		...
		...
		ret

这样，栈中就会连续的压入Shutdown_Thread,startFunc,Launch_Thread的入口地址，依次执行Launch_Thread,startFunc,Shutdown_Thread，然后在ret中依次返回。

执行ret，CPU就会从栈中取出4B载入到eip中执行。

Push函数位于./src/geekos/kthread.c

/*
 * Push a dword value on the stack of given thread.
 * We use this to set up some context for the thread before
 * we make it runnable.
 */
static __inline__ void Push(struct Kernel_Thread* kthread, ulong_t value)
{
    kthread->esp -= 4;
    *((ulong_t *) kthread->esp) = value;
}

kthread->esp在初始化线程的时候指向线程的堆栈底部，注意它是先减4再赋值。

回到Start_Kernel_Thread中看最后一句Make_Runnable_Atomic(kthread)，将线程链路到可运行队列中，由于可运行队列是全局可见的，所以这里加了临界保护。
位于./src/geekos/kthread.c

/*
 * Atomically make a thread runnable.
 * Assumes interrupts are currently enabled.
 */
void Make_Runnable_Atomic(struct Kernel_Thread* kthread)
{
    Disable_Interrupts();
    Make_Runnable(kthread);
    Enable_Interrupts();
}

看Make_Runnable函数，仍旧在kthread.c中

/*
 * Add given thread to the run queue, so that it
 * may be scheduled.  Must be called with interrupts disabled!
 */
void Make_Runnable(struct Kernel_Thread* kthread)
{
    KASSERT(!Interrupts_Enabled());

    Enqueue_Thread(&s_runQueue, kthread);
}

Enqueue_Thread将线程加入到系统的运行队列中，至此线程正式开始了它的生命周期，将会被调度执行。

至此Init_Scheduler()函数结束。

可以看到它初始化了内核主线程，以及两个Idle和Reaper两个内核线程。

还有一些具体细节会在下次添加。

Idle是系统初始化时产生的线程，其优先级为PRIORITY_IDLE，一直位于系统的可运行队列中(因为是while(1)循环),是系统中最低的级别，也就是说，只有系统运行队列中没有其他的可运行进程了，才会调度Idle运行。

位于./src/geekos/kthread.c

/*
 * This is the body of the idle thread.  Its job is to preserve
 * the invariant that a runnable thread always exists,
 * i.e., the run queue is never empty.
 */
static void Idle(ulong_t arg)
{
    while (true)
	Yield();
}

可以看到，Idle任务循环调用Yield函数。
Yield函数用于调度系统中的线程运行。
位于./src/geekos/kthread.c，因为会操作到全局的运行队列，所以要进行临界区保护。

/*
 * Voluntarily give up the CPU to another thread.
 * Does nothing if no other threads are ready to run.
 */
void Yield(void)
{
    Disable_Interrupts();
    Make_Runnable(g_currentThread);
    Schedule();
    Enable_Interrupts();
}

Yield中主要有Make_Runnable和Schedule两个函数，Make_Runnable将当前进程进入运行队列，Schedule调度运行队列中的线程。

看Make_Runnable()是做什么的
位于./src/geekos/kthread.c

/*
 * Add given thread to the run queue, so that it
 * may be scheduled.  Must be called with interrupts disabled!
 */
void Make_Runnable(struct Kernel_Thread* kthread)
{
    KASSERT(!Interrupts_Enabled());

    Enqueue_Thread(&s_runQueue, kthread);
}

Enqueue_Thread函数把内核线程链入到运行队列中去

参数s_runQueue是运行队列的头，定义如下
位于./src/geekos/kthread.c

/*
 * Queue of runnable threads.
 */
static struct Thread_Queue s_runQueue;

再看Thread_Queue的定义
位于./include/geekos/kthread.h
DEFINE_LIST(Thread_Queue, Kernel_Thread);
这里使用DEFINE_LIST宏定义了链表头Thread_Queue的结构

再看一下函数Enqueue_Thread()
位于./include/geekos/kthread.h

static __inline__ void Enqueue_Thread(struct Thread_Queue *queue, struct Kernel_Thread *kthread) {
    Add_To_Back_Of_Thread_Queue(queue, kthread);
}

就是把线程结构体加入到内核的可运行队列中去。

回到Yield函数看下一个函数Schedule()
位于./src/geekos/kthread.c

/*
 * Schedule a thread that is waiting to run.
 * Must be called with interrupts off!
 * The current thread should already have been placed
 * on whatever queue is appropriate (i.e., either the
 * run queue if it is still runnable, or a wait queue
 * if it is waiting for an event to occur).
 */
void Schedule(void)
{
    struct Kernel_Thread* runnable;

    /* Make sure interrupts really are disabled */
    KASSERT(!Interrupts_Enabled());

    /* Preemption should not be disabled. */
    KASSERT(!g_preemptionDisabled);

    /* Get next thread to run from the run queue */
    runnable = Get_Next_Runnable();

    /*
     * Activate the new thread, saving the context of the current thread.
     * Eventually, this thread will get re-activated and Switch_To_Thread()
     * will "return", and then Schedule() will return to wherever
     * it was called from.
     */
    Switch_To_Thread(runnable);
}

看Get_Next_Runnable()函数，它从运行队列中取出下一个运行线程
位于./src/geekos/kthread.c

/*
 * Get the next runnable thread from the run queue.
 * This is the scheduler.
 */
struct Kernel_Thread* Get_Next_Runnable(void)
{
    struct Kernel_Thread* best = 0;

    best = Find_Best(&s_runQueue);
    KASSERT(best != 0);
    Remove_Thread(&s_runQueue, best);

/*
 *    Print("Scheduling %x\n", best);
 */
    return best;
}

看Find_Best()函数，它返回运行队列中优先级最高的线程
位于./src/geekos/kthread.c

/*
 * Find the best (highest priority) thread in given
 * thread queue.  Returns null if queue is empty.
 */
static __inline__ struct Kernel_Thread* Find_Best(struct Thread_Queue* queue)
{
    /* Pick the highest priority thread */
    struct Kernel_Thread *kthread = queue->head, *best = 0;
    while (kthread != 0) {
	if (best == 0 || kthread->priority > best->priority)
	    best = kthread;
	kthread = Get_Next_In_Thread_Queue(kthread);
    }
    return best;
}

可以看到函数Find_Best从头开始遍历整个链表，然后返回在运行队列中优先级最高的线程。

再回到Get_Next_Runnable()中，下一个函数Remove_Thread()
把线程best从运行队列中删除了。
Remove_Thread位于./include/geekos/kthread.h

static __inline__ void Remove_Thread(struct Thread_Queue *queue, struct Kernel_Thread *kthread) {
    Remove_From_Thread_Queue(queue, kthread);
}

回到Schedule()
得到下一个要运行的线程结构体之后，调用函数Switch_To_Thread切换到此线程。
Switch_To_Thread这个函数保存了当前线程(即Idle这个线程)的上下文，然后切换到新线程的堆栈中。
Switch_To_Thread位于./src/geekos/lowlevel.asm

; ----------------------------------------------------------------------
; Switch_To_Thread()
;   Save context of currently executing thread, and activate
;   the thread whose context object is passed as a parameter.
; 
; Parameter: 
;   - ptr to Kernel_Thread whose state should be restored and made active
;
; Notes:
; Called with interrupts disabled.
; This must be kept up to date with definition of Kernel_Thread
; struct, in kthread.h.
; ----------------------------------------------------------------------
align 16
Switch_To_Thread://栈中存储着函数的第一个参数(欲运行的线程结构体)
	; Modify the stack to allow a later return via an iret instruction.
	; We start with a stack that looks like this:
	;
	;            thread_ptr
	;    esp --> return addr
	;
	; We change it to look like this:
	;
	;            thread_ptr
	;            eflags
	;            cs
	;    esp --> return addr	//调整栈状态，因为将来系统会在其他的线程上下文中使用iret切换回来。iret需要从栈中弹出eflag、cs、return addr(线程运行地址)

	push	eax		; save eax
	mov	eax, [esp+4]	; get return address，得到返回地址存到eax中，也就是Schedule中Switch_To_Thread函数执行完后的下条指令地址
	mov	[esp-4], eax	; move return addr down 8 bytes from orig loc，将返回地址向下移8B
	add	esp, 8		; move stack ptr up
	pushfd			; put eflags where return address was
	mov	eax, [esp-4]	; restore saved value of eax
	push	dword KERNEL_CS	; push cs selector
	sub	esp, 4		; point stack ptr at return address，这句运行结束后，堆栈状态就是上面的图示。这里压入的线程参数会在下次

	; Push fake error code and interrupt number，压入错误码，和中断向量，这里是线程切换，无需理会中断向量，所以中断向量设为0即可。
	push	dword 0
	push	dword 0

	; Save general purpose registers.
	Save_Registers

	; Save stack pointer in the thread context struct (at offset 0).g_currentThread指向当前执行的线程，这里就是得到Idle这个线程的结构体指针，并把esp赋给Idle线程结构体
	mov	eax, [g_currentThread]
	mov	[eax+0], esp

	; Clear numTicks field in thread context, since this
	; thread is being suspended.Idle线程结构体中的numTicks变量置0,因为Idle将被挂起
	mov	[eax+4], dword 0

	; Load the pointer to the new thread context into eax.
	; We skip over the Interrupt_State struct on the stack to
	; get the parameter.将要运行的线程的结构体指针赋给eax
	mov	eax, [esp+INTERRUPT_STATE_SIZE]

	; Make the new thread current, and switch to its stack.将线程设置为当前运行线程，并切换esp到新的堆栈。
	mov	[g_currentThread], eax
	mov	esp, [eax+0]


	; Restore general purpose and segment registers, and clear interrupt
	; number and error code.从新的esp栈中弹出数据，在这里还原新进程的上下文。
	Restore_Registers

	; We'll return to the place where the thread was
	; executing last.切换到新进程了，这里是极其核心，极其关键的一步！！！注意iret和ret的区别，iret还还原了寄存器cs和eflag。
	iret

对照Interrupt_State结构看，可以看到，Switch_To_Thread就是先逆序将各个参数入栈。
然后切换到新的线程上下文。
理解最后一句iret至关重要，iret执行完后，新进程开始执行它的第一条指令。
而原来这个Idle进程的Switch_To_Thread也完全的完成了任务，再返回运行Idle线程时候，执行的就是到Schedule函数中的下一句了。
为统一上下文结构，线程上下文和中断上下文Interrupt_State保持了一致。
Interrupt_State结构位于./include/geekos/int.h

/*
 * This struct reflects the contents of the stack when
 * a C interrupt handler function is called.
 * It must be kept up to date with the code in "lowlevel.asm".
 */
struct Interrupt_State {
    /*
     * The register contents at the time of the exception.
     * We save these explicitly.
     */
    uint_t gs;
    uint_t fs;
    uint_t es;
    uint_t ds;
    uint_t ebp;
    uint_t edi;
    uint_t esi;
    uint_t edx;
    uint_t ecx;
    uint_t ebx;
    uint_t eax;

    /*
     * We explicitly push the interrupt number.
     * This makes it easy for the handler function to determine
     * which interrupt occurred.
     */
    uint_t intNum;

    /*
     * This may be pushed by the processor; if not, we push
     * a dummy error code, so the stack layout is the same
     * for every type of interrupt.
     */
    uint_t errorCode;

    /* These are always pushed on the stack by the processor. */
    uint_t eip;
    uint_t cs;
    uint_t eflags;
};

至此Idle线程完毕。。

这里要说明一下这里的线程切换函数Switch_To_Thread和通用中断处理函数Handle_Interrupt的区别和联系。

都在./src/geekos/lowlevel.asm中

它们的共同点都是会将当前进程的上下文保存。
具体过程：
Handle_Interrupt过程：
保护被中断进程的上下文---->从栈中取出中断向量---->根据中断向量取出中断处理函数地址并调用中断处理函数---->中断处理函数结束，返回到Handle_Interrupt---->判断是否需要调度新进程---->需要则调度新进程，调用Make_Runnable得到新进程的上下文并恢复执行;不需要调度则恢复被中断进程的上下文，继续执行被中断进程。

Switch_To_Thread过程：
保护当前进程的上下文---->从栈中取出欲运行进程的结构体---->切换到欲运行进程堆栈，恢复其上下文---->跳到欲运行进程执行。

Handle_Interrupt中使用了三个系统全局变量：g_currentThread、g_preemptionDisabled、g_needReschedule。
Switch_To_Thread中只使用了一个系统全局变量：g_currentThread

Handle_Interrupt是由于外部中断触发，有特权级的变换。

Switch_To_Thread是在某任务A中调用函数Schedule触发，无特权级变换。

再来看另外一个内核线程Reaper，Reaper线程用于回收进程退出时的内存。

位于/src/geekos/kthread.c中

/*
 * The reaper thread.  Its job is to de-allocate memory
 * used by threads which have finished.
 */
static void Reaper(ulong_t arg)
{
    struct Kernel_Thread *kthread;

    Disable_Interrupts();

    while (true) {
	/* See if there are any threads needing disposal. s_graveyardQueue是等待回收内存的队列*/
	if ((kthread = s_graveyardQueue.head) == 0) {
	    /* Graveyard is empty, so wait for a thread to die. */
	    Wait(&s_reaperWaitQueue);
	}
	else {
	    /* Make the graveyard queue empty.清空队列，下面要回收队列中的所有内存 */
	    Clear_Thread_Queue(&s_graveyardQueue);

	    /*
	     * Now we can re-enable interrupts, since we
	     * have removed all the threads needing disposal.
	     */
	    Enable_Interrupts();
	    Yield();   /* allow other threads to run? 因为Reaper线程永远不会退出，所以要给其他线程运行的机会，这里没有Yield的话系统就死机了*/

	    /* Dispose of the dead threads.遍历整个链表 */
	    while (kthread != 0) {
		struct Kernel_Thread* next = Get_Next_In_Thread_Queue(kthread);
#if 0
		Print("Reaper: disposing of thread @ %x, stack @ %x\n",
		    kthread, kthread->stackPage);
#endif
		Destroy_Thread(kthread);//清除线程的栈空间，以及线程结构体，将线程从系统所有线程队列中删除
		kthread = next;
	    }

	    /*
	     * Disable interrupts again, since we're going to
	     * do another iteration.
	     */
	    Disable_Interrupts();
	}
    }
}

可以看到，和Idle线程一样，Reaper也是一个恒久的线程，使用while无限循环。
这里说一下在队列上阻塞等待的Wait函数，和唤醒函数Wake_Up是一对，看一下他俩的实现
位于/src/geekos/kthread.c

/*
 * Wait on given wait queue.
 * Must be called with interrupts disabled!
 * Note that the function will return with interrupts
 * disabled.  This is desirable, because it allows us to
 * atomically test a condition that can be affected by an interrupt
 * and wait for it to be satisfied (if necessary).
 * See the Wait_For_Key() function in keyboard.c
 * for an example.
 */
void Wait(struct Thread_Queue* waitQueue)
{
    struct Kernel_Thread* current = g_currentThread;

    KASSERT(!Interrupts_Enabled());

    /* Add the thread to the wait queue. */
    Enqueue_Thread(waitQueue, current);

    /* Find another thread to run. */
    Schedule();
}

/*
 * Wake up all threads waiting on given wait queue.
 * Must be called with interrupts disabled!
 * See Keyboard_Interrupt_Handler() function in keyboard.c
 * for an example.
 */
void Wake_Up(struct Thread_Queue* waitQueue)
{
    struct Kernel_Thread *kthread = waitQueue->head, *next;

    KASSERT(!Interrupts_Enabled());

    /*
     * Walk throught the list of threads in the wait queue,
     * transferring each one to the run queue.
     */
    while (kthread != 0) {
	next = Get_Next_In_Thread_Queue(kthread);
	Make_Runnable(kthread);
	kthread = next;
    }

    /* The wait queue is now empty. */
    Clear_Thread_Queue(waitQueue);
}

Wait将当前线程进入特定的等待队列，并调度其他线程运行。
Wake则唤醒等待队列中的所有线程，唤醒的就是将其进入系统运行队列，调度程序会从运行队列中取线程执行。
Reaper将线程从系统中永远的删除了，就像它从来没有来过一样。

线程进入s_graveyardQueue就是进入了墓地，等待着最终的躯壳的消散。

在Setup_Kernel_Thread中我们压入了线程退出函数Shutdown_Thread，线程运行完用户执行的函数时候，就会运行Shutdown_Thread

Shutdown_Thread内就是一个Exit(0)函数。

结合Exit(0)函数来看，就能明白Reaper线程的功能了。

/*
 * Exit the current thread.
 * Calling this function initiates a context switch.
 */
void Exit(int exitCode)
{
    struct Kernel_Thread* current = g_currentThread;

    if (Interrupts_Enabled())
	Disable_Interrupts();

    /* Thread is dead */
    current->exitCode = exitCode;
    current->alive = false;

    /* Clean up any thread-local memory */
    Tlocal_Exit(g_currentThread);

    /* Notify the thread's owner, if any */
    Wake_Up(¤t->joinQueue);

    /* Remove the thread's implicit reference to itself. */
    Detach_Thread(g_currentThread);

    /*
     * Schedule a new thread.
     * Since the old thread wasn't placed on any
     * thread queue, it won't get scheduled again.
     */
    Schedule();

    /* Shouldn't get here */
    KASSERT(false);
}

Exit置线程的退出码和存活标志，然后调用Tlocal_Exit释放将线程的本地内存。
再调用Wake_Up通知线程的所有者。Wake_Up将唤醒current->joinQueue上的所有线程。
Detach_Thread则通知Reaper内核线程"收割"此线程。
Detach_Thread位于./src/geekos/kthread.c

/*
 * Hand given thread to the reaper for destruction.
 * Must be called with interrupts disabled!
 */
static void Reap_Thread(struct Kernel_Thread* kthread)
{
    KASSERT(!Interrupts_Enabled());
    Enqueue_Thread(&s_graveyardQueue, kthread);
    Wake_Up(&s_reaperWaitQueue);
}

/*
 * Called when a reference to the thread is broken.
 */
static void Detach_Thread(struct Kernel_Thread* kthread)
{
    KASSERT(!Interrupts_Enabled());
    KASSERT(kthread->refCount > 0);

    --kthread->refCount;
    if (kthread->refCount == 0) {
	Reap_Thread(kthread);
    }
}

可以看到，当线程的引用技术refCount为0时，说明此线程彻底结束了，Reap_Thread将线程进入坟墓s_graveyardQueue，并唤醒s_reaperWaitQueue队列上的所有线程(其实此队列上只有Reaper一个线程)。