操作系统实验五·xv6系统内存布局修改·栈空间位置变更与栈不足时扩充

1.实验目的

更改xv6系统的用户内存布局。

2.实验内容

重新安排地址空间，使得其更像linux。
xv6地址空间当前设置如下：
code
stack (fixed-sized, one page)
heap (grows towards the high-end of the address space)
更改为：
code
heap (grows towards the high-end of the address space)
… (gap)
stack (at end of address space; grows backwards)
当堆栈增长超过其分配的页面时，它将导致页面错误，然后增长栈长度

3. 实验手册

From assignment:
“Right now, the program memory map is determined by how we load the program into memory and set up the page table (so that they are pointing to the right physical pages). This is all implemented in exec.c as part of the exec system call using the underlying support provided to implement virtual memory in vm.c. To change the memory layout, you have to change the exec code to load the program and allocate the stack in the new way that we want.”
Specifically, lets start by opening up exec.c check the exec(…) function which implements the system call. Exec does the following:
Part 1. Opens the executable file and parses it. The rest of this paragraph is FYI. Typically executable files are made up of a header including information that allows us to index the rest of the file. The file after that consists of sections including the code, the global data, and sometimes other sections like uninitialized data. These are the parts of the memory that we need to initialize from the executable. The header information includes the number of sections, the start of each section in the file, and where it maps to in virtual memory, and the length of each section.
Part 2. Initializes the kernel memory using setupkvm() which maps the pages of the kernel to the process address space. We dont really need to know what happens in here.
Part 3. It then moves on to load the sections of the executable file into memory using loaduvm() which creates the memory pages for each section and maps them to the address space (by initializing the page table pointers – more details in a bit). These sections in xv6 are loaded starting at VA 0, and going up. Each new section starts at the beginning of a new page. Recall that sections include code, global/static data, etc…
Conveniently, we can then keep track of where the user address space ends, which also defines the size of the process using one value (proc->sz). So, as we map new pages, sz (rounded up to the next page) can serve as their virtual address since we are simply filling in the address space sequentially.
In VM, we map virtual pages to physical frames. XV6 has no swap so all memory pages have to be in physical memory. Physical memory is allocated by the kernel allocator kmalloc. If you want to dig deeper (you need to for the bonus part) you will see how these pages are initialized using kinit() and kinit2() which are called during the boot process in main.c. As a result, we use kmalloc() as we request each new page inside vm.c to allocate a new physical page.
The vm.c functions such as allocuvm() typically follow this up with a call to mappages which is used to initialize the page table entries mapping the virtual address to the physical page that it just allocated. Otherwise these physical frames that we allocate cannot be used by our process.
Part 4. At this point, we loaded code and data sections, and its time to create the stack. xv6 does not support a heap at the moment (there is no malloc()/free() available to user programs if you noticed). It currently maps the stack in its virtual address space at a page right after the last page we loaded from the executable (i.e., at sz rounded up to the next page boundary).
Since the stack grows down, allocating a page here means there is no room to grow the stack – as it grows down, it will run into the code/data. To protect against that, xv6 adds one page buffer and marks it as unreadable so that in case the stack grows, we get a memory error and can stop the program. The code to create the stack is:

sz = PGROUNDUP(sz);   //round sz up to the next page boundary since stack must start in a new page
  if((sz = allocuvm(pgdir, sz, sz + 2*PGSIZE)) == 0) // our first introduction to allocuvm; it allocates and maps two pages
    goto bad;
  clearpteu(pgdir, (char*)(sz - 2*PGSIZE)); //we clear the PTE for the first page to create a buffer page between stack and code/data

TODO 1: This is the part of the code that we need to change to move the stack. The current code calls allocuvm to create two pages, one for the stack and one as a guard page starting at VA sz which is right after the code and data. It then clears the page table entry for the guard page.
We want to locate the stack starting at the top of the user address space to give it room to grow. To understand what we need to do, lets look at allocuvm.
It takes 3 parameters:
a.The page table (pgdir). This will not change
b.The virtual address of the first page we are mapping – this needs to change to point to the top page of the user part of memory (right under KERNBASE). If you use kernbase, you will try to map the page to the kernel address space.
c. The virtual address of the last page we are mapping. For us, we are creating a stack with only a single page, so this can another address in the same page, slightly bigger than the first address.
Allocuvm allocates the page, and maps it to the page table. So, basically we are done with moving the stack by just changing these parameters to the right value. However, there are a few loose ends to tie up.
Part 5: Finally, we initialize the stack pointer, currently to sz.
TODO 2: you will have to change this to the address of the top word in the stack page. Note that KERNBASE is the first word in the kernel address space, so this is the word right under that.
We proceed to initialize the stack with the parameters for main as per the linux/x86 stack convention. The details are not important to us for now.
Loose Ends/Other changes.
Now that we moved the stack, a few places in the Kernel that hard coded the previous location of the stack have to be changed. These include:
TODO 3: All of the functions that are defined in syscall.c (and sysfile.c) for accessing the user stack have some checks to see if the addresses are indeed on the stack. These checks compare the address against sz since that was the top of the stack in the old implementation. You have to change those checks (or remove them if it is easier). Check
all the accessor functions such as argint, argstr, argptr, argfd, etc…
TODO 4: copyuvm(). This function is used as part of fork() to create a copy of the address space of the parent process calling fork to the child process. It is implemented in vm.c
If you look at this function, it is one big for loop that iterates over the virtual address space and copies the pages one by one. The loop starts with:
for(i = 0; i < sz; i += PGSIZE){
since it assumes the virtual address space starts at 0 and goes to sz. Now this has to be changed to take into account the new stack.
If we look deeper, it reads the page table to get the PTE for the page, allocates a new physical frames, and copies the page from the parent memory to the new page. Finally it uses mappages to map this new copy to the child address space by adding a PTE to its page table.
How do we change it? Now sz tracks the bottom part of the address space, so its ok to leave that loop alone. We have to keep track of the size of the stack, and added another loop that iterates over the stack page(s) and does the same thing (kmallocs a page for each one, memmoves to create a copy from the parent, and then mapages() to add it to page table).
The loop will be very similar, with the exception of the virtual address ranges that iterates over. Before we add stack growth, the stack is only one page, but as the stack grows we need to keep track of the number of stack pages. To prepare for this, we need to add a
variable in struct proc to keep track of the size of the stack (in pages or bytes–either is fine, but I recommend pages). This counter starts out with a stack of one page; set it in exec().
Debugging: If your stack moved correctly, xv6 will be able to boot into shell successfully.
If you dont allocate/map the stack correctly, you will get errors either in the allocation functions (e.g., remapping errors) or as your program runs (page faults).
If you dont take care of all the argint() etc… functions some of your system calls will not be able to pass parameters correctly. The results could be weird. For example, printf won’t print, and wait wont wait (leading to init continuing to fork processes, etc…)
Growing the stack
Now that our stack has been moved, we have room to grow it. When the a program causes the stack to grow to an offset bigger than one page, at this point, we will be accessing a page that is not allocated/mapped. This will cause a page fault. Basically, we will trap to the trap handler in trap.c
In there there is a switch statement with a case for every supported trap. We need to add a case for page faults. This page fault has trap number 14 (or T_PGFLT) as defined in traps.h
TODO 5: Add a case for the page fault. When a page fault occurs, you can check the address that caused the page fault in a hardware register called CR2. The CR register (standing for Control Register) keep track of important hardware state information. You can read the CR2 register using the function rcr2().
Once you have the offending address, next we need to check if it is from the page right under the current bottom of the stack. If it is, we need to grow the stack. You can use allocuvm again, but you have to initialize it with the right parameters to allocate one page at the right place. After that, you can increment your stack size counter, which finishes your trap handler.
Voila! you should be good to go.
To check if the stack grows correctly, write a recursive program that nests deep enough to get a stack longer than 1 page. You should get a page fault and grow the stack correctly if your implementation works.

4. 实验环境

Linux虚拟机
操作系统：Ubantu 16.04 32位
虚拟机软件：VMware Workstation 15
虚拟处理器：1个2核

5. 程序设计和实现

5.1系统函数修改

5.1.1 exec.c
修改栈分配地区。

|--------------|--------------|---------------|
             page1          栈底          KERNBASE
           0x7FFFE000     0x7FFFF000     0x80000000

int
exec(char *path, char **argv)
{
    char *s, *last;
    int i, off;
    uint argc, sz, stack_pos, sp, ustack[3 + MAXARG + 1];
    struct elfhdr elf;
    struct inode *ip;
    struct proghdr ph;
    pde_t *pgdir, *oldpgdir;
    struct proc *curproc = myproc();

    begin_op();

    if ((ip = namei(path)) == 0)
    {
        end_op();
        cprintf("exec: fail\n");
        return -1;
    }
    ilock(ip);
    pgdir = 0;

    // Check ELF header
    //打开可执行文件
    if (readi(ip, (char *) &elf, 0, sizeof(elf)) != sizeof(elf))
        goto bad;
    if (elf.magic != ELF_MAGIC)
        goto bad;
    //初始化内核内存
    if ((pgdir = setupkvm()) == 0)
        goto bad;

    // Load program into memory.
    sz = 0;
    for (i = 0, off = elf.phoff; i < elf.phnum; i++, off += sizeof(ph))
    {
        if (readi(ip, (char *) &ph, off, sizeof(ph)) != sizeof(ph))
            goto bad;
        if (ph.type != ELF_PROG_LOAD)
            continue;
        if (ph.memsz < ph.filesz)
            goto bad;
        if (ph.vaddr + ph.memsz < ph.vaddr)
            goto bad;
        //这里的sz就相当于进程大小
        //sz是用来跟踪虚拟地址空间的末尾
        //我们按顺序填充地址空间
        if ((sz = allocuvm(pgdir, sz, ph.vaddr + ph.memsz)) == 0)
            goto bad;
        if (ph.vaddr % PGSIZE != 0)
            goto bad;
        //将程序的每个部分装入内存中,为每个部分创建内存页并映射到地址空间
        if (loaduvm(pgdir, (char *) ph.vaddr, ip, ph.off, ph.filesz) < 0)
            goto bad;
    }
    iunlockput(ip);
    end_op();
    ip = 0;

    // Allocate two pages at the next page boundary.
    // Make the first inaccessible.  Use the second as the user stack.
    //原来用户栈分配,第一个不可访问,第二个是用户栈
    sz = PGROUNDUP(sz);
    //初始化页表项和物理内存,提供虚拟地址映射
    if ((stack_pos = allocuvm(pgdir, KERNBASE - 2 * PGSIZE, KERNBASE - PGSIZE)) == 0)
        goto bad;
    //clearpteu(pgdir, (char *) (stack_pos - 2 * PGSIZE));
    sp = stack_pos;
    stack_pos -= PGSIZE;

    // Push argument strings, prepare rest of stack in ustack.
    for (argc = 0; argv[argc]; argc++)
    {
        if (argc >= MAXARG)
            goto bad;
        sp = (sp - (strlen(argv[argc]) + 1)) & ~3;
        if (copyout(pgdir, sp, argv[argc], strlen(argv[argc]) + 1) < 0)
            goto bad;
        ustack[3 + argc] = sp;
    }
    ustack[3 + argc] = 0;

    ustack[0] = 0xffffffff;  // fake return PC
    ustack[1] = argc;
    ustack[2] = sp - (argc + 1) * 4;  // argv pointer

    sp -= (3 + argc + 1) * 4;
    if (copyout(pgdir, sp, ustack, (3 + argc + 1) * 4) < 0)
        goto bad;

    // Save program name for debugging.
    for (last = s = path; *s; s++)
        if (*s == '/')
            last = s + 1;
    safestrcpy(curproc->name, last, sizeof(curproc->name));

    // Commit to the user image.
    oldpgdir = curproc->pgdir;
    curproc->pgdir = pgdir;
    curproc->sz = sz;
    curproc->tf->eip = elf.entry;  // main
    curproc->tf->esp = sp;

    curproc->stack_pos = stack_pos;

    switchuvm(curproc);
    freevm(oldpgdir);
    return 0;

    bad:
    if (pgdir)
        freevm(pgdir);
    if (ip)
    {
        iunlockput(ip);
        end_op();
    }
    return -1;
}

5.1.2 proc.h
在proc中增添一项用于记录栈顶

// Per-process state
struct proc {
  uint sz;                     // Size of process memory (bytes)
  pde_t* pgdir;                // Page table
  char *kstack;                // Bottom of kernel stack for this process
  enum procstate state;        // Process state
  int pid;                     // Process ID
  struct proc *parent;         // Parent process
  struct trapframe *tf;        // Trap frame for current syscall
  struct context *context;     // swtch() here to run process
  void *chan;                  // If non-zero, sleeping on chan
  int killed;                  // If non-zero, have been killed
  struct file *ofile[NOFILE];  // Open files
  struct inode *cwd;           // Current directory
  char name[16];               // Process name (debugging)
  uint stack_pos;
};

5.1.3 vm.c
修改copyuvm函数，新增一个自变量用于记录栈顶，这样使得在fork的时候把我们的栈也会拷贝上

// Given a parent process's page table, create a copy
// of it for a child.
pde_t *
copyuvm(pde_t *pgdir, uint sz, uint stack_pos)
{
    pde_t *d;
    pte_t *pte;
    uint pa, i, flags;
    char *mem;

    if ((d = setupkvm()) == 0)
        return 0;
    for (i = 0; i < sz; i += PGSIZE)
    {
        if ((pte = walkpgdir(pgdir, (void *) i, 0)) == 0)
            panic("copyuvm: pte should exist");
        if (!(*pte & PTE_P))
            panic("copyuvm: page not present");
        pa = PTE_ADDR(*pte);
        flags = PTE_FLAGS(*pte);
        if ((mem = kalloc()) == 0)
            goto bad;
        memmove(mem, (char *) P2V(pa), PGSIZE);
        if (mappages(d, (void *) i, PGSIZE, V2P(mem), flags) < 0)
        {
            kfree(mem);
            goto bad;
        }
    }
    for (i = stack_pos; i < KERNBASE - PGSIZE; i += PGSIZE)
    {
        if ((pte = walkpgdir(pgdir, (void *) i, 0)) == 0)
            panic("copyuvm: pte should exist");
        if (!(*pte & PTE_P))
            panic("copyuvm: page not present");
        pa = PTE_ADDR(*pte);
        flags = PTE_FLAGS(*pte);
        if ((mem = kalloc()) == 0)
            goto bad;
        memmove(mem, (char *) P2V(pa), PGSIZE);
        if (mappages(d, (void *) i, PGSIZE, V2P(mem), flags) < 0)
        {
            kfree(mem);
            goto bad;
        }
    }
    return d;

    bad:
    freevm(d);
    return 0;
}

5.1.4 proc.c
修改fork中使用copyuvm的地方

// Create a new process copying p as the parent.
// Sets up stack to return as if from system call.
// Caller must set state of returned proc to RUNNABLE.
int
fork(void)
{
  int i, pid;
  struct proc *np;
  struct proc *curproc = myproc();

  // Allocate process.
  if((np = allocproc()) == 0){
    return -1;
  }

  // Copy process state from proc.
  if((np->pgdir = copyuvm(curproc->pgdir, curproc->sz,curproc->stack_pos)) == 0){
    kfree(np->kstack);
    np->kstack = 0;
    np->state = UNUSED;
    return -1;
  }
  np->sz = curproc->sz;
  np->parent = curproc;
  *np->tf = *curproc->tf;

  // Clear %eax so that fork returns 0 in the child.
  np->tf->eax = 0;

  for(i = 0; i < NOFILE; i++)
    if(curproc->ofile[i])
      np->ofile[i] = filedup(curproc->ofile[i]);
  np->cwd = idup(curproc->cwd);
  safestrcpy(np->name, curproc->name, sizeof(curproc->name));
  pid = np->pid;
  acquire(&ptable.lock);
  np->state = RUNNABLE;
  release(&ptable.lock);
  return pid;
}

5.1.5 trap.c
增加系统中断陷阱，在这里实现栈的增长

case T_PGFLT:
            if (rcr2() < 0x7FFFF000)
            {
                cprintf("page error %x ",rcr2());
                cprintf("stack pos : %x\n", myproc()->stack_pos);
                if ((myproc()->stack_pos = allocuvm(myproc()->pgdir, myproc()->stack_pos - 1 * PGSIZE,
                                                    myproc()->stack_pos)) == 0)
                {
                    myproc()->killed = 1;
                }
                myproc()->stack_pos-=PGSIZE;
                cprintf("create a new page %x\n", myproc()->stack_pos);
                //clearpteu(myproc()->pgdir, (char *) (myproc()->stack_pos - PGSIZE));
                return;
            }
            else
            {
                myproc()->killed = 1;
                break;
            }

5.1.6 syscall.c
修改一切以sz为程序大小判断的地方

// User code makes a system call with INT T_SYSCALL.
// System call number in %eax.
// Arguments on the stack, from the user call to the C
// library system call function. The saved user %esp points
// to a saved program counter, and then the first argument.

// Fetch the int at addr from the current process.
int
fetchint(uint addr, int *ip)
{
    *ip = *(int *) (addr);
    return 0;
}

// Fetch the nul-terminated string at addr from the current process.
// Doesn't actually copy the string - just sets *pp to point at it.
// Returns length of string, not including nul.
int
fetchstr(uint addr, char **pp)
{
    char *s, *ep;
    struct proc *curproc = myproc();

    *pp = (char *) addr;
    ep = (char *) curproc->sz;
    for (s = *pp; s < ep; s++)
    {
        if (*s == 0)
            return s - *pp;
    }
    return -1;
}

// Fetch the nth 32-bit system call argument.
int
argint(int n, int *ip)
{
    return fetchint((myproc()->tf->esp) + 4 + 4 * n, ip);
}

// Fetch the nth word-sized system call argument as a pointer
// to a block of memory of size bytes.  Check that the pointer
// lies within the process address space.
int
argptr(int n, char **pp, int size)
{
    int i;

    if (argint(n, &i) < 0)
        return -1;
    *pp = (char *) i;
    return 0;
}

// Fetch the nth word-sized system call argument as a string pointer.
// Check that the pointer is valid and the string is nul-terminated.
// (There is no shared writable memory, so the string can't change
// between this check and being used by the kernel.)
int
argstr(int n, char **pp)
{
    int addr;
    if (argint(n, &addr) < 0)
        return -1;
    return fetchstr(addr, pp);
}

5.1.7 defs.h
vm.c中修改

pde_t*          copyuvm(pde_t*, uint, uint);

5.1.8 teststack.c

//
// Created by zhj12399 on 2021/12/26.
//
#include "types.h"
#include "stat.h"
#include "user.h"

int
main(int argc, char *argv[])
{
    printf(0, "\nargument num : %d\n", argc - 1);
    int n = 100;
    if (argc > 1)
    {
        for (int i = 1; i < argc; i++)
        {
            printf(0, "%s\n", argv[i]);
        }
        n = atoi(argv[1]);
        printf(0, "\ncreate %d int array\n", n);
    }
    else
    {
        printf(0, "\ncreate 100 int array\n");
    }

    int num[n];
    memset(num, 0, sizeof (num));
    printf(0,"%x\n",num);

    int pid = fork();
    if (pid < 0)
    {
        printf(0, "fork error!\n");
    }
    else if (pid == 0)
    {
        printf(0, "\nchild %d fork\n", getpid());

        printf(0, "***child***\n");
    }
    else
    {
        wait();
        printf(0, "parent %d kill\n\n", getpid());
    }
    return 0;
}

5.1.10 Makefile
修改Makefile，将需要编译的程序加入到清单中

    _teststack\

5.2 编译运行

make qemu 启动xv6系统后执行我们的函数teststack，栈中存储的是函数输出的参数以及函数中的变量，我们这里先不加参数执行，fork，print等函数均运行正常

加参数运行，我们设置一个int大小为1000的数组，会发现其产生了页面异常报错，然后创建了一个新的页面

再创建一个int为3000的数组，不断的产生页面异常然后创建新的页面并移动栈的范围

最后一个数字7FFFC0C0也为我们展示了创建的数组的起始位置

6. 实验结果和分析

成功将栈移动到了用户空间的最后部分，并且可以使得在栈空间不足的情况下增加栈的空间。