6.S081 Lab4 Lazy allocation

6.S081 Lab4 Lazy allocation

0. 说明

这个部分完全是 6.S081-6缺页异常Page Fault （我的课程学习笔记）的第二节的内容。我这里并不是完全实现了它的要求，可能少了一些边界条件的判断，但是应该是够用了。

1. 实验要求如下

Lab: xv6 lazy page allocation

One of the many neat tricks an O/S can play with page table hardware is lazy allocation of user-space heap memory. Xv6 applications ask the kernel for heap memory using the sbrk() system call. In the kernel we’ve given you, sbrk() allocates physical memory and maps it into the process’s virtual address space. It can take a long time for a kernel to allocate and map memory for a large request. Consider, for example, that a gigabyte consists of 262,144 4096-byte pages; that’s a huge number of allocations even if each is individually cheap. In addition, some programs allocate more memory than they actually use (e.g., to implement sparse arrays), or allocate memory well in advance of use. To allow sbrk() to complete more quickly in these cases, sophisticated kernels allocate user memory lazily. That is, sbrk() doesn’t allocate physical memory, but just remembers which user addresses are allocated and marks those addresses as invalid in the user page table. When the process first tries to use any given page of lazily-allocated memory, the CPU generates a page fault, which the kernel handles by allocating physical memory, zeroing it, and mapping it. You’ll add this lazy allocation feature to xv6 in this lab.

Before you start coding, read Chapter 4 (in particular 4.6) of the xv6 book, and related files you are likely to modify:

• kernel/trap.c
• kernel/vm.c
• kernel/sysproc.c

To start the lab, switch to the lazy branch:

  $ git fetch
  $ git checkout lazy
  $ make clean

Eliminate allocation from sbrk() (easy)

Your first task is to delete page allocation from the sbrk(n) system call implementation, which is the function sys_sbrk() in sysproc.c. The sbrk(n) system call grows the process’s memory size by n bytes, and then returns the start of the newly allocated region (i.e., the old size). Your new sbrk(n) should just increment the process’s size (myproc()->sz) by n and return the old size. It should not allocate memory – so you should delete the call to growproc() (but you still need to increase the process’s size!).

Try to guess what the result of this modification will be: what will break?

Make this modification, boot xv6, and type echo hi to the shell. You should see something like this:

init: starting sh
$ echo hi
usertrap(): unexpected scause 0x000000000000000f pid=3
            sepc=0x0000000000001258 stval=0x0000000000004008
va=0x0000000000004000 pte=0x0000000000000000
panic: uvmunmap: not mapped

The “usertrap(): …” message is from the user trap handler in trap.c; it has caught an exception that it does not know how to handle. Make sure you understand why this page fault occurs. The “stval=0x0…04008” indicates that the virtual address that caused the page fault is 0x4008.

Lazy allocation (moderate)

Modify the code in trap.c to respond to a page fault from user space by mapping a newly-allocated page of physical memory at the faulting address, and then returning back to user space to let the process continue executing. You should add your code just before the printf call that produced the “usertrap(): …” message. Modify whatever other xv6 kernel code you need to in order to get echo hi to work.

Here are some hints:

• You can check whether a fault is a page fault by seeing if r_scause() is 13 or 15 in usertrap().
• r_stval() returns the RISC-V stval register, which contains the virtual address that caused the page fault.
• Steal code from uvmalloc() in vm.c, which is what sbrk() calls (via growproc()). You’ll need to call kalloc() and mappages().
• Use PGROUNDDOWN(va) to round the faulting virtual address down to a page boundary.
• uvmunmap() will panic; modify it to not panic if some pages aren’t mapped.
• If the kernel crashes, look up sepc in kernel/kernel.asm
• Use your vmprint function from pgtbl lab to print the content of a page table.
• If you see the error “incomplete type proc”, include “spinlock.h” then “proc.h”.

If all goes well, your lazy allocation code should result in echo hi working. You should get at least one page fault (and thus lazy allocation), and perhaps two.

Lazytests and Usertests (moderate)

We’ve supplied you with lazytests, an xv6 user program that tests some specific situations that may stress your lazy memory allocator. Modify your kernel code so that all of both lazytests and usertests pass.

• Handle negative sbrk() arguments.
• Kill a process if it page-faults on a virtual memory address higher than any allocated with sbrk().
• Handle the parent-to-child memory copy in fork() correctly.
• Handle the case in which a process passes a valid address from sbrk() to a system call such as read or write, but the memory for that address has not yet been allocated.
• Handle out-of-memory correctly: if kalloc() fails in the page fault handler, kill the current process.
• Handle faults on the invalid page below the user stack.

Your solution is acceptable if your kernel passes lazytests and usertests:

$  lazytests
lazytests starting
running test lazy alloc
test lazy alloc: OK
running test lazy unmap...
usertrap(): ...
test lazy unmap: OK
running test out of memory
usertrap(): ...
test out of memory: OK
ALL TESTS PASSED
$ usertests
...
ALL TESTS PASSED
$

Submit the lab

This completes the lab. Make sure you pass all of the make grade tests. If this lab had questions, don’t forget to write up your answers to the questions in answers-lab-name.txt. Commit your changes (including adding answers-lab-name.txt) and type make handin in the lab directory to hand in your lab.Time spentCreate a new file, time.txt, and put in it a single integer, the number of hours you spent on the lab. Don’t forget to git add and git commit the file.SubmitYou will turn in your assignments using the submission website. You need to request once an API key from the submission website before you can turn in any assignments or labs.

After committing your final changes to the lab, type make handin to submit your lab.

$ git commit -am "ready to submit my lab"
[util c2e3c8b] ready to submit my lab
 2 files changed, 18 insertions(+), 2 deletions(-)

$ make handin
tar: Removing leading `/' from member names
Get an API key for yourself by visiting https://6828.scripts.mit.edu/2020/handin.py/
Please enter your API key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 79258  100   239  100 79019    853   275k --:--:-- --:--:-- --:--:--  276k
$

make handin will store your API key in myapi.key. If you need to change your API key, just remove this file and let make handin generate it again (myapi.key must not include newline characters).

If you run make handin and you have either uncomitted changes or untracked files, you will see output similar to the following:

 M hello.c
?? bar.c
?? foo.pyc
Untracked files will not be handed in.  Continue? [y/N]

Inspect the above lines and make sure all files that your lab solution needs are tracked i.e. not listed in a line that begins with ??. You can cause git to track a new file that you create using git add filename.

If make handin does not work properly, try fixing the problem with the curl or Git commands. Or you can run make tarball. This will make a tar file for you, which you can then upload via our web interface.

• Please run make grade to ensure that your code passes all of the tests
• Commit any modified source code before running make handin
• You can inspect the status of your submission and download the submitted code at https://6828.scripts.mit.edu/2020/handin.py/

2. Lazy allocation具体实现

(0) 再次说明

1.我并没有完成他们的全部要求（可以通过全部测试用例是需要判断更多边界条件的，这样很累，但是我的初衷是复习，因此一部分条件就省去了）。但是我这里的处理是完全完成了一个基础的lazy allocation (缺少边界条件)

2.实现思路： sbrk的时候，do nothing，只是p->sz会增加相应的page数量，但是并没有真正的分配内存。之后的某个时间段，用户如果用到了那部分并没有被真正分配的内存，就会触发page fault，因为这个时候还没有把新的内存映射到page table。——如果我们发现，我们调用的page地址并不存在（page fault），并且这个地址小于p->sz（因为p->sz是目前真正拥有地址 + n ，即理应拥有的page 数量），这个时候我们希望OS可以为我们分配一个内存page，并重新开始执行指令。

(1) sbrk

sbrk是XV6提供的系统调用，它使得用户应用程序能扩大自己的heap。当一个应用程序启动的时候，sbrk指向的是heap的最底端，同时也是stack的最顶端。这个位置通过代表进程的数据结构中的sz字段表示，这里以p->sz表示。

当调用sbrk时，它的参数是整数，代表了你想要申请的字节的数量。

用户程序倾向于申请多于自己需求的内存数量，sbrk的内存申请是eager allocation（一旦调用，就会去申请空间，让用户的heap变大）——但是很多内存可能永远都用不上。

(2) Lazy page allocation

使用lazy allocation——思想：sbrk的时候，do nothing，只是p->sz会增加相应的page数量，但是并没有真正的分配内存。之后的某个时间段，用户如果用到了那部分并没有被真正分配的内存，就会触发page fault，因为这个时候还没有把新的内存映射到page table。——如果我们发现，我们调用的page地址并不存在（page fault），并且这个地址小于p->sz（因为p->sz是目前真正拥有地址 + n ，即理应拥有的page 数量），这个时候我们希望OS可以为我们分配一个内存page，并重新开始执行指令。

过程：

if (page fault) {
	if (stack < page fault vm_address < p -> sz) {
		page fault handler {
			kalloc 1 page and initialize it to 0;
			map this page to user page table;
			return to the page fault address;
		}
	}
}

<1> 修改sys_brk() —— 触发page fault

原始代码👇：

uint64
sys_sbrk(void)
{
  int addr;
  int n;

  if(argint(0, &n) < 0)
    return -1;
  addr = myproc()->sz;
  if(growproc(n) < 0)
    return -1;
  return addr;
}

修改函数，让她不分配内存，只是将p->sz增加n👇

uint64
sys_sbrk(void)
{
  int addr;
  int n;

  if(argint(0, &n) < 0)
    return -1;
  addr = myproc()->sz;
  myproc -> sz = myporc() -> sz + n;
  // if(growproc(n) < 0)
  // return -1;
  return addr;
}

shell 执行echo hi，得到page fault👇（熟悉的panic）

$echo hi
usertrap(): unexpected scause 0x000000000000000f pid = 3
			sepc=0x00000000000012a4 stval=0x0000000000004008
panic: uvmunmap: not mapped

• SCAUSE寄存器 = 15，说明这是一个store page fault

• pid = 3，这个应该是shell的pid

• sepc寄存器的值 ： 0x12a4

• 出错的虚拟内存地址（stval） 0x4008

page fault是出现在malloc的实现代码中。这也非常合理，在malloc的实现中，我们使用sbrk系统调用来获得一些内存，之后会初始化我们刚刚获取到的内存，在0x12a4位置，刚刚获取的内存中写入数据，但是实际上我们在向未被分配的内存写入数据。

<2> Lazy allocation 的实现：如何处理page fault

首先打开usertrap函数👇（由于我20年曾经做过这个实验，所以可能和原始代码有出入，即else 里面的注释部分）：

//
// handle an interrupt, exception, or system call from user space.
// called from trampoline.S
//
void
usertrap(void)
{
  int which_dev = 0;

  if((r_sstatus() & SSTATUS_SPP) != 0)
    panic("usertrap: not from user mode");

  // send interrupts and exceptions to kerneltrap(),
  // since we're now in the kernel.
  w_stvec((uint64)kernelvec);

  struct proc *p = myproc();
  
  // save user program counter.
  p->tf->epc = r_sepc();
  
  if(r_scause() == 8){
    // system call

    if(p->killed)
      exit(-1);

    // sepc points to the ecall instruction,
    // but we want to return to the next instruction.
    p->tf->epc += 4;

    // an interrupt will change sstatus &c registers,
    // so don't enable until done with those registers.
    intr_on();

    syscall();
  } else if((which_dev = devintr()) != 0){
    // ok
  } else {
    printf("usertrap(): unexpected scause %p pid=%d\n", r_scause(), p->pid);
    printf("            sepc=%p stval=%p\n", r_sepc(), r_stval());
    // printf("page down:%d\n",PGROUNDDOWN(r_stval()));
    // printf("r :%d\n",r_scause());
    // int sz = 0;
    // while( !(r_stval()>=sz && r_stval()<sz+4096) )
    // {
    //   sz = sz + 4096;
    // }
    // printf("sz:%d\n",sz);
    p->killed = 1;
  }

  if(p->killed)
    exit(-1);

  // give up the CPU if this is a timer interrupt.
  if(which_dev == 2)
    yield();

  usertrapret();
}

在6.S081 附加Lab1 用户执行系统调用的过程（Trap）中已经提到，r_scause() == 8是系统调用，所以这里增加一个trap的处理即r_scause() == 15：此时我们需要一些定制化的处理。

什么处理？

检查p->sz是否大于STVAL，如果大于就分配物理内存。

  else if(r_scause() == 15)
  {
    uint64 va = r_stval();
    printf("page fault %p\n", va);
    uint64 ka = (uint64)kalloc();
    if (ka == 0) {
      p -> killed  = 1;
    } else {
      memset((void *)ka, 0, PGSIZE);
      va = PGROUNDDOWN(va);
      if (mappages(p -> pagetable, va, PGSIZE, ka, PTE_W | PTE_U | PTE_R) != 0) {
        kfree((void *)ka);
        p -> killed = 1;
      }
    }
  }

增加上面的代码，会分配一个物理page，如果ka == 0，说明没有足够的物理内存，需要kill当前进程。如果有物理内存，首先会将内存内容设置为0，之后将物理内存page指向用户地址空间中合适的虚拟内存地址。具体来说，我们首先将虚拟地址向下取整，这里引起page fault的虚拟地址是0x4008，向下取整之后是0x4000。之后我们将物理内存地址跟取整之后的虚拟内存地址的关系加到page table中。对应的PTE需要设置常用的权限标志位，在这里是u，w，r bit位。

接下来继续echo hi，发现报错信息反而变多了👇。 —— unmap的是之前lazy allocated，但是又还没有用到的地址。——因为都是调用sbrk实现的所以对于这个内存，并没有对应的物理内存。所以在uvmunmap函数中，当PTE的v标志位为0并且没有对应的mapping，这并不是一个实际的panic，这是我们预期的行为。

$ echo hi
page fault 0x0000000000004008
page fault 0x0000000000013f48
panic: uvmunmap: not mapped

// Remove mappings from a page table. The mappings in
// the given range must exist. Optionally free the
// physical memory.
void
uvmunmap(pagetable_t pagetable, uint64 va, uint64 size, int do_free)
{
  uint64 a, last;
  pte_t *pte;
  uint64 pa;

  a = PGROUNDDOWN(va);
  last = PGROUNDDOWN(va + size - 1);
  for(;;){
    if((pte = walk(pagetable, a, 0)) == 0)
      panic("uvmunmap: walk");
    if((*pte & PTE_V) == 0)
      panic("uvmunmap: not mapped");
    if(PTE_FLAGS(*pte) == PTE_V)
      panic("uvmunmap: not a leaf");
    if(do_free){
      pa = PTE2PA(*pte);
      kfree((void*)pa);
    }
    *pte = 0;
  }
}

找到👇（此时不再是panic，而是do nothing）

if((*pte & PTE_V) == 0)
   panic("uvmunmap: not mapped");      

可以修改为

if((*pte & PTE_V) == 0)
  continue;

接下来，我们再重新编译XV6，并执行“echo hi”。——仍然有page fault，但是我们成功执行了。

$ echo hi
page fault 0x0000000000004008
page fault 0x0000000000013f48
hi

——这就是最基本的lazy allocation的实现。