MIT6.S081实验五学习记录

这个实验比较友好，因为Kasshoek基本上把前两个实验都讲了，但是我在试验中也遇到了一个一直没过的样例，最后也是查阅资料，看网上各路大神的代码修改通过的。

首先切换分支到实验五

$ git fetch
$ git checkout lazy
$ make clean

1、Eliminate allocation from sbrk() (easy)

第一个实验就是对 sbrk() 函数做一个修改，让他不再调用growproc()，而是直接把地址增加。视频中讲到，xv6本来是一种eager allocation，也就是需要就立即分配，而现在这个改动就是让需要内存时不分配，而是只做一个记录。进程现在知道自己有了一块更大的内存空间，但是这个内存实际上还没有分配给他，这就是lazy allocation。这部分代码相当简单。

## /kernel/sysproc.c
uint64
sys_sbrk(void)
{
  int addr;
  int n;

  if(argint(0, &n) < 0)
    return -1;
  addr = myproc()->sz;
  //只需要将新增地址的值赋给myproc()->sz，但没有实际分配内存
  uint64 newsize = addr + n;
  myproc()->sz = newsize; 
  //if(growproc(n) < 0)
  //  return -1;
  return addr;
}

然后可以直接编译，输入echo hi，发现会报错，正是因为“承诺”给进程的内存，实际并没有分配给它。而且对照下面的图，显示是usertrap里面的缺页中断。

2、Lazy allocation (moderate)

这个实验其实是在修正上一个实验未完成的部分，而在上一个实验中发现，如果仅仅记录而不分配，就会导致缺页中断（r_scause=13或者r_scause=15）。所以需要在发生中断时，判断类型，然后给相应的进程分配好内存。

## /kernel/trap.c
void
usertrap(void)
{
  if(r_scause() == 8){
    ...
  } else if((which_dev = devintr()) != 0){
  }
  
  //***********从这里开始添加判断***********//
  else if(r_scause() == 13 || r_scause() == 15){
  //r_stval()是中断处的虚拟地址，需要判断其是否合法
    if(r_stval() > p->sz || r_stval() < p->trapframe->sp){
      p->killed = 1;
    }else{
    //向下取值到页面边界
      uint64 va = PGROUNDDOWN(r_stval());
    //分配一段内存
      char* mem = kalloc();
    //分配成功
      if(mem != 0){
        memset(mem, 0 ,PGSIZE);
     //建立映射，失败就kill进程
        if(mappages(p->pagetable, va, PGSIZE, (uint64)mem, PTE_W|PTE_X|PTE_R|PTE_U) != 0){
            kfree(mem);
            p->killed = 1;
          }
       }else{//分配失败，说明没有多余内存，直接kill
        p->killed = 1;
       }
    }
  }
  /****************/
  else {
    printf("usertrap(): unexpected scause %p pid=%d\n", r_scause(), p->pid);
    printf("            sepc=%p stval=%p\n", r_sepc(), r_stval());
    p->killed = 1;
  }

  if(p->killed)
    exit(-1);
  // give up the CPU if this is a timer interrupt.
  if(which_dev == 2)
    yield();
  usertrapret();
}

修改完后，再编译整个xv6，可以看到echo hi已经可以打印出来hi了。

$ echo hi
hi

3、Lazytests and Usertests (moderate)

最后这个实验比较复杂，而且debug过程比较繁琐。因此建议跟着实验的hints来进行。
首先需要来处理如果n为负的情况，这样应该对相应的映射做一个缩小的操作。因为在这里，还不一定完成实际分配内存，所以要使用函数uvmunmap()。

uint64
sys_sbrk(void)
{
  int addr;
  int n;

  if(argint(0, &n) < 0)
    return -1;
  addr = myproc()->sz;
  uint64 newsize = addr + n;
  //如果新加n后，大于了最大地址，就不分配n了
  if(newsize >= MAXVA)
    return addr;
  //使用uvmumap来缩小
  if(n < 0){
    uvmunmap(myproc()->pagetable, PGROUNDUP(newsize), (PGROUNDUP(addr) - PGROUNDUP(newsize)) / PGSIZE, 1);
  }
  myproc()->sz = newsize;
  //if(growproc(n) < 0)
  //  return -1;
  return addr;
}

然后trap.c里面的***usertrap()***函数是基本不用改的，因为上面已经做了判断。接下来就是将一些panic处理一下，让其直接进入下一页，而不是报错。

## /kernel/vm.c
int
uvmcopy(pagetable_t old, pagetable_t new, uint64 sz)
{
  pte_t *pte;
  uint64 pa, i;
  uint flags;
  char *mem;

  for(i = 0; i < sz; i += PGSIZE){
    if((pte = walk(old, i, 0)) == 0)
      continue;  //这里修改1
      //panic("uvmcopy: pte should exist");
    if((*pte & PTE_V) == 0)
      //panic("uvmcopy: page not present");
      continue; //这里修改2
    pa = PTE2PA(*pte);
    flags = PTE_FLAGS(*pte);
    if((mem = kalloc()) == 0)
      goto err;
    memmove(mem, (char*)pa, PGSIZE);
    if(mappages(new, i, PGSIZE, (uint64)mem, flags) != 0){
      kfree(mem);
      goto err;
    }
  }
  return 0;

 err:
  uvmunmap(new, 0, i / PGSIZE, 1);
  return -1;
}

void
uvmunmap(pagetable_t pagetable, uint64 va, uint64 npages, int do_free)
{
  uint64 a;
  pte_t *pte;

  if((va % PGSIZE) != 0)
    panic("uvmunmap: not aligned");

  for(a = va; a < va + npages*PGSIZE; a += PGSIZE){
    if((pte = walk(pagetable, a, 0)) == 0)
      //panic("uvmunmap: walk");
      continue;  //这里修改1
    if((*pte & PTE_V) == 0)
      //panic("uvmunmap: not mapped");
      continue;  //这里修改2
    if(PTE_FLAGS(*pte) == PTE_V)
      panic("uvmunmap: not a leaf");
    if(do_free){
      uint64 pa = PTE2PA(*pte);
      kfree((void*)pa);
    }
    *pte = 0;
  }
}

这两处修改完后，我就跑了一下测试用例。发现 sbrkbug 报了终端错误，而且是指令缺页中断(12号),sbrkarg 也报错，说使用write()出现了问题。write()属于系统调用，usertrap是捕捉不到的。它是调用了 sys_write 操作，而 sys_write 最终调用了copyin的操作，如果使用lazy allocation，那么实际的内存地址是写不进入东西的（因为还没映射，只分配了虚拟地址）。因此，需要在寻找实际物理内存的过程中做个判断（是否虚拟地址已经映射了），可以写在copyin,copyout,但是出问题实际是在walkaddr函数，所以直接在walkaddr寻找物理地址时，如果发现虚拟地址没有映射，就给它实际分配一下即可。

uint64
walkaddr(pagetable_t pagetable, uint64 va)
{
  pte_t *pte;
  uint64 pa;

  if(va >= MAXVA)
    return 0;

  pte = walk(pagetable, va, 0);
  if(pte == 0)
    return 0;
  //加在这里，做一个映射（分配）操作即可，然后返回分配后的物理地址
  if((*pte & PTE_V) == 0){
    struct proc *p = myproc();
    if(va >= p->sz) return 0;
      char *mem = kalloc();
    if(mem == 0) return 0;
    if(mappages(pagetable, PGROUNDDOWN(va), PGSIZE, (uint64)mem, PTE_W|PTE_X|PTE_R|PTE_U) != 0){
         kfree(mem);
         return 0;
     }
     return (uint64)mem;
  }
  
  if((*pte & PTE_U) == 0)
    return 0;
  pa = PTE2PA(*pte);
  return pa;
}

最后，使用脚本测试一下，发现全部通过。

jimmy@ubuntu:~/xv6-test/xv6-labs-2020$ ./grade-lab-lazy 
make: 'kernel/kernel' is up to date.
== Test running lazytests == (3.6s) 
== Test   lazy: map == 
  lazy: map: OK 
== Test   lazy: unmap == 
  lazy: unmap: OK 
== Test usertests == (227.3s) 
== Test   usertests: pgbug == 
  usertests: pgbug: OK 
== Test   usertests: sbrkbugs == 
  usertests: sbrkbugs: OK 
== Test   usertests: argptest == 
  usertests: argptest: OK 
== Test   usertests: sbrkmuch == 
  usertests: sbrkmuch: OK 
== Test   usertests: sbrkfail == 
  usertests: sbrkfail: OK 
== Test   usertests: sbrkarg == 
  usertests: sbrkarg: OK 
== Test   usertests: stacktest == 
  usertests: stacktest: OK 
== Test   usertests: execout == 
  usertests: execout: OK 
== Test   usertests: copyin == 
  usertests: copyin: OK 
== Test   usertests: copyout == 
  usertests: copyout: OK 
== Test   usertests: copyinstr1 == 
  usertests: copyinstr1: OK 
== Test   usertests: copyinstr2 == 
  usertests: copyinstr2: OK 
== Test   usertests: copyinstr3 == 
  usertests: copyinstr3: OK 
== Test   usertests: rwsbrk == 
  usertests: rwsbrk: OK 
== Test   usertests: truncate1 == 
  usertests: truncate1: OK 
== Test   usertests: truncate2 == 
  usertests: truncate2: OK 
== Test   usertests: truncate3 == 
  usertests: truncate3: OK 
== Test   usertests: reparent2 == 
  usertests: reparent2: OK 
== Test   usertests: badarg == 
  usertests: badarg: OK 
== Test   usertests: reparent == 
  usertests: reparent: OK 
== Test   usertests: twochildren == 
  usertests: twochildren: OK 
== Test   usertests: forkfork == 
  usertests: forkfork: OK 
== Test   usertests: forkforkfork == 
  usertests: forkforkfork: OK 
== Test   usertests: createdelete == 
  usertests: createdelete: OK 
== Test   usertests: linkunlink == 
  usertests: linkunlink: OK 
== Test   usertests: linktest == 
  usertests: linktest: OK 
== Test   usertests: unlinkread == 
  usertests: unlinkread: OK 
== Test   usertests: concreate == 
  usertests: concreate: OK 
== Test   usertests: subdir == 
  usertests: subdir: OK 
== Test   usertests: fourfiles == 
  usertests: fourfiles: OK 
== Test   usertests: sharedfd == 
  usertests: sharedfd: OK 
== Test   usertests: exectest == 
  usertests: exectest: OK 
== Test   usertests: bigargtest == 
  usertests: bigargtest: OK 
== Test   usertests: bigwrite == 
  usertests: bigwrite: OK 
== Test   usertests: bsstest == 
  usertests: bsstest: OK 
== Test   usertests: sbrkbasic == 
  usertests: sbrkbasic: OK 
== Test   usertests: kernmem == 
  usertests: kernmem: OK 
== Test   usertests: validatetest == 
  usertests: validatetest: OK 
== Test   usertests: opentest == 
  usertests: opentest: OK 
== Test   usertests: writetest == 
  usertests: writetest: OK 
== Test   usertests: writebig == 
  usertests: writebig: OK 
== Test   usertests: createtest == 
  usertests: createtest: OK 
== Test   usertests: openiput == 
  usertests: openiput: OK 
== Test   usertests: exitiput == 
  usertests: exitiput: OK 
== Test   usertests: iput == 
  usertests: iput: OK 
== Test   usertests: mem == 
  usertests: mem: OK 
== Test   usertests: pipe1 == 
  usertests: pipe1: OK 
== Test   usertests: preempt == 
  usertests: preempt: OK 
== Test   usertests: exitwait == 
  usertests: exitwait: OK 
== Test   usertests: rmdot == 
  usertests: rmdot: OK 
== Test   usertests: fourteen == 
  usertests: fourteen: OK 
== Test   usertests: bigfile == 
  usertests: bigfile: OK 
== Test   usertests: dirfile == 
  usertests: dirfile: OK 
== Test   usertests: iref == 
  usertests: iref: OK 
== Test   usertests: forktest == 
  usertests: forktest: OK 
== Test time == 
time: OK 
Score: 119/119

总结

这个实验如果自己动手确实有难度，但是在看完课后，发现关键的点和思路，Kasshoek已经都讲到了。总结一下，这个实验主要就是实现lazy allocation，核心思想是当进程申请内存时，只在其虚拟地址上增加，而不会实际分配（因为分配了可能暂时也不会用到），所以这是一种很好的优化。

当操作系统检测到在实际使用这部分虚拟内存时，发现缺页中断，会利用trap进入内核，处理这个缺页中断，也就是如果有实际物理内存可以用，那就分配给它（建立映射）。

需要注意的就是采用系统调用函数（write ,read 等）时，往往不会进入usertrap，这部分就需要在访问物理内存时发现没有映射再建立，比如修改walkaddr函数，这样就较好地处理了一些意料外的状况。
[1]:https://pdos.csail.mit.edu/6.S081/2020/labs/lazy.html
[2]:https://zhuanlan.zhihu.com/p/403196090