MIT6.824 MapReduce 实践过程1

首先用Git拉取2020最新项目地址

$ git clone git://g.csail.mit.edu/6.824-golabs-2020 6.824
$ cd 6.824
$ ls
Makefile src
$

然后首先执行这段简单的串行化demo程序

$ cd ~/6.824
$ cd src/main
$ go build -buildmode=plugin ../mrapps/wc.go
$ rm mr-out*
$ go run mrsequential.go wc.so pg*.txt
$ more mr-out-0
A 509
ABOUT 2
ACT 8
...

注意：项目需要放在GOPATH路径下或者使用GOMOD，对于后者，需要修改包文件的目录为当前GOMOD路径。

 

这个demo首先将wc.go文件编译成wc.so文件，wc.go文件中提供了Map和Reduce函数

Map： contents string -> k-v 数组， 其中key=词，value=1

//
// The map function is called once for each file of input. The first
// argument is the name of the input file, and the second is the
// file's complete contents. You should ignore the input file name,
// and look only at the contents argument. The return value is a slice
// of key/value pairs.
//
func Ma