MapFile检索序列文件

1、MapFile写入文件

  /**
     * mapfile写入文件
     * @throws IOException
     */
    @Test
    public void save() throws IOException {
        Configuration conf = new Configuration();
        conf.set("fs.defaultFS", "file:///");
        FileSystem fs = FileSystem.get(conf);
        Path path = new Path("D:\\sequence\\1.seq");
        MapFile.Writer writer  = new MapFile.Writer(conf,fs,"d:/map",IntWritable.class,Text.class);
        for(int i = 0; i < 10; i++){
            writer.append(new IntWritable(i),new Text("hello" + i));
        }
        writer.close();
    }

MapFile写入的时候产生两个文件，一个index序列索引文件，一个data数据序列文件

index文件定义key的区间范围便于快速查找和定位

2、MapFile读取文件

    /**
     * MapFile读取文件
     * @throws IOException
     */
    @Test
    public void read() throws IOException {
        Configuration conf = new Configuration();
        conf.set("fs.defaultFS", "file:///");
        FileSystem fs = FileSystem.get(conf);
        MapFile.Reader reader  = new MapFile.Reader(fs,"d:/map",conf);
        IntWritable key = new IntWritable();
        Text value = new Text();
        while (reader.next(key,value)){
            System.out.println(key.get() + "===>" + value.toString());
        }
    }