关于gridmix3(Gridmix3 – Emulating Production Workload for Apache Hadoop)

前言：

Tracking, modeling, and mimicking this adaptive, complex workload is a prerequisite to effective performance engineering.

gridmix的意义：模拟和跟踪复杂的、适应性的复杂对性能测评先决意义

gridmix1-2的特性：

Gridmix consists the following parts:

1、A data generation script that must be run once to generate data needed for running the actual benchmark.

一个用于产生运行benchmark所需数据的脚本

2、Several types of jobs in the "mix". The particular jobs varied slightly between Gridmix and Gridmix2, but examples include sorts of text data and SequenceFiles, jobs sampling from large, compressed datasets, chains of MapReduce jobs, and jobs exercising the combiner

混合的作业类型。特定的作业在gridmix和gridmix2间有细微的差别。但是都包括文本数据和sequencefile排序、大数据量取样、压缩数据集、Mapreduce chain和运用了combiner的作业例子。

3、Configuration files- either XML files or shell scripts- that permitted the operator to tune the number of jobs in each type and of each size

用xml或者shell脚本的配置文件形式，提供调整各类作业数量和规模的借口

4、A driver that parses the configuration file, submits MapReduce jobs to the cluster, and (in Gridmix2) collects statistics for each job.

一个driver:解析配置文件，提交Mapreduce作业，搜集统计数据

While running Gridmix as an end-to-end benchmark yielded many insights into framework bottlenecks in medium-sized clusters under a saturating load, it does not model the diverse mix of jobs running in our environment. On the contrary, framework improvements motivated by studies based on Gridmix often had ambiguous effects in production after showing dramatic gains in a test environment.

用gridmix作为测评工具时，其实隐藏了在集群搞复杂情况下的一些性能瓶颈。他没有模拟出混合的