hadoop3.0.3-SYNTH运行成功

最新推荐文章于 2022-06-02 07:08:02 发布

hahachenchen789

最新推荐文章于 2022-06-02 07:08:02 发布

阅读量300

点赞数

分类专栏： hadoop Hadoop与Kubernetes学习

本文链接：https://blog.csdn.net/hahachenchen789/article/details/81038473

版权

Hadoop与Kubernetes学习同时被 2 个专栏收录

65 篇文章 6 订阅

订阅专栏

hadoop

41 篇文章 2 订阅

订阅专栏

在上篇博文：Hadoop3.0.3-SYNTH运行，SYNTH由于NULL user导致错误。

当时以为是hadoop的源码没有写好，但仔细研究后发现，可以在SYNTH的json文件中增加user。

也就是说官网给出的synth的json文件是不完整的。

下面给出可以完美运行的json文件：

{
  "description": "tiny jobs workload",
  "num_nodes": 20,
  "nodes_per_rack": 4,
  "num_jobs": 10,
  "rand_seed": 2,
  "workloads": [
    {
      "workload_name": "tiny-test",
      "workload_weight": 0.5,
      "description": "Sort jobs",
      "queue_name": "sls_queue_1",
      "job_classes": [
        {
          "class_name": "class_1",
          "user_name": "foobar",
          "class_weight": 1.0,
          "mtasks_avg": 5,
          "mtasks_stddev": 1,
          "rtasks_avg": 5,
          "rtasks_stddev": 1,
          "dur_avg": 60,
          "dur_stddev": 5,
          "mtime_avg": 10,
          "mtime_stddev": 2,
          "rtime_avg": 20,
          "rtime_stddev": 4,
          "map_max_memory_avg": 1024,
          "map_max_memory_stddev": 0.001,
          "map_execution_type": "GUARANTEED",
          "reduce_max_memory_avg": 2048,
          "reduce_max_memory_stddev": 0.001,
          "reduce_execution_type": "GUARANTEED",
          "map_max_vcores_avg": 1,
          "map_max_vcores_stddev": 0.001,
          "reduce_max_vcores_avg": 2,
          "reduce_max_vcores_stddev": 0.001,
          "chance_of_reservation": 0.5,
          "deadline_factor_avg": 10.0,
          "deadline_factor_stddev": 0.001
        }
      ],
      "time_distribution": [
        {
          "time": 1,
          "weight": 100
        },
        {
          "time": 60,
          "weight": 0
        }
      ]
    }
  ]
}