Hama: GroomServer-CSDN博客

本文链接：https://blog.csdn.net/ylzhjlinux/article/details/84703463

GroomServer作为Apache Hama的一部分，主要负责管理bsptasks。它不仅检查本地磁盘是否可写，还清理指定目录下的文件，并初始化作业。此外，GroomServer能够启动HTTP服务器、任务报告服务器及工作服务器等组件。

摘要由CSDN通过智能技术生成

http://wiki.apache.org/hama/GroomServer

Introduction

GroomServer is a process whose main responsibility is to manage bsp tasks. In addition to task management, GroomServer collaborates with BSPMaster so that job execution can be done correctly. Works that GroomServer performs include:

Check local disk:
1. Check local disk if writable.
2. Delete files under local dir (bsp.local.dir)
Clear state table:
1. Clean up tasks (Map[TaskAttemptID -> TaskInProgress])
2. Initialize job (Map[BSPJobID -> RunningJob])
3. Cleanup running tasks (Map[TaskAttemptID -> TaskInProgress])
4. Configure max tasks, default to 3.
Start http server: An embedded http service.
Start task report server: Communication between GroomServer and spawned child task. See TaskRunner.BspChildRunner
Start worker server: RPC service listens to master's direction.
Register to BSPMaster: Enroll itself to BSPMaster with GroomServerStatus.
Start message dispatcher (Instructor): (TODO: refactor needed)
Start monitor service: A process export metrics, task status, etc. information.

Task Management

The GroomServer

receives instructions from BSPMasters.
spawns one or more tasks as separated jvm processes where tasks are then executed.
monitors spawned processes via ping; when a task is
- out of contact (failure/ crashed): launch a new process and restart the task with max attempt set to 3
- exceeding max attempt: update task status/ notify BSPMasters
sends heartbeat to BSPMasters

Scenario

Normal Case

A GroomServer forks a new task.
The spawned task ack with metrics to GroomServer.
Upon reception of task's ack, the GroomServer exports metrics to monitor system.

Failure Case

A task failure event happens.
A corresponded GroomServer detects the task failure.
The GroomServer tries to restart the task.
If failing restart, the GroomServer export by marking the task as failed.

Components

Registrator: Register to GroomManager
TaskManager: Perform tasks management.
- Launch a task.
- Stop a task.
- Kill a task.
- Resume a task.
Monitor: Report GroomServer and tasks related information.