概述:
特性:
Non-work-preserving RM restart:
Work-preserving RM restart:
Enable RM Restart
Property | Description |
---|---|
yarn.resourcemanager.recovery.enabled | true |
Configure the state-store for persisting the RM state
Property | Description |
---|---|
yarn.resourcemanager.store.class | The class name of the state-store to be used for saving application/attempt state and the credentials. The available state-store implementations areorg.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore, a ZooKeeper based state-store implementation andorg.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore, a Hadoop FileSystem based state-store implementation like HDFS and local FS. org.apache.hadoop.yarn.server.resourcemanager.recovery.LeveldbRMStateStore, a LevelDB based state-store implementation. The default value is set to org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore. |
How to choose the state-store implementation
ZooKeeper based state-store: User is free to pick up any storage to set up RM restart, but must use ZooKeeper based state-store to support RM HA. The reason is that only ZooKeeper based state-store supports fencing mechanism to avoid a split-brain situation where multiple RMs assume they are active and can edit the state-store at the same time.
FileSystem based state-store: HDFS and local FS based state-store are supported. Fencing mechanism is not supported.
LevelDB based state-store: LevelDB based state-store is considered more light weight than HDFS and ZooKeeper based state-store. LevelDB supports better atomic operations, fewer I/O ops per state update, and far fewer total files on the filesystem. Fencing mechanism is not supported.