hadoop-yarn-site.xml配置文件详解

hadoop-yarn-site.xml配置文件详解

yarn-site.xml配置文件简介

yarn配置相关

yarn-site.xml配置
namevaluedescription
yarn.ipc.client.factory.classFactory to create client IPC classes.
yarn.ipc.serializer.typeprotocolbuffersType of serialization to use.
yarn.ipc.server.factory.classFactory to create server IPC classes.
yarn.ipc.exception.factory.classFactory to create IPC exceptions.
yarn.ipc.record.factory.classFactory to create serializeable records.
yarn.ipc.rpc.classorg.apache.hadoop.yarn.ipc.HadoopYarnProtoRPCRPC class implementation
yarn.resourcemanager.hostname0.0.0.0The hostname of the RM.
yarn.resourcemanager.address${yarn.resourcemanager.hostname}:8032The address of the applications manager interface in the RM.
yarn.resourcemanager.client.thread-count50The number of threads used to handle applications manager requests.
yarn.am.liveness-monitor.expiry-interval-ms600000The expiry interval for application master reporting.
yarn.resourcemanager.principalThe Kerberos principal for the resource manager.
yarn.resourcemanager.scheduler.address${yarn.resourcemanager.hostname}:8030The address of the scheduler interface.
yarn.resourcemanager.scheduler.client.thread-count50Number of threads to handle scheduler interface.
yarn.http.policyHTTP_ONLYThis configures the HTTP endpoint for Yarn Daemons.The following values are supported: - HTTP_ONLY : Service is provided only on http - HTTPS_ONLY : Service is provided only on https
yarn.resourcemanager.webapp.address${yarn.resourcemanager.hostname}:8088The http address of the RM web application.
yarn.resourcemanager.webapp.https.address${yarn.resourcemanager.hostname}:8090The https adddress of the RM web application.
yarn.resourcemanager.resource-tracker.address${yarn.resourcemanager.hostname}:8031
yarn.acl.enablefalseAre acls enabled.
yarn.admin.acl*ACL of who can be admin of the YARN cluster.
yarn.resourcemanager.admin.address${yarn.resourcemanager.hostname}:8033The address of the RM admin interface.
yarn.resourcemanager.admin.client.thread-count1Number of threads used to handle RM admin interface.
yarn.resourcemanager.amliveliness-monitor.interval-ms1000How often should the RM check that the AM is still alive.
yarn.resourcemanager.connect.max-wait.ms900000Maximum time to wait to establish connection to ResourceManager.
yarn.resourcemanager.connect.retry-interval.ms30000How often to try connecting to the ResourceManager.
yarn.resourcemanager.am.max-attempts2The maximum number of application attempts. It’s a global setting for all application masters. Each application master can specify its individual maximum number of application attempts via the API, but the individual number cannot be more than the global upper bound. If it is, the resourcemanager will override it. The default number is set to 2, to allow at least one retry for AM.
yarn.resourcemanager.container.liveness-monitor.interval-ms600000How often to check that containers are still alive.
yarn.resourcemanager.keytab/etc/krb5.keytabThe keytab for the resource manager.
yarn.nm.liveness-monitor.expiry-interval-ms600000How long to wait until a node manager is considered dead.
yarn.resourcemanager.nm.liveness-monitor.interval-ms1000How often to check that node managers are still alive.
yarn.resourcemanager.nodes.include-pathPath to file with nodes to include.
yarn.resourcemanager.nodes.exclude-pathPath to file with nodes to exclude.
yarn.resourcemanager.resource-tracker.client.thread-count50Number of threads to handle resource tracker calls.
yarn.resourcemanager.scheduler.classorg.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerThe class to use as the resource scheduler.
yarn.scheduler.minimum-allocation-mb1024The minimum allocation for every container request at the RM, in MBs. Memory requests lower than this won’t take effect, and the specified value will get allocated at minimum.
yarn.scheduler.maximum-allocation-mb8192The maximum allocation for every container request at the RM, in MBs. Memory requests higher than this won’t take effect, and will get capped to this value.
yarn.scheduler.minimum-allocation-vcores1The minimum allocation for every container request at the RM, in terms of virtual CPU cores. Requests lower than this won’t take effect, and the specified value will get allocated the minimum.
yarn.scheduler.maximum-allocation-vcores32The maximum allocation for every container request at the RM, in terms of virtual CPU cores. Requests higher than this won’t take effect, and will get capped to this value.
yarn.resourcemanager.recovery.enabledfalseEnable RM to recover state after starting. If true, then yarn.resourcemanager.store.class must be specified.
yarn.resourcemanager.store.classorg.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStoreThe class to use as the persistent store. If org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore is used, the store is implicitly fenced; meaning a single ResourceManager is able to use the store at any point in time. More details on this implicit fencing, along with setting up appropriate ACLs is discussed under yarn.resourcemanager.zk-state-store.root-node.acl.
yarn.resourcemanager.state-store.max-completed-applications${yarn.resourcemanager.max-completed-applications}The maximum number of completed applications RM state store keeps, less than or equals to ${yarn.resourcemanager.max-completed-applications}. By default, it equals to ${yarn.resourcemanager.max-completed-applications}. This ensures that the applications kept in the state store are consistent with the applications remembered in RM memory. Any values larger than ${yarn.resourcemanager.max-completed-applications} will be reset to ${yarn.resourcemanager.max-completed-applications}. Note that this value impacts the RM recovery performance.Typically, a smaller value indicates better performance on RM recovery.
yarn.resourcemanager.zk-addressHost:Port of the ZooKeeper server to be used by the RM. This must be supplied when using the ZooKeeper based implementation of the RM state store and/or embedded automatic failover in a HA setting.
yarn.resourcemanager.zk-num-retries500Number of times RM tries to connect to ZooKeeper.
yarn.resourcemanager.zk-retry-interval-ms2000Retry interval in milliseconds when connecting to ZooKeeper.
yarn.resourcemanager.zk-state-store.parent-path/rmstoreFull path of the ZooKeeper znode where RM state will be stored. This must be supplied when using org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore as the value for yarn.resourcemanager.store.class
yarn.resourcemanager.zk-timeout-ms10000ZooKeeper session timeout in milliseconds. Session expiration is managed by the ZooKeeper cluster itself, not by the client. This value is used by the cluster to determine when the client’s session expires. Expirations happens when the cluster does not hear from the client within the specified session timeout period (i.e. no heartbeat).
yarn.resourcemanager.zk-aclworld:anyone:rwcdaACL’s to be used for ZooKeeper znodes.
yarn.resourcemanager.zk-state-store.root-node.aclACLs to be used for the root znode when using ZKRMStateStore in a HA scenario for fencing. ZKRMStateStore supports implicit fencing to allow a single ResourceManager write-access to the store. For fencing, the ResourceManagers in the cluster share read-write-admin privileges on the root node, but the Active ResourceManager claims exclusive create-delete permissions. By default, when this property is not set, we use the ACLs from yarn.resourcemanager.zk-acl for shared admin access and rm-address:random-number for username-based exclusive create-delete access. This property allows users to set ACLs of their choice instead of using the default mechanism. For fencing to work, the ACLs should be carefully set differently on each ResourceManger such that all the ResourceManagers have shared admin access and the Active ResourceManger takes over (exclusively) the create-delete access.
yarn.resourcemanager.fs.state-store.uri${hadoop.tmp.dir}/yarn/system/rmstoreURI pointing to the location of the FileSystem path where RM state will be stored. This must be supplied when using org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore as the value for yarn.resourcemanager.store.class
yarn.resourcemanager.fs.state-store.retry-policy-spec2000, 500hdfs client retry policy specification. hdfs client retry is always enabled. Specified in pairs of sleep-time and number-of-retries and (t0, n0), (t1, n1), …, the first n0 retries sleep t0 milliseconds on average, the following n1 retries sleep t1 milliseconds on average, and so on.
yarn.resourcemanager.ha.enabledfalseEnable RM high-availability. When enabled, (1) The RM starts in the Standby mode by default, and transitions to the Active mode when prompted to. (2) The nodes in the RM ensemble are listed in yarn.resourcemanager.ha.rm-ids (3) The id of each RM either comes from yarn.resourcemanager.ha.id if yarn.resourcemanager.ha.id is explicitly specified or can be figured out by matching yarn.resourcemanager.address.{id} with local address (4) The actual physical addresses come from the configs of the pattern - {rpc-config}.{id}
yarn.resourcemanager.ha.automatic-failover.enabledtrueEnable automatic failover. By default, it is enabled only when HA is enabled
yarn.resourcemanager.ha.automatic-failover.embeddedtrueEnable embedded automatic failover. By default, it is enabled only when HA is enabled. The embedded elector relies on the RM state store to handle fencing, and is primarily intended to be used in conjunction with ZKRMStateStore.
yarn.resourcemanager.ha.automatic-failover.zk-base-path/yarn-leader-electionThe base znode path to use for storing leader information, when using ZooKeeper based leader election.
yarn.resourcemanager.cluster-idName of the cluster. In a HA setting, this is used to ensure the RM participates in leader election fo this cluster and ensures it does not affect other clusters
yarn.resourcemanager.ha.rm-idsThe list of RM nodes in the cluster when HA is enabled. See description of yarn.resourcemanager.ha .enabled for full details on how this is used.
yarn.resourcemanager.ha.idThe id (string) of the current RM. When HA is enabled, this is an optional config. The id of current RM can be set by explicitly specifying yarn.resourcemanager.ha.id or figured out by matching yarn.resourcemanager.address.{id} with local address See description of yarn.resourcemanager.ha.enabled for full details on how this is used.
yarn.client.failover-proxy-providerorg.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProviderWhen HA is enabled, the class to be used by Clients, AMs and NMs to failover to the Active RM. It should extend org.apache.hadoop.yarn.client.RMFailoverProxyProvider
yarn.client.failover-max-attemptsWhen HA is enabled, the max number of times FailoverProxyProvider should attempt failover. When set, this overrides the yarn.resourcemanager.connect.max-wait.ms. When not set, this is inferred from yarn.resourcemanager.connect.max-wait.ms.
yarn.client.failover-sleep-base-msWhen HA is enabled, the sleep base (in milliseconds) to be used for calculating the exponential delay between failovers. When set, this overrides the yarn.resourcemanager.connect.* settings. When not set, yarn.resourcemanager.connect.retry-interval.ms is used instead.
yarn.client.failover-sleep-max-msWhen HA is enabled, the maximum sleep time (in milliseconds) between failovers. When set, this overrides the yarn.resourcemanager.connect.* settings. When not set, yarn.resourcemanager.connect.retry-interval.ms is used instead.
yarn.client.failover-retries0When HA is enabled, the number of retries per attempt to connect to a ResourceManager. In other words, it is the ipc.client.connect.max.retries to be used during failover attempts
yarn.client.failover-retries-on-socket-timeouts0When HA is enabled, the number of retries per attempt to connect to a ResourceManager on socket timeouts. In other words, it is the ipc.client.connect.max.retries.on.timeouts to be used during failover attempts
yarn.resourcemanager.max-completed-applications10000The maximum number of completed applications RM keeps.
yarn.resourcemanager.delayed.delegation-token.removal-interval-ms30000Interval at which the delayed token removal thread runs
yarn.resourcemanager.application-tokens.master-key-rolling-interval-secs86400Interval for the roll over for the master key used to generate application tokens
yarn.resourcemanager.container-tokens.master-key-rolling-interval-secs86400Interval for the roll over for the master key used to generate container tokens. It is expected to be much greater than yarn.nm.liveness-monitor.expiry-interval-ms and yarn.rm.container-allocation.expiry-interval-ms. Otherwise the behavior is undefined.
yarn.resourcemanager.nodemanagers.heartbeat-interval-ms1000The heart-beat interval in milliseconds for every NodeManager in the cluster.
yarn.resourcemanager.nodemanager.minimum.versionNONEThe minimum allowed version of a connecting nodemanager. The valid values are NONE (no version checking), EqualToRM (the nodemanager’s version is equal to or greater than the RM version), or a Version String.
yarn.resourcemanager.scheduler.monitor.enablefalseEnable a set of periodic monitors (specified in yarn.resourcemanager.scheduler.monitor.policies) that affect the scheduler.
yarn.resourcemanager.scheduler.monitor.policiesorg.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicyThe list of SchedulingEditPolicy classes that interact with the scheduler. A particular module may be incompatible with the scheduler, other policies, or a configuration of either.
yarn.resourcemanager.history-writer.multi-threaded-dispatcher.pool-size10Number of worker threads that write the history data.
yarn.resourcemanager.configuration.provider-classorg.apache.hadoop.yarn.LocalConfigurationProviderThe class to use as the configuration provider. If org.apache.hadoop.yarn.LocalConfigurationProvider is used, the local configuration will be loaded. If org.apache.hadoop.yarn.FileSystemBasedConfigurationProvider is used, the configuration which will be loaded should be uploaded to remote File system first.
yarn.nodemanager.hostname0.0.0.0The hostname of the NM.
yarn.nodemanager.address${yarn.nodemanager.hostname}:0The address of the container manager in the NM.
yarn.nodemanager.admin-envMALLOC_ARENA_MAX=$MALLOC_ARENA_MAXEnvironment variables that should be forwarded from the NodeManager’s environment to the container’s.
yarn.nodemanager.env-whitelistJAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,HADOOP_YARN_HOMEEnvironment variables that containers may override rather than use NodeManager’s default.
yarn.nodemanager.container-executor.classorg.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutorwho will execute(launch) the containers.
yarn.nodemanager.container-manager.thread-count20Number of threads container manager uses.
yarn.nodemanager.delete.thread-count4Number of threads used in cleanup.
yarn.nodemanager.delete.debug-delay-sec0Number of seconds after an application finishes before the nodemanager’s DeletionService will delete the application’s localized file directory and log directory. To diagnose Yarn application problems, set this property’s value large enough (for example, to 600 = 10 minutes) to permit examination of these directories. After changing the property’s value, you must restart the nodemanager in order for it to have an effect. The roots of Yarn applications’ work directories is configurable with the yarn.nodemanager.local-dirs property (see below), and the roots of the Yarn applications’ log directories is configurable with the yarn.nodemanager.log-dirs property (see also below).
yarn.nodemanager.keytab/etc/krb5.keytabKeytab for NM.
yarn.nodemanager.local-dirs${hadoop.tmp.dir}/nm-local-dirList of directories to store localized files in. An application’s localized file directory will be found in: y a r n . n o d e m a n a g e r . l o c a l − d i r s / u s e r c a c h e / {yarn.nodemanager.local-dirs}/usercache/ yarn.nodemanager.localdirs/usercache/{user}/appcache/application_KaTeX parse error: Expected group after '_' at position 68: …alled container_̲{contid}, will be subdirectories of this.
yarn.nodemanager.local-cache.max-files-per-directory8192It limits the maximum number of files which will be localized in a single local directory. If the limit is reached then sub-directories will be created and new files will be localized in them. If it is set to a value less than or equal to 36 [which are sub-directories (0-9 and then a-z)] then NodeManager will fail to start. For example; [for public cache] if this is configured with a value of 40 ( 4 files + 36 sub-directories) and the local-dir is “/tmp/local-dir1” then it will allow 4 files to be created directly inside “/tmp/local-dir1/filecache”. For files that are localized further it will create a sub-directory “0” inside “/tmp/local-dir1/filecache” and will localize files inside it until it becomes full. If a file is removed from a sub-directory that is marked full, then that sub-directory will be used back again to localize files.
yarn.nodemanager.localizer.address${yarn.nodemanager.hostname}:8040Address where the localizer IPC is.
yarn.nodemanager.localizer.cache.cleanup.interval-ms600000Interval in between cache cleanups.
yarn.nodemanager.localizer.cache.target-size-mb10240Target size of localizer cache in MB, per local directory.
yarn.nodemanager.localizer.client.thread-count5Number of threads to handle localization requests.
yarn.nodemanager.localizer.fetch.thread-count4Number of threads to use for localization fetching.
yarn.nodemanager.log-dirs${yarn.log.dir}/userlogsWhere to store container logs. An application’s localized log directory will be found in KaTeX parse error: Expected group after '_' at position 40: …rs}/application_̲{appid}. Individual containers’ log directories will be below this, in directories named container_{$contid}. Each container directory will contain the files stderr, stdin, and syslog generated by that container.
yarn.log-aggregation-enablefalseWhether to enable log aggregation
yarn.log-aggregation.retain-seconds-1How long to keep aggregation logs before deleting them. -1 disables. Be careful set this too small and you will spam the name node.
yarn.log-aggregation.retain-check-interval-seconds-1How long to wait between aggregated log retention checks. If set to 0 or a negative value then the value is computed as one-tenth of the aggregated log retention time. Be careful set this too small and you will spam the name node.
yarn.nodemanager.log.retain-seconds10800Time in seconds to retain user logs. Only applicable if log aggregation is disabled
yarn.nodemanager.remote-app-log-dir/tmp/logsWhere to aggregate logs to.
yarn.nodemanager.remote-app-log-dir-suffixlogsThe remote log dir will be created at {yarn.nodemanager.remote-app-log-dir}/${user}/{thisParam}
yarn.nodemanager.resource.memory-mb8192Amount of physical memory, in MB, that can be allocated for containers.
yarn.nodemanager.pmem-check-enabledtrueWhether physical memory limits will be enforced for containers.
yarn.nodemanager.vmem-check-enabledtrueWhether virtual memory limits will be enforced for containers.
yarn.nodemanager.vmem-pmem-ratio2.1Ratio between virtual memory to physical memory when setting memory limits for containers. Container allocations are expressed in terms of physical memory, and virtual memory usage is allowed to exceed this allocation by this ratio.
yarn.nodemanager.resource.cpu-vcores8Number of CPU cores that can be allocated for containers.
yarn.nodemanager.webapp.address${yarn.nodemanager.hostname}:8042NM Webapp address.
yarn.nodemanager.container-monitor.interval-ms3000How often to monitor containers.
yarn.nodemanager.container-monitor.resource-calculator.classClass that calculates containers current resource utilization.
yarn.nodemanager.health-checker.interval-ms600000Frequency of running node health script.
yarn.nodemanager.health-checker.script.timeout-ms1200000Script time out period.
yarn.nodemanager.health-checker.script.pathThe health check script to run.
yarn.nodemanager.health-checker.script.optsThe arguments to pass to the health check script.
yarn.nodemanager.disk-health-checker.interval-ms120000Frequency of running disk health checker code.
yarn.nodemanager.disk-health-checker.min-healthy-disks0.25The minimum fraction of number of disks to be healthy for the nodemanager to launch new containers. This correspond to both yarn-nodemanager.local-dirs and yarn.nodemanager.log-dirs. i.e. If there are less number of healthy local-dirs (or log-dirs) available, then new containers will not be launched on this node.
yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage100.0The maximum percentage of disk space utilization allowed after which a disk is marked as bad. Values can range from 0.0 to 100.0. If the value is greater than or equal to 100, the nodemanager will check for full disk. This applies to yarn-nodemanager.local-dirs and yarn.nodemanager.log-dirs.
yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb0The minimum space that must be available on a disk for it to be used. This applies to yarn-nodemanager.local-dirs and yarn.nodemanager.log-dirs.
yarn.nodemanager.linux-container-executor.pathThe path to the Linux container executor.
yarn.nodemanager.linux-container-executor.resources-handler.classorg.apache.hadoop.yarn.server.nodemanager.util.DefaultLCEResourcesHandlerThe class which should help the LCE handle resources.
yarn.nodemanager.linux-container-executor.cgroups.hierarchy/hadoop-yarnThe cgroups hierarchy under which to place YARN proccesses (cannot contain commas). If yarn.nodemanager.linux-container-executor.cgroups.mount is false (that is, if cgroups have been pre-configured), then this cgroups hierarchy must already exist and be writable by the NodeManager user, otherwise the NodeManager may fail. Only used when the LCE resources handler is set to the CgroupsLCEResourcesHandler.
yarn.nodemanager.linux-container-executor.cgroups.mountfalseWhether the LCE should attempt to mount cgroups if not found. Only used when the LCE resources handler is set to the CgroupsLCEResourcesHandler.
yarn.nodemanager.linux-container-executor.cgroups.mount-pathWhere the LCE should attempt to mount cgroups if not found. Common locations include /sys/fs/cgroup and /cgroup; the default location can vary depending on the Linux distribution in use. This path must exist before the NodeManager is launched. Only used when the LCE resources handler is set to the CgroupsLCEResourcesHandler, and yarn.nodemanager.linux-container-executor.cgroups.mount is true.
yarn.nodemanager.linux-container-executor.nonsecure-mode.local-usernobodyThe UNIX user that containers will run as when Linux-container-executor is used in nonsecure mode (a use case for this is using cgroups).
yarn.nodemanager.linux-container-executor.nonsecure-mode.user-pattern1[-@_.A-Za-z0-9]{0,255}?[ ] ? ]? ]?The allowed pattern for UNIX user names enforced by Linux-container-executor when used in nonsecure mode (use case for this is using cgroups). The default value is taken from /usr/sbin/adduser
yarn.nodemanager.log-aggregation.compression-typenoneT-file compression types used to compress aggregated logs.
yarn.nodemanager.principalThe kerberos principal for the node manager.
yarn.nodemanager.aux-servicesthe valid service name should only contain a-zA-Z0-9_ and can not start with numbers
yarn.nodemanager.sleep-delay-before-sigkill.ms250No. of ms to wait between sending a SIGTERM and SIGKILL to a container
yarn.nodemanager.process-kill-wait.ms2000Max time to wait for a process to come up when trying to cleanup a container
yarn.nodemanager.resourcemanager.connect.wait.secs900Max time, in seconds, to wait to establish a connection to RM when NM starts. The NM will shutdown if it cannot connect to RM within the specified max time period. If the value is set as -1, then NM will retry forever.
yarn.nodemanager.resourcemanager.connect.retry_interval.secs30Time interval, in seconds, between each NM attempt to connect to RM.
yarn.nodemanager.resourcemanager.minimum.versionNONEThe minimum allowed version of a resourcemanager that a nodemanager will connect to. The valid values are NONE (no version checking), EqualToNM (the resourcemanager’s version is equal to or greater than the NM version), or a Version String.
yarn.client.nodemanager-client-async.thread-pool-max-size500Max number of threads in NMClientAsync to process container management events
yarn.client.max-nodemanagers-proxies500Maximum number of proxy connections for node manager. It should always be more than 1. NMClient and MRAppMaster will use this to cache connection with node manager. There will be at max one connection per node manager. Ex. configuring it to a value of 5 will make sure that client will at max have 5 connections cached with 5 different node managers. These connections will be timed out if idle for more than system wide idle timeout period. The token if used for authentication then it will be used only at connection creation time. If new token is received then earlier connection should be closed in order to use newer token. This and (yarn.client.nodemanager-client-async.thread-pool-max-size) are related and should be sync (no need for them to be equal).
yarn.nodemanager.aux-services.mapreduce_shuffle.classorg.apache.hadoop.mapred.ShuffleHandler
mapreduce.job.jar
mapreduce.job.hdfs-servers${fs.defaultFS}
yarn.web-proxy.principalThe kerberos principal for the proxy, if the proxy is not running as part of the RM.
yarn.web-proxy.keytabKeytab for WebAppProxy, if the proxy is not running as part of the RM.
yarn.web-proxy.addressThe address for the web proxy as HOST:PORT, if this is not given then the proxy will run as part of the RM
yarn.application.classpathCLASSPATH for YARN applications. A comma-separated list of CLASSPATH entries. When this value is empty, the following default CLASSPATH for YARN applications would be used. For Linux: $HADOOP_CONF_DIR, $HADOOP_COMMON_HOME/share/hadoop/common/, $HADOOP_COMMON_HOME/share/hadoop/common/lib/, $HADOOP_HDFS_HOME/share/hadoop/hdfs/, $HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/, $HADOOP_YARN_HOME/share/hadoop/yarn/, $HADOOP_YARN_HOME/share/hadoop/yarn/lib/ For Windows: %HADOOP_CONF_DIR%, %HADOOP_COMMON_HOME%/share/hadoop/common/, %HADOOP_COMMON_HOME%/share/hadoop/common/lib/, %HADOOP_HDFS_HOME%/share/hadoop/hdfs/, %HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/, %HADOOP_YARN_HOME%/share/hadoop/yarn/, %HADOOP_YARN_HOME%/share/hadoop/yarn/lib/
yarn.timeline-service.enabledfalseIndicate to clients whether timeline service is enabled or not. If enabled, clients will put entities and events to the timeline server.
yarn.timeline-service.hostname0.0.0.0The hostname of the timeline service web application.
yarn.timeline-service.address${yarn.timeline-service.hostname}:10200This is default address for the timeline server to start the RPC server.
yarn.timeline-service.webapp.address${yarn.timeline-service.hostname}:8188The http address of the timeline service web application.
yarn.timeline-service.webapp.https.address${yarn.timeline-service.hostname}:8190The https address of the timeline service web application.
yarn.timeline-service.store-classorg.apache.hadoop.yarn.server.applicationhistoryservice.timeline.LeveldbTimelineStoreStore class name for timeline store.
yarn.timeline-service.ttl-enabletrueEnable age off of timeline store data.
yarn.timeline-service.ttl-ms604800000Time to live for timeline store data in milliseconds.
yarn.timeline-service.leveldb-timeline-store.path${hadoop.tmp.dir}/yarn/timelineStore file name for leveldb timeline store.
yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms300000Length of time to wait between deletion cycles of leveldb timeline store in milliseconds.
yarn.timeline-service.leveldb-timeline-store.read-cache-size104857600Size of read cache for uncompressed blocks for leveldb timeline store in bytes.
yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size10000Size of cache for recently read entity start times for leveldb timeline store in number of entities.
yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size10000Size of cache for recently written entity start times for leveldb timeline store in number of entities.
yarn.timeline-service.handler-thread-count10Handler thread count to serve the client RPC requests.
yarn.timeline-service.generic-application-history.enabledfalseIndicate to ResourceManager as well as clients whether history-service is enabled or not. If enabled, ResourceManager starts recording historical data that ApplicationHistory service can consume. Similarly, clients can redirect to the history service when applications finish if this is enabled.
yarn.timeline-service.generic-application-history.fs-history-store.uri${hadoop.tmp.dir}/yarn/timeline/generic-historyURI pointing to the location of the FileSystem path where the history will be persisted. This must be supplied when using org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore as the value for yarn.timeline-service.generic-application-history.store-class
yarn.timeline-service.generic-application-history.fs-history-store.compression-typenoneT-file compression types used to compress history data.
yarn.timeline-service.generic-application-history.store-classorg.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStoreStore class name for history store, defaulting to file system store
yarn.client.application-client-protocol.poll-interval-ms200The interval that the yarn client library uses to poll the completion status of the asynchronous API of application client protocol.
yarn.nodemanager.container-monitor.procfs-tree.smaps-based-rss.enabledfalse

  1. _.A-Za-z0-9 ↩︎

评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值