history service 没有启…


2015-11-21 23:07:04,523 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1448025206651_0026_000002
2015-11-21 23:07:05,041 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-11-21 23:07:05,094 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2015-11-21 23:07:05,094 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 26 cluster_timestamp: 1448025206651 } attemptId: 2 } keyId: -161280683)
2015-11-21 23:07:05,929 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Attempt num: 2 is last retry: true because a commit was started.
2015-11-21 23:07:05,932 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$NoopEventHandler
2015-11-21 23:07:05,939 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2015-11-21 23:07:05,941 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2015-11-21 23:07:06,016 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]
2015-11-21 23:07:06,042 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]
2015-11-21 23:07:06,067 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]
2015-11-21 23:07:06,097 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled
2015-11-21 23:07:06,099 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Will not try to recover. recoveryEnabled: true recoverySupportedByCommitter: false numReduceTasks: 1 shuffleKeyValidForRecovery: true ApplicationAttemptID: 2
2015-11-21 23:07:06,123 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]
2015-11-21 23:07:06,130 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Previous history file is at hdfs://cluster1:8020/tmp/hadoop-yarn/staging/root/.staging/job_1448025206651_0026/job_1448025206651_0026_1.jhist
2015-11-21 23:07:06,582 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2015-11-21 23:07:06,619 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-11-21 23:07:06,675 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-11-21 23:07:06,675 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2015-11-21 23:07:06,693 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2015-11-21 23:07:06,694 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2015-11-21 23:07:06,694 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2015-11-21 23:07:06,733 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at hadoop0/172.18.32.177:8030
2015-11-21 23:07:06,810 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: 
2015-11-21 23:07:06,811 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: default
2015-11-21 23:07:06,863 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://cluster1:8020]
2015-11-21 23:07:06,873 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryCopyService: History file is at hdfs://cluster1:8020/tmp/hadoop-yarn/staging/root/.staging/job_1448025206651_0026/job_1448025206651_0026_1.jhist
2015-11-21 23:07:06,979 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1448025206651_0026, File: hdfs://cluster1:8020/tmp/hadoop-yarn/staging/root/.staging/job_1448025206651_0026/job_1448025206651_0026_2.jhist 







问题是 history service 没有启动 解决办法: 配置:选择(xxxxxxxxxx: 作为history sever) yarn.log-aggregation-enable true mapreduce.jobhistory.address xxxxxxxxxx::10020
mapreduce.jobhistory.webapp.address xxxxxxxxxx:19888
sbin/mr-jobhistory-daemon.sh start historyserver 

我的mapred-site.xml改为下面的配置则可以在http://hadoop0:8088/cluster/apps/FINISHED看到执行成功的mapreduce任务: 
history <wbr>service <wbr>没有启动导致看不到运行成功的mapreduce程序



阅读更多
换一批

没有更多推荐了,返回首页