现象
Hadoop启动后 执行jps 查看进程,缺少了 ResourceManager
解决思路
start-all.sh分别会有五个日志产生
缺少哪个进程,就去看谁的日志
报错内容
resourcemanager的log文件
2023-05-20 14:12:00,780 INFO org.apache.hadoop.yarn.security.YarnAuthorizationProvider: org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer is instiantiated.
2023-05-20 14:12:00,781 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list
2023-05-20 14:12:00,782 INFO org.apache.hadoop.conf.Configuration: found resource capacity-scheduler.xml at file:/software/hadoop/hadoop-2.7.7/etc/hadoop/capacity-scheduler.xml
2023-05-20 14:12:00,805 FATAL org.apache.hadoop.conf.Configuration: error parsing conf java.io.BufferedInputStream@6a8658ff
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2552)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2628)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2574)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2477)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1047)
at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1101)
at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1321)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.validateConf(CapacityScheduler.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:294)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:334)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:556)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:964)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:252)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1187)
2023-05-20 14:12:00,807 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler failed in state INITED; cause: java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2728)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2574)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2477)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1047)
at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1101)
at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1321)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.validateConf(CapacityScheduler.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:294)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:334)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:556)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:964)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:252)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1187)
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2552)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2628)
... 16 more
2023-05-20 14:12:00,808 INFO org.apache.hadoop.service.AbstractService: Service RMActiveServices failed in state INITED; cause: java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2728)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2574)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2477)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1047)
at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1101)
at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1321)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.validateConf(CapacityScheduler.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:294)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:334)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:556)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:964)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:252)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1187)
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2552)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2628)
... 16 more
2023-05-20 14:12:00,808 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager metrics system...
2023-05-20 14:12:00,809 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system stopped.
2023-05-20 14:12:00,809 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete.
2023-05-20 14:12:00,809 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2023-05-20 14:12:00,809 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager failed in state INITED; cause: java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2728)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2574)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2477)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1047)
at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1101)
at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1321)
at
解决
关键点就是这里
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.这个报错基本上就是文件编码出了问题。
排查
1:检查文件的编码 是不是 utf-8-bom这种
2:文件是否能正常打开
3:文件是否删除了格式定义
………………………………………………………………………………
我的问题解决是因为capacity-scheduler.xml 乱码了。
具体造成问题的原因不明
………………………………………………………………………………
总结
解决这个问题 也花了不少时间去排查、搜索
但是日志已经说明了问题所在
自己的疏忽大意,造成了花了不少时间
认真看日志、认真看日志、认真看日志
认真看日志、认真看日志、认真看日志
认真看日志、认真看日志、认真看日志