Flink on yarn Couldn‘t deploy Yarn cluster或者The YARN application unexpectedly switched to state FAIL

Error while deploying YARN cluster: Couldn't deploy Yarn cluster
java.lang.RuntimeException: Couldn't deploy Yarn cluster
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:443)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:630)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:486)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:483)
	at org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
	at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:483)
Caused by: org.apache.flink.yarn.AbstractYarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment. 
Diagnostics from YARN: Application application_1553768779226_0005 failed 1 times due to AM Container for appattempt_1553768779226_0005_000001 exited with  exitCode: -103
For more detailed output, check application tracking page:http://uplooking01:8088/proxy/application_1553768779226_0005/Then, click on links to logs of each attempt.
Diagnostics: Container [pid=2322,containerID=container_1553768779226_0005_01_000001] is running beyond virtual memory limits. Current usage: 75.1 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memor
y used. Killing container.Dump of the process-tree for container_1553768779226_0005_01_000001 :
	|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
	|- 2322 2321 2322 2322 (bash) 0 0 108650496 300 /bin/bash -c /opt/jdk/bin/java -Xmx424m  -Dlog.file=/opt/hadoop/logs/userlogs/application_1553768779226_0005/container_1553768779226_0005_01_000001/jobma
nager.log -Dlogback.configurationFile=file:logback.xml -Dlog4j.configuration=file:log4j.properties org.apache.flink.yarn.YarnApplicationMasterRunner  1> /opt/hadoop/logs/userlogs/application_1553768779226_0005/container_1553768779226_0005_01_000001/jobmanager.out 2> /opt/hadoop/logs/userlogs/application_1553768779226_0005/container_1553768779226_0005_01_000001/jobmanager.err 	|- 2330 2322 2322 2322 (java) 267 71 2165399552 18925 /opt/jdk/bin/java -Xmx424m -Dlog.file=/opt/hadoop/logs/userlogs/application_1553768779226_0005/container_1553768779226_0005_01_000001/jobmanager.lo
g -Dlogback.configurationFile=file:logback.xml -Dlog4j.configuration=file:log4j.properties org.apache.flink.yarn.YarnApplicationMasterRunner 
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Failing this attempt. Failing the application.
If log aggregation is enabled on your cluster, use this command to further investigate the issue:
yarn logs -applicationId application_1553768779226_0005
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.startAppMaster(AbstractYarnClusterDescriptor.java:902)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:582)
	at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:441)
	... 9 more
2019-03-28 06:46:32,070 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Cancelling deployment from Deployment Failure Hook
2019-03-28 06:46:32,070 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Killing YARN application
2019-03-28 06:46:32,082 INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl         - Killed application application_1553768779226_0005

出现此类错误,主要的原因是Current usage: 75.1 MB of 1 GB physical memory used; 2.1 GB of 2.1 GB virtual memor
y used. Killing container.

字面原因是容器内存不够,实际上是flink on yarn启动时检查虚拟内存造成的

所以修改配置文件,让它不检查就没事了

修改etc/hadoop/yarn-site.xml

<property>
        <name>yarn.nodemanager.pmem-check-enabled</name>
        <value>false</value>
</property>
<property>
        <name>yarn.nodemanager.vmem-check-enabled</name>
        <value>false</value>
</property>

 

  • 4
    点赞
  • 6
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值