spark调用python_使用spark-submit将Python文件提交给Spark时,输出消息会到达哪里

[hadoop@ip-172-31-34-124~]$./spark/bin/spark-submit--master yarn-cluster./submit-test.py15/08/0423:50:25INFO client.RMProxy:ConnectingtoResourceManagerat/172.31.34.124:902215/08/0423:50:25INFO yarn.Client:Requestinga new applicationfromclusterwith2NodeManagers15/08/0423:50:25INFO yarn.Client:Verifyingour application hasnotrequested more than the maximum memory capability of the cluster(11520MB per container)15/08/0423:50:25INFO yarn.Client:Willallocate AM container,with896MB memory including384MB overhead15/08/0423:50:25INFO yarn.Client:Settingup container launch contextforour AM15/08/0423:50:25INFO yarn.Client:Preparingresourcesforour AM container15/08/0423:50:25INFO yarn.Client:Uploadingresource file:/home/hadoop/.versions/spark-1.3.1.e/lib/spark-assembly-1.3.1-hadoop2.4.0.jar->hdfs://172.31.34.124:9000/user/hadoop/.sparkStaging/application_1438724051797_0007/spark-assembly-1.3.1-hadoop2.4.0.jar15/08/0423:50:26INFO metrics.MetricsSaver:MetricsConfigRecorddisabledInCluster:false instanceEngineCycleSec:60clusterEngineCycleSec:60disableClusterEngine:false maxMemoryMb:3072maxInstanceCount:50015/08/0423:50:26INFO metrics.MetricsSaver:CreatedMetricsSaverj-2LU0EQ3JH58CK:i-048c1ded:SparkSubmit:24928period:60/mnt/var/em/raw/i-048c1ded_20150804_SparkSubmit_24928_raw.bin15/08/0423:50:27INFO metrics.MetricsSaver:1aggregatedHDFSWriteDelay1053raw values into1aggregated values,total115/08/0423:50:27INFO yarn.Client:Uploadingresource file:/home/hadoop/submit-test.py->hdfs://172.31.34.124:9000/user/hadoop/.sparkStaging/application_1438724051797_0007/submit-test.py15/08/0423:50:27INFO yarn.Client:Settingup the launch environmentforour AM container15/08/0423:50:27INFO spark.SecurityManager:Changingview acls to:hadoop15/08/0423:50:27INFO spark.SecurityManager:Changingmodify acls to:hadoop15/08/0423:50:27INFO spark.SecurityManager:SecurityManager:authentication disabled;ui acls disabled;userswithview permissions:Set(hadoop);userswithmodify permissions:Set(hadoop)15/08/0423:50:27INFO yarn.Client:Submittingapplication7toResourceManager15/08/0423:50:27INFO impl.YarnClientImpl:Submittedapplication application_1438724051797_000715/08/0423:50:28INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:ACCEPTED)15/08/0423:50:28INFO yarn.Client:client token:N/A

diagnostics:N/AApplicationMasterhost:N/AApplicationMasterRPC port:-1queue:default

start time:1438732227551final status:UNDEFINED

tracking URL:http://172.31.34.124:9046/proxy/application_1438724051797_0007/user:hadoop15/08/0423:50:29INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:ACCEPTED)15/08/0423:50:30INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:ACCEPTED)15/08/0423:50:31INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:ACCEPTED)15/08/0423:50:32INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:ACCEPTED)15/08/0423:50:33INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:ACCEPTED)15/08/0423:50:34INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:34INFO yarn.Client:client token:N/A

diagnostics:N/AApplicationMasterhost:ip-172-31-39-205.ec2.internalApplicationMasterRPC port:0queue:default

start time:1438732227551final status:UNDEFINED

tracking URL:http://172.31.34.124:9046/proxy/application_1438724051797_0007/user:hadoop15/08/0423:50:35INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:36INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:37INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:38INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:39INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:40INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:41INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:42INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:43INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:RUNNING)15/08/0423:50:44INFO yarn.Client:Applicationreportforapplication_1438724051797_0007(state:FINISHED)15/08/0423:50:44INFO yarn.Client:client token:N/A

diagnostics:N/AApplicationMasterhost:ip-172-31-39-205.ec2.internalApplicationMasterRPC port:0queue:default

start time:1438732227551final status:SUCCEEDED

tracking URL:http://172.31.34.124:9046/proxy/application_1438724051797_0007/A

user:hadoop

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值