<SPARK-轉載> spark 集群运行python作业

最新推荐文章于 2024-02-04 17:11:34 发布

vivianwang0709

最新推荐文章于 2024-02-04 17:11:34 发布

阅读量1.5k

点赞数

http://blog.csdn.net/xiaolewennofollow/article/details/47171951

今天尝试用刚搭建好的spark集群运行python作业，遇到了一些问题，解决了一些坑的同时也对spark集群的运作和配置方式有了一些比较浅的认识，不像之前那么没有概念了，记录如下，之后还要继续更多的对Hadoop生态圈和spark并行计算框架的探究。

首先说下环境，集群有五个节点，集群环境是用cloudera manager 搭建的，hadoop用的是cloudera的CDH，我对CDH和hadoop之间关系的理解就是与linux和CentOS的关系一样，其他的的相关组件例如Hbase和Hive也是用cloudera安装的，之前我看到服务器上已经有实验室学长下好的spark parcels安装包，于是也没看具体版本就直接用了，之后发现是0.9版本的，略囧。。。因为spark已经发布到1.4版本了，0.9版本都没有spark-submit,而且也不没有R的原生API，不过不影响，直接看0.9的文档就可以了，如果必须用到新版的功能就重新部署吧。。。。。

首先记录一下spark的四种运行模式

local：本地单进程模式，用于本地开发测试Spark代码
standalone：分布式集群模式，Master-Worker架构，Master负责调度，Worker负责具体Task的执行
on yarn/mesos：运行在yarn/mesos等资源管理框架之上，yarn/mesos提供资源管理，spark提供计算调度，并可与其他计算框架(如MapReduce/MPI/Storm)共同运行在同一个集群之上 (使用cloudera搭建的集群就是这种情况)
on cloud(EC2)：运行在AWS的EC2之上。

下面用python的一个简单作业SimpleApp.py为例，记录下脚本的运行过程

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">from pyspark import SparkContext,SparkConf

conf=SparkConf()
conf<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.setMaster</span>(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"spark://192.168.2.241:7077"</span>)
conf<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.setAppName</span>(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"test application"</span>)


logFile=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hdfs://hadoop241:8020/user/root/testfile"</span>
sc=SparkContext(conf=conf)
logData=sc<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.textFile</span>(logFile)<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cache</span>()


numAs=logData<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.filter</span>(lambda s: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'a'</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> s)<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.count</span>()
numBs=logData<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.filter</span>(lambda s: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'b'</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> s)<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.count</span>()

print <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Lines with a:%i,lines with b:%i"</span> % (numAs,numBs)
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>

关于这里的问题主要涉及到连接集群的配置问题，也就是上述代码的conf部分，首先要连接集群的master节点，注意这里的配置写法

spark://192.168.2.241:7077

前缀spark不可少，否则会报“could not parse master URL”的错误即无法解析URL的错误，至于端口号可以在/etc/spark/conf中查询$SPARK_MASTER_PORT这个环境变量，（具体安装方式配置文件位置也不同，根据具体情况来确定）

还有

logFile=”hdfs://hadoop241:8020/user/root/testfile”

我观察到这里默认是从hdfs文件系统上读取文件的，所以首先要把待处理文件put到hdfs上，同样注意路径的写法，这里写的是hdfs上得绝对路径，也可以写相对路径
这里的testfile里只有两句话，用来测试作业能否正确执行

<code class="hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">stay hungery,stay foolish
steve jobs</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

之后执行

$ pyspark SimpleApp.py

运行结果贴出如下，可以从中观察运算任务的分配调度过程

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@hadoop241 workspace]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># pyspark SimpleApp.py</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO slf4j<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Slf</span>4jLogger: Slf4jLogger started
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO Remoting: Starting remoting
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO Remoting: Remoting started<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">; listening on addresses :[akka.tcp://spark@hadoop241:34248]</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO Remoting: Remoting now listens on addresses: [akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tcp</span>://spark@hadoop241:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">34248</span>]
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO spark<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkEnv</span>: Registering BlockManagerMaster
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DiskBlockManager</span>: Created local directory at /tmp/spark-local-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162227</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">804</span>b
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.MemoryStore</span>: MemoryStore started with capacity <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.9</span> MB.
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO network<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ConnectionManager</span>: Bound socket to port <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">42522</span> with id = ConnectionManagerId(hadoop241,<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">42522</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMaster</span>: Trying to register BlockManager
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMasterActor</span>$BlockManagerInfo: Registering block manager hadoop241:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">42522</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.9</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMaster</span>: Registered BlockManager
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO spark<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.HttpServer</span>: Starting HTTP Server
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO server<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Server</span>: jetty-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7.6</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.v</span>20121106
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO server<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AbstractConnector</span>: Started SocketConnector<span class="hljs-localvars" style="box-sizing: border-box;">@0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">42944</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO broadcast<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.HttpBroadcast</span>: Broadcast server started at http://<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.241</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">42944</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO spark<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkEnv</span>: Registering MapOutputTracker
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO spark<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.HttpFileServer</span>: HTTP File server directory is /tmp/spark-c6a0d067-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">075</span>c-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">493</span>b-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">81</span>c8-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">754</span>f569a91b5
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO spark<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.HttpServer</span>: Starting HTTP Server
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO server<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Server</span>: jetty-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7.6</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.v</span>20121106
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO server<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AbstractConnector</span>: Started SocketConnector<span class="hljs-localvars" style="box-sizing: border-box;">@0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">33063</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO server<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Server</span>: jetty-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7.6</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.v</span>20121106
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/storage/rdd,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/storage,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/stages/stage,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/stages/pool,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/stages,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/environment,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/executors,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/metrics/json,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/static,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO handler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>: started o<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.e</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.j</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.s</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.h</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ContextHandler</span>{/,null}
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO server<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AbstractConnector</span>: Started SelectChannelConnector<span class="hljs-localvars" style="box-sizing: border-box;">@0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4040</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> INFO ui<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkUI</span>: Started Spark Web UI at http://hadoop241:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4040</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Connecting to master spark://<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.241</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7077.</span>..
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Connected to Spark cluster with app ID app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor added: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> on worker-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150728175302</span>-hadoop246-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> (hadoop246:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span>) with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span> cores
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Granted executor ID app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> on hostPort hadoop246:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span> cores, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">512.0</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor added: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> on worker-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150728175303</span>-hadoop245-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> (hadoop245:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span>) with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> cores
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Granted executor ID app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> on hostPort hadoop245:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> cores, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">512.0</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor added: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> on worker-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150728175303</span>-hadoop254-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> (hadoop254:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span>) with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> cores
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Granted executor ID app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> on hostPort hadoop254:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> cores, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">512.0</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor added: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> on worker-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150728175302</span>-hadoop241-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> (hadoop241:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span>) with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> cores
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Granted executor ID app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> on hostPort hadoop241:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> cores, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">512.0</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor added: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span> on worker-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150728175302</span>-hadoop217-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> (hadoop217:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span>) with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> cores
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Granted executor ID app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span> on hostPort hadoop217:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7078</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> cores, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">512.0</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor updated: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> is now RUNNING
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor updated: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> is now RUNNING
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor updated: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> is now RUNNING
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor updated: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> is now RUNNING
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.AppClient</span>$ClientActor: Executor updated: app-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20150731162228</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0018</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span> is now RUNNING
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.MemoryStore</span>: ensureFreeSpace(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">125687</span>) called with curMem=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, maxMem=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">309225062</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.MemoryStore</span>: Block broadcast_0 stored as values to memory (estimated size <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">122.7</span> KB, free <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.8</span> MB)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO mapred<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.FileInputFormat</span>: Total input paths to process : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO spark<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkContext</span>: Starting job: count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Got job <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> (count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>) with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> output partitions (allowLocal=false)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Final stage: Stage <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> (count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Parents of final stage: List()
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Missing parents: List()
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Submitting Stage <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> (PythonRDD[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>] at count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>), which has no missing parents
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Submitting <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> missing tasks from Stage <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> (PythonRDD[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>] at count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSchedulerImpl</span>: Adding task <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">set</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> tasks
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Registered executor: Actor[akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tcp</span>://sparkExecutor@hadoop246:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">48226</span>/user/Executor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#-1281030996] with ID 0</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Starting task <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> as TID <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> on executor <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>: hadoop246 (PROCESS_LOCAL)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Serialized task <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> as <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3119</span> bytes <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span> ms
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Starting task <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> as TID <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> on executor <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>: hadoop246 (PROCESS_LOCAL)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Serialized task <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> as <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3119</span> bytes <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> ms
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Registered executor: Actor[akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tcp</span>://sparkExecutor@hadoop217:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">34233</span>/user/Executor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#-994522395] with ID 4</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">30</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Registered executor: Actor[akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tcp</span>://sparkExecutor@hadoop241:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">53345</span>/user/Executor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#1663802475] with ID 3</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">30</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMasterActor</span>$BlockManagerInfo: Registering block manager hadoop246:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">34291</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.9</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">30</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMasterActor</span>$BlockManagerInfo: Registering block manager hadoop217:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">35324</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.9</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">30</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMasterActor</span>$BlockManagerInfo: Registering block manager hadoop241:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">54770</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.9</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Registered executor: Actor[akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tcp</span>://sparkExecutor@hadoop254:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">49492</span>/user/Executor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#-967494826] with ID 2</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> INFO cluster<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkDeploySchedulerBackend</span>: Registered executor: Actor[akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tcp</span>://sparkExecutor@hadoop245:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">34383</span>/user/Executor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#266145334] with ID 1</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMasterActor</span>$BlockManagerInfo: Registering block manager hadoop254:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">52135</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.9</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMasterActor</span>$BlockManagerInfo: Registering block manager hadoop245:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16696</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.9</span> MB RAM
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMasterActor</span>$BlockManagerInfo: Added rdd_1_0 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> memory on hadoop246:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">34291</span> (size: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">208.0</span> B, free: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.9</span> MB)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> INFO storage<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BlockManagerMasterActor</span>$BlockManagerInfo: Added rdd_1_1 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> memory on hadoop246:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">34291</span> (size: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">176.0</span> B, free: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">294.9</span> MB)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Finished TID <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2128</span> ms on hadoop246 (progress: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Finished TID <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2142</span> ms on hadoop246 (progress: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Completed ResultTask(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSchedulerImpl</span>: Remove TaskSet <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span> from pool
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Completed ResultTask(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Stage <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> (count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>) finished <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.476</span> s
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO spark<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkContext</span>: Job finished: count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>, took <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.550761544</span> s
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO spark<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkContext</span>: Starting job: count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Got job <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>) with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> output partitions (allowLocal=false)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Final stage: Stage <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Parents of final stage: List()
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Missing parents: List()
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Submitting Stage <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (PythonRDD[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>] at count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>), which has no missing parents
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Submitting <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> missing tasks from Stage <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (PythonRDD[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>] at count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSchedulerImpl</span>: Adding task <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">set</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.0</span> with <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> tasks
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Starting task <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> as TID <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> on executor <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>: hadoop246 (PROCESS_LOCAL)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Serialized task <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> as <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3121</span> bytes <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> ms
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Starting task <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> as TID <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> on executor <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>: hadoop246 (PROCESS_LOCAL)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Serialized task <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> as <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3121</span> bytes <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> ms
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Finished TID <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> ms on hadoop246 (progress: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Completed ResultTask(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSetManager</span>: Finished TID <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> ms on hadoop246 (progress: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Completed ResultTask(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.TaskSchedulerImpl</span>: Remove TaskSet <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.0</span> from pool
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DAGScheduler</span>: Stage <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>) finished <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.034</span> s
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">07</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">31</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> INFO spark<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.SparkContext</span>: Job finished: count at SimpleApp<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.py</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>, took <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.04234127</span> s
Lines with a:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>,lines with b:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li><li style="box-sizing: border-box; padding: 0px 5px;">93</li><li style="box-sizing: border-box; padding: 0px 5px;">94</li><li style="box-sizing: border-box; padding: 0px 5px;">95</li><li style="box-sizing: border-box; padding: 0px 5px;">96</li><li style="box-sizing: border-box; padding: 0px 5px;">97</li><li style="box-sizing: border-box; padding: 0px 5px;">98</li><li style="box-sizing: border-box; padding: 0px 5px;">99</li><li style="box-sizing: border-box; padding: 0px 5px;">100</li><li style="box-sizing: border-box; padding: 0px 5px;">101</li><li style="box-sizing: border-box; padding: 0px 5px;">102</li><li style="box-sizing: border-box; padding: 0px 5px;">103</li><li style="box-sizing: border-box; padding: 0px 5px;">104</li><li style="box-sizing: border-box; padding: 0px 5px;">105</li></ul><div class="save_code tracking-ad" data-mod="popu_249" style="box-sizing: border-box; position: absolute; height: 60px; right: 30px; top: 5px; color: rgb(255, 255, 255); cursor: pointer; z-index: 2;"><a target=_blank target="_blank" style="color: rgb(202, 0, 0); box-sizing: border-box;"><img src="http://static.blog.csdn.net/images/save_snippets.png" style="border: none; box-sizing: border-box; max-width: 100%;" alt="" /></a></div>