![](https://img-blog.csdnimg.cn/20201014180756922.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
Spark
Rocky000000
加油,致最好的年华!!!
展开
-
Spark技术架构,概念及运行过程
Spark分布式内存计算平台采用的是Master-Slave架构,集群中含有Master进程的节点ClusterManager即为这里的Master,而Slave则是集群中的Work进程节点。Master作为整个集群的控制器,负责整个集群的正常运行,Worker则相当于是计算节点,接收主节点的命令,运行Driver或Excutor,并进行状态汇报;Executor运行在Worker节点。...转载 2018-12-02 20:04:59 · 1070 阅读 · 0 评论 -
spark启动slave时提示 JAVA_HOME is not set
出现的问题如下所示:admin01: failed to launch: nice -n 0 /home/hadoop/modules/spark-2.1.1-bin-hadoop2.7/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://admin01:7077admin01: J...原创 2018-12-01 12:26:19 · 1631 阅读 · 0 评论 -
Spark连接master报Connection refused: no further information:
错误信息如下所示:[hadoop@admin01 spark-2.1.1-bin-hadoop2.7]$ bin/spark-shell \> --master spark://admin01:7077 \> --executor-memory 2g \> --total-executor-cores 2Setting default log level to "WAR...原创 2018-12-01 13:31:26 · 3489 阅读 · 0 评论