Spark link集合

Part1. 各种参数的意义及如何配置

Spark官方文档——Spark Configuration（Spark配置）
http://www.cnblogs.com/vincent-hv/p/3316502.html

Standalone spark-env.sh 配置
http://www.sxt.cn/u/2839/blog/4730

spark1.0.0属性配置
http://blog.csdn.net/book_mmicky/article/details/29472439

spark-submit提交脚本中各项参数的配置
http://blog.csdn.net/book_mmicky/article/details/25714545

spark配置指南（全）：
http://database.51cto.com/art/201407/445881.htm

 

Part2. 不同的部署方式：standlone yarn

Spark源码分析：多种部署方式之间的区别与联系
http://www.tuicool.com/articles/jAZZZf

Spark不同部署方式：
http://developer.51cto.com/art/201309/410276_3.htm

 

 

Part3. 资源调度相关

Spark Executor Driver资源调度小结
http://blog.csdn.net/oopsoom/article/details/38763985

Spark任务调度executors分配问题 in yarn
http://www.cnblogs.com/jh818012/p/3936286.html

Spark技术内幕之任务调度：从SparkContext开始
http://www.tuicool.com/articles/VrqqYne

 

Part4. 性能调优

spark性能调优
http://itindex.net/detail/50670-spark-spark-%E6%80%A7%E8%83%BD%E8%B0%83%E4%BC%98

spark参数配置调优
http://blog.csdn.net/chenxingzhen001/article/details/11835399

Spark:Master High Availability（HA）高可用配置的2种实现
http://www.aboutyun.com/forum.php?mod=viewthread&tid=10823

spar性能调优
http://spark-config.readthedocs.org/en/latest/

spark性能相关参数http://blog.csdn.net/colorant/article/details/38681627

 

Part5. Something else

spark编程指南
http://my.oschina.net/kingwjb/blog/304945

Spark技术内幕：Executor分配详解
http://www.sxt.cn/u/2521/blog/3823

 

Part6. Some issues

centos6.2 spark cluster问题（持续追加）
http://blog.csdn.net/qianlong4526888/article/details/22899355

Spark fail when running pi.py example with yarn-client mode
http://stackoverflow.com/questions/27792839/spark-fail-when-running-pi-py-example-with-yarn-client-mode 

How do I set spark.akka.frameSize in spark-shell?
http://stackoverflow.com/questions/27215288/how-do-i-set-spark-akka-framesize-in-spark-shell 

 

Part7. 一些改进

 

Bloom Filter与大表连接结合

 

论文地址：Spark大表等值连接的优化及其在网络流量数据分析的应用研究（知网）

 

Bloom Filter（布隆过滤器）：
http://www.cnblogs.com/haippy/archive/2012/07/13/2590351.html
http://blog.csdn.net/jiaomeng/article/details/1495500

 

转载于:https://www.cnblogs.com/gnivor/p/4455846.html