SPARK 源码分析技术分享(带bilibili视频)

SPARK 源码分析技术分享 (带bilibili视频)

【本站点正在持续更新中…2018-12-05…】

• SPARK 1.6.0-cdh5.15.0
• Hadoop 2.6.0-cdh5.15.0
• spark-scala-maven
• 微信(技术交流) : thinktothings
• SPARK 源码分析技术分享(视频汇总套装视频): https://www.bilibili.com/video/av37442139/
• SPARK 源码分析技术分享(视频汇总在线看):https://blog.csdn.net/thinktothings/article/details/84726769
• SPARK 源码分析技术分享 (github) : https://github.com/opensourceteams/spark-scala-maven

RDD依赖 Dependency

NarrowDependency

• OneToOneDependency

  • bilibili 视频: https://www.bilibili.com/video/av37442139/?p=1
  • 详细说明文档： https://github.com/opensourceteams/spark-scala-maven/blob/master/md/OneToOneDenpendency.md
  width="800" height="500" src="//player.bilibili.com/player.html?aid=37442139&cid=65822237&page=1" scrolling="no" border="0" allowfullscreen="true">

• RangeDependency

  • bilibili 视频: https://www.bilibili.com/video/av37442139/?p=2
  • 详细说明文档： https://github.com/opensourceteams/spark-scala-maven/blob/master/md/RangeDependency.md
  width="800" height="500" src="//player.bilibili.com/player.html?aid=37442139&cid=65822237&page=2" scrolling="no" border="0" allowfullscreen="true">

• PruneDependency filter

  • bilibili 视频: https://www.bilibili.com/video/av37442139/?p=3
  • 详细说明文档： https://github.com/opensourceteams/spark-scala-maven/blob/master/md/PruneDependency.md
  width="800" height="500" src="//player.bilibili.com/player.html?aid=37442139&cid=65822237&page=3" scrolling="no" border="0" allowfullscreen="true">

• PruneDependency RangePartitioner

  • bilibili 视频: https://www.bilibili.com/video/av37442139/?p=4
  • 详细说明文档： https://github.com/opensourceteams/spark-scala-maven/blob/master/md/PruneDependencyRangePartitioner.md
  width="800" height="500" src="//player.bilibili.com/player.html?aid=37442139&cid=65822237&page=4" scrolling="no" border="0" allowfullscreen="true">

ShuffleDependency

• bilibili 视频: https://www.bilibili.com/video/av37442139/?p=5
• 详细说明文档：https://github.com/opensourceteams/spark-scala-maven/blob/master/md/ShuffleDependency.md

width="800" height="500" src="//pla