你不能错过的 spark 学习资源_spark2.x书籍推荐-CSDN博客

上面两篇是 databricks 出的关于 databricks 专业版的描述，虽然没有从根本上解决问题，但是读起来还是挺有说服力的，哈哈，因为采用了很多很细节的方案。不错不错，各位有在做云产品的，在宣传自己的安全方案时可用参考参考哦。

Deep Dive into Spark SQL’s Catalyst Optimizer
A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets
Understanding your Apache Spark application through visualization
New Visualizations for Understanding Apache Spark Streaming Applications
An Introduction to Writing Apache Spark Applications on Databricks
A Gentle Introduction to Apache Spark on Databricks
Apache Spark on Databricks for Data Scientists
Import Notebook Apache Spark on Databricks for Data Engineers
Structured Streaming In Apache Spark: A new high-level API for streaming
Spark SQL Supported Syntax
Combining Machine Learning Frameworks with Apache Spark
Deep Dive: Memory Management in Apache Spark
https://databricks.com/blog/2017/07/11/introducing-apache-spark-2-2.html
https://spark-summit.org/east-2017/events/improving-python-and-spark-performance-and-interoperability/
http://www.slideshare.net/wesm/high-performance-python-on-apache-spark
https://www.slideshare.net/SparkSummit/spark-and-online-analytics-spark-summit-east-talky-by-shubham-chopra
https://www.slideshare.net/SparkSummit/spark-parquet-in-depth-spark-summit-east-talk-by-emily-curtin-and-robbie-strickland
https://www.slideshare.net/databricks/sparksql-a-compiler-from-queries-to-rdds

4. 文章，博客

RDD论文英文版
RDD论文中文版
An Architecture for Fast and General Data Processing on Large Clusters
How-to: Tune Your Apache Spark Jobs (Part 1)
How-to: Tune Your Apache Spark Jobs (Part 2)
By 0x0fff Spark Misconceptions
By 0x0fff Spark Architecture
By 0x0fff Spark DataFrames are faster, aren’t they?
By 0x0fff Spark Architecture: Shuffle
By 0x0fff Modern Data Architecture
By 0x0fff Spark Architecture Talk
By 0x0fff Apache Spark Future
By 0x0fff Data Industry Trends
By 0x0fff Spark Memory Management
By 0x0fff Spark Architecture Video
借助 Redis ，让 Spark 提速 45 倍！
量化派基于Hadoop、Spark、Storm的大数据风控架构
基于Spark的异构分布式深度学习平台
你对Hadoop和Spark生态圈了解有几许？
Hadoop vs Spark
雅虎开源CaffeOnSpark：基于Hadoop/Spark的分布式深度学习
2016 上海第二次 spark meetup: 1. spark_meetup.pdf
2016 上海第二次 spark meetup: 2. Flink_ An unified stream engine.pdf
2016 上海第二次 spark meetup: 3. Spark在计算广告领域的应用实践.pdf
2016 上海第二次 spark meetup: 4. splunk_spark.pdf
基于Spark的医疗和金融大数据
Monitoring Spark with Graphite and Grafana
Spark使用CombineTextInputFormat缓解小文件过多导致Task数目过多的问题
Databricks Empowers Enterprises to Secure Their Apache Spark Workloads
Spark配置参数
Running Spark Python Applications
How-to: Prepare Your Apache Hadoop Cluster for PySpark Jobs
Apache Spark’s Hidden REST API
[SQL, sqlContext, hiveContext]
- What is the difference between Apache Spark SQLContext vs HiveContext?
- SQL Window Functions
- Performance Tuning SQL Queries
- HIVE-vs-RDBMS
- Windowing and Analytics Functions
[Spark Memory Issues]
- http://www.cnblogs.com/wrencai/p/4231934.html
- http://stackoverflow.com/questions/32349611/what-should-be-the-optimal-value-for-spark-sql-shuffle-partitions-or-how-do-we-i
- http://stackoverflow.com/questions/21138751/spark-java-lang-outofmemoryerror-java-heap-space
- A Beginner’s Guide on Troubleshooting Spark Applications
- http://www.mincoder.com/article/2381.shtml
http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-1/
http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/
http://blog.cloudera.com/blog/2014/05/apache-spark-resource-management-and-yarn-app-models/
https://spark.apache.org/docs/latest/job-scheduling.html
https://databricks.com/blog/2015/06/22/understanding-your-spark-application-through-visualization.html
http://blog.csdn.net/u012684933/article/details/50378725

5. 视频

YouTube: what is apache spark
Introduction to Spark Architecture
Top 5 Mistakes When Writing Spark Applications
slide Top 5 mistakes when writing Spark applications
Tuning and Debugging Apache Spark
slide Tuning and Debugging Apache Spark
A Deeper Understanding of Spark Internals - Aaron Davidson (Databricks)
slide A Deeper Understanding of Spark Internals - Aaron Davidson (Databricks)
Building, Debugging, and Tuning Spark Machine Learning Pipelines - Joseph Bradley (Databricks)
slide Building, Debugging, and Tuning Spark Machine Learning Pipelines
Spark DataFrames Simple and Fast Analysis of Structured Data - Michael Armbrust (Databricks)
slide Spark DataFrames Simple and Fast Analysis of Structured Data - Michael Armbrust (Databricks)
Spark Tuning for Enterprise System Administrators
slide Spark Tuning for Enterprise System Administrators
Structuring Spark: DataFrames, Datasets, and Streaming
slide Structuring Spark: DataFrames, Datasets, and Streaming
Spark in Production: Lessons from 100+ Production Users
slide Spark in Production: Lessons from 100+ Production Users
Production Spark and Tachyon use Cases
slide Production Spark and Tachyon use Cases
SparkUI Visualization
slide SparkUI Visualization
Everyday I’m Shuffling - Tips for Writing Better Spark Programs, Strata San Jose 2015
slide Everyday I’m Shuffling - Tips for Writing Better Spark Programs, Strata San Jose 2015
Large Scale Distributed Machine Learning on Apache Spark
Securing your Spark Applications
slide Securing your Spark Applications
Building a REST Job Server for Interactive Spark as a Service
slide Building a REST Job Server for Interactive Spark as a Service
Exploiting GPUs for Columnar DataFrame Operations
slide Exploiting GPUs for Columnar DataFrame Operations
Easy JSON Data Manipulation in Spark - Yin Huai (Databricks)
slide Easy JSON Data Manipulation in Spark - Yin Huai (Databricks)
Sparkling: Speculative Partition of Data for Spark Applications - Peilong Li
slide Sparkling: Speculative Partition of Data for Spark Applications - Peilong Li
Advanced Spark Internals and Tuning – Reynold Xin
slide Advanced Spark Internals and Tuning – Reynold Xin
The Future of Real Time in Spark
slide The Future of Real Time in Spark
Spark 2 0
slide Spark 2 0
Democratizing Access to Data
slide Democratizing Access to Data
Not Your Father’s Database: How to Use Apache Spark Properly in Your Big Data Architecture
slide Not Your Father’s Database: How to Use Apache Spark Properly in Your Big Data Architecture
Disrupting Big Data with Apache Spark in the Cloud
slide Disrupting Big Data with Apache Spark in the Cloud

我一直很欣赏 databricks 出的 video 和 slide，结构非常清晰，这个是其中一个非常好的演讲，里面有很多值得借鉴的地方，特别是当你像别人介绍你的工作，产品的时候。[我有一个感受，很少有人能清晰，有条理的介绍自己正在做的产品，对于一些小众的产品，甚至一些职业的销售也难以做到清晰，简明的叙述。这个 video 和 slide 有很大的参考价值。我自己感觉仔细研究这些 video 和 slide 有时候比看上一两本专业讲销售的书还要管用。]