![](https://img-blog.csdnimg.cn/20201014180756754.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
Spark
文章平均质量分 93
哈天奇不奇
这个作者很懒,什么都没留下…
展开
-
Spark HA
原创转载请注明出处:http://agilestyle.iteye.com/blog/2294076 前期准备 zookeeper集群搭建完毕 Scala环境配置完毕 export JAVA_HOME=/home/hadoop/app/jdk1.8.0_77 export HADOOP_HOME=/home/hadoop/app/hadoop-2.6.4 export HIV...原创 2016-04-26 16:37:36 · 113 阅读 · 0 评论 -
Spark整合HDFS、WordCount示例
原创转载请注明出处:http://agilestyle.iteye.com/blog/2294233 前提条件 Hadoop HA搭建完毕 Spark HA搭建完毕 整合步骤 cd到spark的conf的目录,修改spark-env.sh 添加如下 export HADOOP_CONF_DIR=/home/hadoop/app/hadoop-2.6.4/et...原创 2016-04-27 11:02:16 · 192 阅读 · 0 评论 -
Spark RDD特点
RDD:Resilient Distributed Dataset A Resilient Distributed Dataset(RDD), the basic abstraction in Spark. Represents an immutable, partitioned collection of elements that can be operated on in parall...原创 2016-04-27 14:03:19 · 226 阅读 · 0 评论 -
High-level Spark architecture
原创转载请注明出处:http://agilestyle.iteye.com/blog/2335696 Spark Introduction MapReduce is the primary workhorse at the core of most Hadoop clusters. While highly effective for very large batch-analyti...原创 2016-11-07 09:49:50 · 137 阅读 · 0 评论 -
Spark Build
原创转载请注明出处:http://agilestyle.iteye.com/blog/2337293 Prerequisite 硬件环境:Ubuntu16.04(8G内存) 软件环境:jdk1.7.0_80+scala-2.11.8+apache-maven-3.3.9 配置Linux下的环境变量(hadoop、hbase、hive以及zookeeper可以忽略) ...原创 2016-11-11 11:05:42 · 250 阅读 · 0 评论