Spark
文章平均质量分 93
哈天奇不奇
这个作者很懒,什么都没留下…
展开
-
Spark HA
原创转载请注明出处:http://agilestyle.iteye.com/blog/2294076 前期准备zookeeper集群搭建完毕Scala环境配置完毕export JAVA_HOME=/home/hadoop/app/jdk1.8.0_77export HADOOP_HOME=/home/hadoop/app/hadoop-2.6.4export HIV...原创 2016-04-26 16:37:36 · 112 阅读 · 0 评论 -
Spark整合HDFS、WordCount示例
原创转载请注明出处:http://agilestyle.iteye.com/blog/2294233 前提条件Hadoop HA搭建完毕Spark HA搭建完毕 整合步骤cd到spark的conf的目录,修改spark-env.sh 添加如下export HADOOP_CONF_DIR=/home/hadoop/app/hadoop-2.6.4/et...原创 2016-04-27 11:02:16 · 191 阅读 · 0 评论 -
Spark RDD特点
RDD:Resilient Distributed DatasetA Resilient Distributed Dataset(RDD), the basic abstraction in Spark. Represents an immutable, partitioned collection of elements that can be operated on in parall...原创 2016-04-27 14:03:19 · 225 阅读 · 0 评论 -
High-level Spark architecture
原创转载请注明出处:http://agilestyle.iteye.com/blog/2335696 Spark IntroductionMapReduce is the primary workhorse at the core of most Hadoop clusters. While highly effective for very large batch-analyti...原创 2016-11-07 09:49:50 · 136 阅读 · 0 评论 -
Spark Build
原创转载请注明出处:http://agilestyle.iteye.com/blog/2337293 Prerequisite硬件环境:Ubuntu16.04(8G内存) 软件环境:jdk1.7.0_80+scala-2.11.8+apache-maven-3.3.9 配置Linux下的环境变量(hadoop、hbase、hive以及zookeeper可以忽略)...原创 2016-11-11 11:05:42 · 249 阅读 · 0 评论