spark
KNIFE_PAN
这个作者很懒,什么都没留下…
展开
-
cdh 运行spark yarn-cluster
1.如果用cdh安装sparn on yarn 直接用集群模式运行spark-submit --class org.apache.spark.examples.SparkPi \ --master yarn-cluster \ --num-executors 3 \ --driver-memory 4g \ --executor-memory 2g \ -原创 2015-04-23 13:22:12 · 4841 阅读 · 0 评论 -
SparkFlumeEvent转换为实际输入字符串
JavaReceiverInputDStream<SparkFlumeEvent> flumeStream = FlumeUtils .createStream(ssc, host, port); flumeStream.flatMap(new FlatMapFunction<SparkFlumeEvent, String>() {原创 2015-04-15 10:40:56 · 1617 阅读 · 0 评论 -
SparkStreaming+Flume 环境搭建配置
1.JavaFlumeEventCountpackage cn.gitv.bi.log.analysis.spark; import org.apache.spark.SparkConf; import org.apache.spark.api.java.function.Function; import org.apache.spark.streaming.Duration; import org原创 2015-04-15 10:32:28 · 956 阅读 · 0 评论