spark
JHC_binge
励志专业IT开发
展开
-
sprark maven pom.xml依赖
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://原创 2018-12-20 19:07:14 · 271 阅读 · 0 评论 -
spark分布式安装配置
1.配置worker vim conf/slaves.template 2.配置master vim spark-env.sh export JAVA_HOME=/usr/local/jdk1.8.0_201 export SCALA_HOME=/usr/local/scala export HADOOP_HOME=/usr/local/hadoop export HADOOP...原创 2019-01-25 17:35:33 · 228 阅读 · 0 评论 -
java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutabl
19/02/22 19:06:55 INFO VerifiableProperties: Property auto.offset.reset is overridden to smallest 19/02/22 19:06:55 INFO VerifiableProperties: Property group.id is overridden to 19/02/22 19:06:55 INFO...原创 2019-02-22 19:12:12 · 2065 阅读 · 0 评论 -
spark接收kafka数据,进行WordCount
package kafka import org.apache.spark.SparkConf import org.apache.spark.streaming.{Seconds, StreamingContext} import org.apache.spark.streaming.kafka010._ import scala.collection.mutable //1.打开zk,...原创 2019-02-26 18:35:51 · 489 阅读 · 2 评论