sparkstreaming
果不其燃
这个作者很懒,什么都没留下…
展开
-
CheckPoint-Java开发
CheckPoint-Java开发public final class JavaRecoverableNetworkWordCount { private static final Pattern SPACE = Pattern.compile(" "); private static JavaStreamingContext createContext(String ip, int port原创 2020-08-03 14:05:46 · 260 阅读 · 0 评论 -
Windows算子-使用Java开发
Windows算子-使用Java开发 Logger.getLogger("org").setLevel(Level.WARN); SparkConf conf = new SparkConf(); conf.setMaster("local[2]"); conf.setAppName("windows"); JavaStreamingContext jsc = new JavaStreamingContext(conf, Durations.原创 2020-05-13 21:54:56 · 128 阅读 · 0 评论 -
Transform算子-使用Java开发
Transform算子-使用Java开发 Logger.getLogger("org").setLevel(Level.WARN); SparkConf conf = new SparkConf(); conf.setMaster("local[2]").setAppName("WordBlock"); JavaStreamingContext jssc = new JavaStreamingContext(conf, Durations.seconds(原创 2020-05-13 19:44:42 · 264 阅读 · 0 评论 -
Exception in thread "JobGenerator" java.lang.NoClassDefFoundError: org/apache/hadoop/crypto/key/KeyP
Exception in thread “JobGenerator” java.lang.NoClassDefFoundError: org/apache/hadoop/crypto/key/KeyProviderException in thread “JobGenerator” java.lang.NoClassDefFoundError: org/apache/hadoop/crypto/key/KeyProviderat org.apache.hadoop.hdfs.DistributedFil原创 2020-05-11 13:35:01 · 452 阅读 · 0 评论 -
Error connecting to localhost:9999 - java.net.ConnectException: Connection refused: connect
Error connecting to localhost:9999 - java.net.ConnectException: Connection refused: connect20/05/11 13:25:54 ERROR ReceiverTracker: Deregistered receiver for stream 0: Restarting receiver with delay 2000ms: Error connecting to localhost:9999 - java.net.Co原创 2020-05-11 13:28:10 · 2878 阅读 · 2 评论 -
Tuple2 报错Incompatible equality constrain T1 and String
Tuple2 报错Incompatible equality constrain T1 and String**解决方式:**可能是spark_core依赖包版本问题,更换一个版本就行了。原创 2020-05-10 19:43:47 · 1593 阅读 · 0 评论 -
使用Flume工具采集数据到SparkStreaming应用程序
从HDFS中使用Flume工具采集数据到SparkStreaming应用程序原创 2019-12-04 09:52:46 · 127 阅读 · 0 评论