十一 kafka数据安全，以及Spark Kafka Streaming API

一基本网址
http://spark.apache.org/docs/1.6.2/api/java/index.html
在API中搜索org.apache.spark.streaming.kafka

二spark对接kafka流两种方案
在org.apache.spark.streaming.kafka中有个KafkaUtils ，里面有两种方案
其中获取kafka的流有两种方案
2.1其中第一种
Modifier and type
static <K,V,KD extends kafka.serializer.Decoder<K>,VD extends kafka.serializer.Decoder<V>,R>
JavaInputDStream<R>

Method and Description
createDirectStream(JavaStreamingContext jssc, java.lang.Class<K> keyClass, java.lang.Class<V> valueClass, java.lang.Class<KD> keyDecoderClass, java.lang.Class<VD> valueDecoderClass, java.lang.Class<R> recordClass, java.util.Map<java.lang.String,java.lang.String> kafkaParams, java.util.Map<kafka.common.TopicAndPartition,java.lang.Long> fromOffsets, Function<kafka.message.MessageAndMetadata<K,V>,R> messageHandler)

Create an input stream that directly pulls messages from Kafka Brokers without using any receiver.
2.2另外一种方案，方法简单
static JavaPairReceiverInputDStream<java.lang.String,java.lang.String>
static JavaPairReceiverInputDStream<java.lang.String,java.lang.String> createStream(JavaStreamingContext jssc, java.lang.String zkQuorum, java.lang.String groupId, java.util.Map<java.lang.String,java.lang.Integer> topics)
Create an input stream that pulls messages from Kafka Brokers.

static JavaPairReceiverInputDStream<java.lang.String,java.lang.String>
static JavaPairReceiverInputDStream<java.lang.String,java.lang.String> createStream(JavaStreamingContext jssc, java.lang.String zkQuorum, java.lang.String groupId, java.util.Map<java.lang.String,java.lang.Integer> topics, StorageLevel storageLevel)
Create an input stream that pulls messages from Kafka Brokers.

三网络文章
https://www.cnblogs.com/seaspring/p/5920414.html