Spark Streaming's Kafka libraries not found in class path. Try one of the following.
1. Include the Kafka library and its dependencies with in the
spark-submit command as
$ bin/spark-submit --packages org.apache.spark:spark-streaming-kafka-0-8:2.3.1 ...
2. Download the JAR of the artifact from Maven Central http://search.maven.org/,
Group Id = org.apache.spark, Artifact Id = spark-streaming-kafka-0-8-assembly, Version = 2.3.1.
Then, include the jar in the spark-submit command as
$ bin/spark-submit --jars <spark-streaming-kafka-0-8-assembly.jar> ...
这个时候需要下一个jar包放到py的D:\setup\Anaconda\Lib\site-packages\pyspark\jars下,如果放到服务器上则需要制定jar包
<dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-streaming-kafka-0-8-assembly_2.11</artifactId> <version>2.3.1</version> </dependency>
可以通过maven下载