问题:
消费者程序consumer_result.py消费kafka中内容,将py文件提交到spark中运行时,提示“Failed to find data source: kafka”
运行代码如下:
/usr/local/spark/bin/spark-submit consumer_result.py
解决:
运行命令改为如下:
#2.3.3对应spark2.3.3,2.11对应scala2.11
/usr/local/spark/bin/spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.3.3 consumer_result.py