前提:配置文件/etc/spark/conf/spark-env.sh添加
export SPARK_CLASSPATH=$SPARK_CLASSPATH:/opt/mysql-connector-java/mysql-connector-java-5.1.40-bin.jar
这样的配置有时报错:
WARN spark.SparkConf: Setting 'spark.executor.extraClassPath' to ':/opt/mysql-connector-java/mysql-connector-java-5.1.40-bin.jar' as a work-around.
解决方案:
去掉上面的配置,编辑spark-defaults.conf
spark.executor.extraClassPath /opt/mysql-connector-java/mysql-connector-java-5.1.40-bin.jar