CDH-5.8.3集群上spark-submit --master yarn。。。提交任务时,当使用mysql-connector-java-5.1.40-bin.jar等jar包时可能会出现以下问题:
Did not find registered driver with class com.mysql.jdbc.Driver
解决方案:
每个节点(master节点和slave节点)把mysql-connector-java-5.1.40-bin.jar拷贝到HADOOP_YARN_HOME下。
CDH-5.8.3组件的HOME路径:/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2
HADOOP_YARN_HOME的jar包所在路径:
/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/hadoop-yarn
在每个节点(master和slave)中把mysql-connector-java-5.1.40-bin.jar复制到/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/hadoop-yarn/路径下。