运行spark项目的问题

运行spark项目的问题：

[root@aaaaaaa apptooljar]# spark-submit --master  spark://aaaaaaa:6066 --deploy-mode client --class controller.KafkaConsumeFlume /opt/software/apptooljar/BIOUSERAI.jar 
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
解决方案
使用压缩软件打开jar包 删除：META-INF目录下的除了mainfest.mf，.rsa及maven文件夹 以外的其他所有文件


19/09/18 20:59:09 WARN hdfs.HDFSEventSink: HDFS IO error
java.net.ConnectException: Call From bioyuanc/ip to ip:8020 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
Caused by: java.net.ConnectException: 拒绝连接
解决问题：修改core-site.xml中的localhost:8020为namenode的hostname/ip:8020



19/09/19 11:59:31 INFO yarn.Client: Application report for application_1568863151495_0009 (state: FINISHED)
19/09/19 11:59:31 ERROR spark.SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.
19/09/19 11:59:31 INFO ui.SparkUI: Stopped Spark web UI at http://ip:4040
19/09/19 11:59:31 ERROR client.TransportClient: Failed to send RPC 8762334242618233667 to aaaaaaa/ip:45080: java.nio.channels.ClosedChannelException
java.nio.channels.ClosedChannelException
19/09/19 11:59:31 ERROR cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Sending RequestExecutors(0,0,Map(),Set()) to AM was unsuccessful
java.io.IOException: Failed to send RPC 8762334242618233667 to aaaaaaa/ip:45080: java.nio.channels.ClosedChannelException
Caused by: java.nio.channels.ClosedChannelException
19/09/19 11:59:31 ERROR util.Utils: Uncaught exception in thread main
java.io.IOException: Failed to send RPC 8762334242618233667 to aaaaaaa/ip:45080: java.nio.channels.ClosedChannelException
Caused by: java.nio.channels.ClosedChannelException
19/09/19 11:59:31 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
19/09/19 11:59:31 ERROR util.Utils: Uncaught exception in thread main
java.lang.NullPointerException
19/09/19 11:59:31 INFO spark.SparkContext: Successfully stopped SparkContext
Exception in thread "main" org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.
解决方案：请求的core大于服务器的core


Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
解决方案：spark/scala版本不一致