kUI' failed after 16 retries (starting from 4040)! Consider explicitly setting the appropriate port for the service 'SparkUI' (for example spark.ui.port for SparkUI) to an available port or increasing spark.port.maxRetries.
at org.sparkproject.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349)
at org.sparkproject.jetty.server.ServerConnector.open(ServerConnector.java:310)
at org.sparkproject.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.sparkproject.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
at org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.apache.spark.ui.JettyUtils$.newConnector$1(JettyUtils.scala:302)
at org.apache.spark.ui.JettyUtils$.httpConnect$1(JettyUtils.scala:333)
at org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$5(JettyUtils.scala:336)
at org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$5$adapted(JettyUtils.scala:336)
at org.apache.spark.util.Utils$.$anonfun$startServiceOnPort$2(Utils.scala:2331)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)
at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2323)
at org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:337)
at org.apache.spark.ui.WebUI.bind(WebUI.scala:146)
at org.apache.spark.SparkContext.$anonfun$new$11(SparkContext.scala:486)
at org.apache.spark.SparkContext.$anonfun$new$11$adapted(SparkContext.scala:486)
at scala.Option.foreach(Option.scala:407)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:486)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2672)
at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:945)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:939)
at com.mm.rdd.WordCount$.main(WordCount.scala:6)
at com.mm.rdd.WordCount.main(WordCount.scala)
23/05/08 11:46:11 INFO DiskBlockManager: Shutdown hook called
23/05/08 11:46:11 INFO ShutdownHookManager: Shutdown hook called
解决办法:
最终解决方案:
我是在IntellijIdea下运行的,试着安装了windows版本下的hadoop3.2.4核spark3.2.4,都不行;最后是创建统一入口SparkSession,指令后加了一条.config("spark.port.maxRetries",10000):
val spark = SparkSession.builder().master("local[*]").appName("word count").config("spark.port.maxRetries",10000).getOrCreate();成功了。