- 描述
安装好CDH、Coudera Manager之后,通过Coudera Manager安装所有自带的应用,包括hdfs、hive、yarn、spark、hbase等。利用远程客户端在安装有spark的节点上,通过spark-shell启动spark,满怀期待的启动spark,but,来了个晴天霹雳,报错了,报错了!错误信息如下:
19/02/28 13:45:06 ERROR spark.SparkContext: Error initializing SparkContext.
java.lang.IllegalArgumentException: Required executor memory (1024+384 MB) is above the max threshold (1024 MB) of this cluster! Please check the values of 'yarn.scheduler.maximum-allocation-mb' and/or 'yarn.nodemanager.resource.memory-mb'.
at org.apache.spark.deploy.yarn.Client.verifyClusterResources(Client.scala:281)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:140)
at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:158)
(后面还有好多信息