spark
刘大宁
这个作者很懒,什么都没留下…
展开
-
Spark写ORC异常
报错信息 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/o.原创 2021-04-22 18:59:24 · 809 阅读 · 0 评论 -
SPARK任务提交资源分配
spark-defaults.conf spark.master=yarn # spark_deploy_mode=cluster spark.driver.memory=1G spark.executor.memory=1G spark.testing.reservedMemory=0 # 去除1.5*300M 最小提交内存限制,环境资源紧张时可以设为0 yarn-site.xml <property> <name>yarn.nodem...原创 2021-04-20 12:29:30 · 562 阅读 · 0 评论