spark
刘大宁
这个作者很懒,什么都没留下…
展开
-
Spark写ORC异常
报错信息SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/o.原创 2021-04-22 18:59:24 · 800 阅读 · 0 评论 -
SPARK任务提交资源分配
spark-defaults.confspark.master=yarn# spark_deploy_mode=clusterspark.driver.memory=1Gspark.executor.memory=1Gspark.testing.reservedMemory=0# 去除1.5*300M 最小提交内存限制,环境资源紧张时可以设为0yarn-site.xml<property> <name>yarn.nodem...原创 2021-04-20 12:29:30 · 534 阅读 · 0 评论