PYSPARK_DRIVER_PYTHON="ipython" bin/pysparkling
from pysparkling import *
import h2o
hc = H2OContext.getOrCreate(spark)
此时会出现错误:
Unsupported argument:(spark.dynamicAllocation.enabled,true)
查到如下解决方法:这里 和 这里
right now Sparkling Water/RSparkling does not support dynamic Spark cluster. SO you just need to disable it:
config = list(“spark.dynamicAllocation.enabled” = “false”)
所以这样使用:
PYSPARK_DRIVER_PYTHON="ipython" bin/pysparkling --conf spark.dynamicAllocation.enabled=false
sparkling-water Unsupported argument: (spark.dynamicAllocation.enabled,true) on CDH
最新推荐文章于 2024-07-19 17:32:19 发布