在R或Rstudio中调用SparkR

原文地址：http://blog.csdn.net/u010022051/article/details/51119008

libpath <- .libPaths()
libpath <- c(libpath, "/home/SparkR/spark-1.6.1-bin-hadoop2.6/R/lib")
.libPaths(libpath)
rm(libpath)
library(rJava)
library(devtools)

library(SparkR)

sc <- sparkR.init("local")
或sc <- sparkR.init(master="Spark://172.26.40.74:7077")
sqlContext <- sparkRSQL.init(sc)

Linux下加载R包：
install.packages('Cairo', dependencies=TRUE, repos='http://cran.rstudio.com/')

在spark中启动SparkR的命令：

bin/sparkR --master yarn

bin/sparkR --172.26.40.75 local[2]

bin/sparkR --master spark://172.26.40.74:7077 --executor-memory 8g --total-executor-cores 45 --conf spark.ui.port=54089