集群运行SparkR代码

编写R语言的代码（sparktest.R ）

library(SparkR)

# Initialize SparkSession
sparkR.session(appName = "SparkR-DataFrame-example")

# Create a simple local data.frame
localDF <- data.frame(name=c("John", "Smith", "Sarah"), age=c(19, 23, 18))

# Convert local data frame to a SparkDataFrame
df <- createDataFrame(localDF)

# Print its schema
printSchema(df)

# Create a DataFrame from a JSON file
path <- file.path(Sys.getenv("SPARK_HOME"), "hdfs:///user/asmp/flume_test/people.json")
peopleDF <- read.json(path)
printSchema(peopleDF)

# Register this DataFrame as a table.
createOrReplaceTempView(peopleDF, "people")

# SQL statements can be run by using the sql methods
teenagers <- sql("SELECT name FROM people WHERE age >= 13 AND age <= 19")

# Call collect to get a local data.frame
teenagersLocalDF <- collect(teenagers)

# Print the teenagers in our dataset
print(teenagersLocalDF)

# Stop the SparkSession now
sparkR.session.stop()

数据集people.json

{"name":"Michael"}
{"name":"Andy", "age":30}
{"name":"Justin", "age":19}

集群运行：shell>spark-submit ./sparktest.R 

运行结果：
root  
 |-- name: string (nullable = true)  
 |-- age: double (nullable = true)  
root  
 |-- age: long (nullable = true)  
 |-- name: string (nullable = true)  

name
1 Justin