spark中读取json_在缺少json数据源的spark中读取json

我试图使用下面的代码将示例json文件读入SqlContext，但它失败并出现数据源错误 .

val sqlContext = new org.apache.spark.sql.SQLContext(sc)

val path = "C:\\samplepath\\sample.json"

val jsondata = sqlContext.read.json(path)

java.lang.ClassNotFoundException：无法找到数据源：json . 请在orsp.apache.spache.spark.sql.execution.datasources的orsp.apache.spark.sql.execution.datasources.ResolvedDataSource $ .lookupDataSource(ResolvedDataSource.scala：77)的http://spark-packages.org上找到软件包 . ResolvedDataSource $ .apply(ResolvedDataSource.scala：102)位于org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala：119)的org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala：109)at at org.apache.spark.sql.DataFrameReader.json(DataFrameReader.scala：244)org.apache.spark.deploy.SparkSubmit $ .doRunMain $ 1(SparkSubmit.scala：181)at org.apache.spark.deploy.SparkSubmit $ .submit(SparkSubmit.scala：206)org.apache.spark.deploy.SparkSubmit $ .main(SparkSubmit.scala：121)at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)引起：java .lang.ClassNotFoundException：java.lang.ClassLoader上java.lang.ClassLoader.loadClass(ClassLoader.java:424)中的scala.tools.nsc.interpreter.AbstractFileClassLoader.findClass(AbstractFileClassLoader.scala：83)中的json.DefaultSource .loadClass(ClassLoader.java:357)atg.apache.spark.sql.execution.datasources.ResolvedDataSource $$ anonfun $ 4 $$ anonfun $ apply $ 1.apply(ResolvedDataSource.scala：62)at org.apache.spark.sql .execution.datasources.ResolvedDataSource $$ anonfun $ 4 $$ anonfun $ apply $ 1.apply(ResolvedDataSource.scala：62)在scala.util.Try $ .apply(Try.scala：161)org.apache.spark.sql . execution.datasources.ResolvedDataSource $$ anonfun $ 4.apply(ResolvedDataSource.scala：62)位于scala.util的org.apache.spark.sql.execution.datasources.ResolvedDataSource $$ anonfun $ 4.apply(ResolvedDataSource.scala：62) . Try.orElse(Try.scala：82)at org.apache.spark.sql.execution.datasources.ResolvedDataSource $ .lookupDataSource(ResolvedDataSource.scala：62)... 50更多

我试图寻找可能缺少的火花包，但找不到任何有用的解决方法 .

我尝试使用Pyspark的类似代码，但它失败了类似的json数据源ClassNotFoundException .

在进一步尝试将现有RDD转换为JsonRDD后，我能够成功获得结果 . 有什么我想念的吗？我在Scala-2.10.5上使用Spark-1.6.1 . 任何帮助表示赞赏 . 谢谢

val stringRDD = sc.parallelize(Seq("""

{ "isActive": false,

"balance": "$1,431.73",

"picture": "http://placehold.it/32x32",

"age": 35,

"eyeColor": "blue"

}""",

"""{

"isActive": true,

"balance": "$2,515.60",

"picture": "http://placehold.it/32x32",

"age": 34,

"eyeColor": "blue"

}""",

"""{

"isActive": false,

"balance": "$3,765.29",

"picture": "http://placehold.it/32x32",

"age": 26,

"eyeColor": "blue"

}""")

)

sqlContext.jsonRDD(stringRDD).registerTempTable("testjson")

sqlContext.sql("SELECT age from testjson").collect