spark
jacklinping
这个作者很懒,什么都没留下…
展开
-
pyspark mongodb问题
问题1:root 用户运行pyspark 没问题hadoop 用户运行就报如下错df1 = spark.read.format("com.mongodb.spark.sql.DefaultSource").option("collection","devices").load()17/02/15 19:34:26 WARN MongoInferSchema: Field 'de...原创 2017-02-15 19:36:25 · 306 阅读 · 0 评论 -
spark-note
import org.apache.spark.sql.SQLContextimport org.apache.spark.{SparkConf, SparkContext}import org.apache.spark.sql.types._import spark.implicits._import org.apache.spark.sql.functions.udfi...原创 2017-02-25 10:57:57 · 240 阅读 · 0 评论