pyspark
关注
分享
扫一扫
zhangztSky
这个作者很懒，什么都没留下…
展开
-
pysaprk踩坑之-Using RDD of dict to inferSchema is deprecated.Use pyspark.sql.Row instead
1.下图不能work的原因就是不能根据dict 推断schema上图传个string类型可以work，很精髓，我做的2.但是也不尽然，下图就可以，呵呵原创 2020-08-10 21:56:45 · 682 阅读 · 1 评论 -
pyspark之Cannot run program “....bin/python“: error=2, No such file or directory
配置python环境变量，yarn模式需要每台机器都要有python环境原创 2020-08-09 16:34:48 · 2704 阅读 · 1 评论 -
pyspark踩坑记之rg.apache.spark.api.python.PythonAccumulatorV2([class java.lang.String, class java.lang.
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).Traceback (most recent call last): File "tfifd.py", line 24, in <module> .appName("TfIdfExample")\ File "/opt/module/software/miniconda3/envs/superse原创 2020-08-09 00:59:36 · 1395 阅读 · 1 评论