![](https://img-blog.csdnimg.cn/20201014180756724.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
Spark
zajbetterme
这个作者很懒,什么都没留下…
展开
-
spark textFile基本语法
val sparkConf = new SparkConf().setAppName("test")val sc = new SparkContext(sparkConf)val cacheRDD = sc.textFile(path,2)textFile(path,minPartitions):path是String类型的一个URI,可以是HDFS和本地文件URI;minParti...原创 2020-01-07 17:06:49 · 569 阅读 · 0 评论 -
Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.4.2:testCompile
修改一下版本号<groupId>net.alchim31.maven</groupId><artifactId>scala-maven-plugin</artifactId><!--<version>3.4.2</version>--><version>3.2.0</version&g...转载 2019-12-30 16:32:42 · 1115 阅读 · 0 评论 -
value '$' is not a member of StringContext
解决方案:在获取sparkSession之后,添加import spark.implicits._原创 2019-10-08 16:37:03 · 1580 阅读 · 0 评论 -
与python2.7冲突,yum install 失败
1.参考https://blog.csdn.net/m0_37082962/article/details/80757364重点首先在http://rpm.pbone.net/下载各种rpm2.清除原有RHEL的YUM及相关软件包。参考https://blog.csdn.net/zhaosenoo/article/details/70919881rpm -qa | grep yu...转载 2019-04-17 20:06:47 · 1015 阅读 · 0 评论 -
IDEA通过SparkSql访问hive报错:Error while instantiating 'org.apache.spark.sql.hive.HiveSessionStateBuilder
Exception in thread "main" java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionStateBuilder': at org.apache.spark.sql.SparkSession$.org$apache$spark...原创 2019-03-25 20:37:51 · 6929 阅读 · 0 评论