![](https://img-blog.csdnimg.cn/20201014180756928.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
Spark小练习
喜洋洋学习
这个作者很懒,什么都没留下…
展开
-
spark pom.xml配置
IDEA软件scala版本2.12.11pml.xml <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-library</artifactId> <version>2.12.4</version> </dependency> <dependency&原创 2022-02-11 09:25:10 · 1831 阅读 · 0 评论 -
Transformation操作
object Transformation {def main(args: Array[String]): Unit = {//权限被拒绝加下面代码System.setProperty("HADOOP_USER_NAME", "root")val conf = new SparkConf() .setAppName("WordCount2000") .setMaster("loc...原创 2020-01-11 20:47:53 · 894 阅读 · 0 评论 -
Actions触发操作
package Scalaimport org.apache.spark._import org.apache.spark.rdd.RDDobject Actions {def main(args: Array[String]): Unit = {val conf = new SparkConf() .setAppName("WordCount2000") .setMaster(...原创 2020-01-10 12:06:03 · 619 阅读 · 0 评论 -
spark单词统计以及排序
object RddT {//单词统计def main(args: Array[String]): Unit = {//权限被拒绝加下面代码System.setProperty(“HADOOP_USER_NAME”, “root”)// 实例化配置val conf = new SparkConf()// 指定 spark 任务名称conf.setAppName(“Spark Pi”...原创 2020-01-08 11:14:13 · 758 阅读 · 0 评论